How to save more than one field at a time

How to use openEMS. Discussion on examples, tutorials etc

Moderator: thorsten

jam
Posts: 18
Joined: Wed 28 Mar 2012, 00:17

How to save more than one field at a time

Post by jam » Wed 28 Mar 2012, 00:26

Hello,
Great software package.

I was playing with the tutorials and I found that I cannot save more than one field at a time per run even if I define many dumboxes for each field output.
For example, in the Coax.m example under waveguide, this is declared in the program:

Code: Select all

%% define dump boxes... %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
CSX = AddDump(CSX,'Et_','DumpMode',2);
start = [mesh.x(1) , 0 , mesh.z(1)];
stop = [mesh.x(end) , 0 , mesh.z(end)];
CSX = AddBox(CSX,'Et_',0 , start,stop);

CSX = AddDump(CSX,'Ht_','DumpType',1,'DumpMode',2);
CSX = AddBox(CSX,'Ht_',0,start,stop);

but, no outputs are provided for the Et_ and Ht_ fields. Sometimes, one field is save but never both at the same time. Is this a normal operating mode for the program where you have to run it twice to save both fields?

Youry
Posts: 51
Joined: Wed 22 Feb 2012, 12:50

Re: How to save more than one field at a time

Post by Youry » Wed 28 Mar 2012, 04:26

I think the mistake is here :

CSX = AddDump(CSX,'Ht_','DumpType',1,'DumpMode',2);

If you want magnetic field and electric current you should do (perhaps)

CSX = AddDump(CSX,'Ht_','DumpType',1)
CSX = AddDump(CSX,'jE_','DumpType',2)
CSX = AddBox(CSX,'Ht_',0,start,stop);
CSX = AddBox(CSX,'jE_',0,start,stop);

jam
Posts: 18
Joined: Wed 28 Mar 2012, 00:17

Re: How to save more than one field at a time

Post by jam » Wed 28 Mar 2012, 04:41

No.
This is just an example I give to show my point.
Just run this file and you will see that only ONE field is save. Using version 0.27
I want to be able to save more than one field data at a time into separate VTK files.

thorsten
Posts: 1438
Joined: Mon 27 Jun 2011, 12:26

Re: How to save more than one field at a time

Post by thorsten » Wed 28 Mar 2012, 09:40

Hi,

I dumpas many field data to files as you want and I don't see any problem in your initial code example. This should work...

Dumping as vtk files results in maybe hundreds of files, maybe you missed them? Sometimes the explorer is sorting them strangely...

Try hdf5 file type --> 'FileType',1 then you get only 2 files: Et_.h5 + Ht_.h5

btw.: You can omit the "_" sign. In earlier version you would get strange names without it. But now you don't need it any more.

I just saw that you were talking about an openEMS example. I just ran it and both types of dumps were there...

regards
Thorsten

Youry
Posts: 51
Joined: Wed 22 Feb 2012, 12:50

Re: How to save more than one field at a time

Post by Youry » Wed 28 Mar 2012, 10:04

I am sorry I didn't see difference in option between 'DumpType' and 'DumpMode'
Here : CSX = AddDump(CSX,'Ht_','DumpType',1,'DumpMode',2);
I believed you tried to store two field (DumpType) in same files... I readed somethings like : CSX = AddDump(CSX,'Ht_','DumpType',1,'DumpType',2);

So I have no idea, maybe you can forced the vtk files saving with 'FileType',0 option ?
like : CSX = AddDump(CSX,'Ht_','DumpType',1,'DumpMode',2,'FileType',0);
But it shouldn't do anything...

I try this in my own file...

Code: Select all

CSX = AddDump(CSX,'E1','DumpType',0);
CSX = AddBox(CSX,'E1',0,[0 0 15],[200 200 15]);
CSX = AddDump(CSX,'H1','DumpType',1);
CSX = AddBox(CSX,'H1',0,[0 0 15],[200 200 15]);
And it works...
And this :

Code: Select all

CSX = AddDump(CSX,'E1','DumpType',0,'DumpMode',2);
CSX = AddBox(CSX,'E1',0,[0 0 15],[200 200 15]);
CSX = AddDump(CSX,'H1','DumpType',1,'DumpMode',2);
CSX = AddBox(CSX,'H1',0,[0 0 15],[200 200 15]);
Also works...

So you should wait for openEMS master answer.

Edit : Crossing with the master

jam
Posts: 18
Joined: Wed 28 Mar 2012, 00:17

Re: How to save more than one field at a time

Post by jam » Wed 28 Mar 2012, 16:09

Thank you for your prompt answers.
Indeed , the example works for 'FileType"1 and I get both files int the directory.
But for 'FileType,0 not vtk files are to be found.

I think I know where the problem is. This is my header message

Code: Select all

 ---------------------------------------------------------------------- 
 | openEMS 64bit -- version 0.0.27
 | (C) 2010-2012 Thorsten Liebig <thorsten.liebig@gmx.de>  GPL license
 ---------------------------------------------------------------------- 
	Used external libraries:
		CSXCAD -- Version: 0.1.2a
		hdf5   -- Version: 1.8.4
		          compiled against: HDF5 library version: 1.8.4
		tinyxml -- compiled against: 2.6.2
		fparser
		boost  -- compiled against: 1_40
		vtk -- Version: 5.8.0
		       compiled against: 5.8.0
		MPI -- Version: 2.1
		       compiled against: openMPI1.5.5
the vtk version could be too high but I need the MPI version.

thorsten
Posts: 1438
Joined: Mon 27 Jun 2011, 12:26

Re: How to save more than one field at a time

Post by thorsten » Wed 28 Mar 2012, 16:57

Hi,

I have not used version 5.8 of vtk yet. I guess you needed to fix the .pro files to compile openEMS?
But so far I think this should be no problem... I know of a couple of people that use vtk 5.8 and so far they reported no problems...

Have you tried some of the Tutorial examples having vtk dumps?

Regarding MPI:
Please be warned that the MPI functionality is highly untested... I even encountered some problems I was not yet able to fix...

Are you sure you really need MPI? FDTD is very memory efficient and to get a real speed-up by using MPI you need a very special simulation configuration...


regards
Thorsten

jam
Posts: 18
Joined: Wed 28 Mar 2012, 00:17

Re: How to save more than one field at a time

Post by jam » Wed 28 Mar 2012, 18:14

BTW, I am using octave. This should not have any detrimental effects on my problem.
In both Tutorials and Examples directories, I have the same results as far as not being able to dump more than one vtk(vtr) files at a time.

Concerning MPI, it took me a while to get it to work but I found the trick.
To compile, just run qmake on the MPI.pro file, and make.

At first, I do a run using octave myprog.m , then I cd to the tmp folder and run : mpirun -np 4 ../../../../openEMS xxxxx.xml

I let the octave run until the first steps are printed, then ctrl_C to stop the job and complete as shown.

Each processor will produce a series of IDX_timed_datafile.vtr where X is 1 2 3 4 in my case. This helps a lot when visualizing the results with paraview.

thorsten
Posts: 1438
Joined: Mon 27 Jun 2011, 12:26

Re: How to save more than one field at a time

Post by thorsten » Wed 28 Mar 2012, 20:55

Hi,

there is an easier way to use MPI, using it directly from Octave.

Have a look at the matlab function: "RunOpenEMS_MPI"

It should work like this (see help of RunOpenEMS_MPI and SetupMPI):

Code: Select all

[..]
% example, split the FDTD mesh in 2 equal parts in x-direction
% and split the FDTD mesh in 3 parts in z-direction, split at z=-500 and z=500
% this will need a Settings.MPI.NrProc of 2*3=6
FDTD = SetupMPI(FDTD,'SplitN_X',2 ,'SplitPos_Z', '-500,500');

[..]
% mpi binary path on all nodes needed
Settings.MPI.Binary = '/opt/openEMS/openEMS';
% number of processes to run
Settings.MPI.NrProc = 3;
% define the mpi hosts :
Settings.MPI.Hosts = {'host1','host2','host3'};
 
RunOpenEMS(Sim_Path, Sim_File, NrProc, opts, Settings)
If you find some problems etc. I would be happy if you would sent me some patches fixing it...
As I mentioned, it is poorly tested...

Do you see a real speed-up using 4 cpu's?? Do you really need MPI? Why?
I implemented it just out of curiosity, but never really needed it, since the FDTD method is so memory and CPU efficient...

regards
Thorsten

jam
Posts: 18
Joined: Wed 28 Mar 2012, 00:17

Re: How to save more than one field at a time

Post by jam » Thu 29 Mar 2012, 13:53

Test #1 Compile with MPI but run without MPI

Code: Select all

../../Tutorials/tmp$ ../../../openEMS tmp.xml --disable-dumps
 ---------------------------------------------------------------------- 
 | openEMS 64bit -- version 0.0.27
 | (C) 2010-2012 Thorsten Liebig <thorsten.liebig@gmx.de>  GPL license
 ---------------------------------------------------------------------- 
	Used external libraries:
		CSXCAD -- Version: 0.1.2a
		hdf5   -- Version: 1.8.4
		          compiled against: HDF5 library version: 1.8.4
		tinyxml -- compiled against: 2.6.2
		fparser
		boost  -- compiled against: 1_40
		vtk -- Version: 5.8.0
		       compiled against: 5.8.0
		MPI -- Version: 2.1
		       compiled against: openMPI1.5.5

openEMS - disabling all field dumps
Create FDTD operator (compressed SSE + multi-threading)
Operator::CalcTimestep(): Using timestep algorithm by Andreas Rennings, Dissertation @ University Duisburg-Essen, 2008, pp. 76, eq. 4.77 ff.
FDTD simulation size: 21x21x41 --> 18081 FDTD cells 
FDTD timestep is: 1.92583e-09 s; Nyquist rate: 25 timesteps @1.03851e+07 Hz
Excitation signal length is: 10000 timesteps (1.92583e-05s)
Max. number of timesteps: 10000 ( --> 1 * Excitation signal length)
Create FDTD engine (compressed SSE + multi-threading)
Running FDTD engine... this may take a while... grab a cup of coffee?!?
[@        4s] Timestep:         1350 || Speed:    6.0 MC/s (3.018e-03 s/TS) || Energy: ~1.10e-07 (- 0.00dB)
[@        8s] Timestep:         2700 || Speed:    6.0 MC/s (3.010e-03 s/TS) || Energy: ~1.11e-07 (- 0.00dB)
[@       12s] Timestep:         4025 || Speed:    6.0 MC/s (3.030e-03 s/TS) || Energy: ~1.09e-07 (- 0.06dB)
[@       16s] Timestep:         5375 || Speed:    6.0 MC/s (2.996e-03 s/TS) || Energy: ~1.09e-07 (- 0.06dB)
[@       20s] Timestep:         6725 || Speed:    6.0 MC/s (3.007e-03 s/TS) || Energy: ~1.09e-07 (- 0.05dB)
[@       24s] Timestep:         8050 || Speed:    6.0 MC/s (3.032e-03 s/TS) || Energy: ~1.07e-07 (- 0.13dB)
[@       28s] Timestep:         9400 || Speed:    6.1 MC/s (2.976e-03 s/TS) || Energy: ~1.08e-07 (- 0.12dB)
Time for 10000 iterations with 18081.00 cells : 30.11 sec
Speed: 6.01 MCells/s 
Test #2 Compile without MPI

Code: Select all

../../Tutorials/tmp$ ../../../openEMS_no_MPI tmp.xml --disable-dumps
 ---------------------------------------------------------------------- 
 | openEMS 64bit -- version 0.0.27
 | (C) 2010-2012 Thorsten Liebig <thorsten.liebig@gmx.de>  GPL license
 ---------------------------------------------------------------------- 
	Used external libraries:
		CSXCAD -- Version: 0.1.2a
		hdf5   -- Version: 1.8.4
		          compiled against: HDF5 library version: 1.8.4
		tinyxml -- compiled against: 2.6.2
		fparser
		boost  -- compiled against: 1_40
		vtk -- Version: 5.8.0
		       compiled against: 5.8.0

openEMS - disabling all field dumps
Create FDTD operator (compressed SSE + multi-threading)
FDTD simulation size: 21x21x41 --> 18081 FDTD cells 
FDTD timestep is: 1.92583e-09 s; Nyquist rate: 25 timesteps @1.03851e+07 Hz
Excitation signal length is: 10000 timesteps (1.92583e-05s)
Max. number of timesteps: 10000 ( --> 1 * Excitation signal length)
Create FDTD engine (compressed SSE + multi-threading)
Running FDTD engine... this may take a while... grab a cup of coffee?!?
[@        4s] Timestep:         1375 || Speed:    6.2 MC/s (2.923e-03 s/TS) || Energy: ~1.12e-07 (- 0.00dB)
[@        8s] Timestep:         2775 || Speed:    6.2 MC/s (2.899e-03 s/TS) || Energy: ~1.13e-07 (- 0.00dB)
[@       12s] Timestep:         4150 || Speed:    6.2 MC/s (2.933e-03 s/TS) || Energy: ~1.13e-07 (- 0.01dB)
[@       16s] Timestep:         5550 || Speed:    6.2 MC/s (2.896e-03 s/TS) || Energy: ~1.11e-07 (- 0.09dB)
[@       20s] Timestep:         6925 || Speed:    6.2 MC/s (2.936e-03 s/TS) || Energy: ~1.09e-07 (- 0.16dB)
[@       24s] Timestep:         8300 || Speed:    6.2 MC/s (2.918e-03 s/TS) || Energy: ~1.07e-07 (- 0.24dB)
[@       28s] Timestep:         9675 || Speed:    6.2 MC/s (2.918e-03 s/TS) || Energy: ~1.05e-07 (- 0.34dB)
Time for 10000 iterations with 18081.00 cells : 29.18 sec
Speed: 6.20 MCells/s 
Test #3 Compile with MPI and run with MPI

Code: Select all

../../Tutorials/tmp$ mpirun -np 4 ../../../openEMS tmp.xml --disable-dumps
 ---------------------------------------------------------------------- 
 | openEMS 64bit -- version 0.0.27
 | (C) 2010-2012 Thorsten Liebig <thorsten.liebig@gmx.de>  GPL license
 ---------------------------------------------------------------------- 
	Used external libraries:
		CSXCAD -- Version: 0.1.2a
		hdf5   -- Version: 1.8.4
		          compiled against: HDF5 library version: 1.8.4
		tinyxml -- compiled against: 2.6.2
		fparser
		boost  -- compiled against: 1_40
		vtk -- Version: 5.8.0
		       compiled against: 5.8.0
		MPI -- Version: 2.1
		       compiled against: openMPI1.5.5

openEMS - disabling all field dumps
Create FDTD operator (compressed SSE + multi-threading)
Operator::CalcTimestep(): Using timestep algorithm by Andreas Rennings, Dissertation @ University Duisburg-Essen, 2008, pp. 76, eq. 4.77 ff.
FDTD simulation size: 21x21x11 --> 5292 FDTD cells 
FDTD timestep is: 1.92583e-09 s; Nyquist rate: 25 timesteps @1.03851e+07 Hz
Excitation signal length is: 10000 timesteps (1.92583e-05s)
Max. number of timesteps: 10000 ( --> 1 * Excitation signal length)
Create FDTD engine (compressed SSE + multi-threading)
Running MPI-FDTD engine... this may take a while... grab a cup of coffee?!?
Time for 10000 iterations with 5292 cells : 2.25427 sec
Speed: 91.9457 MCells/s 
To compare the calculation part only, --disable-dumps is active for all tests. As can be seen, a lot of speed increase is acheived with MPI (4 procs).

However, I saw a strange behaviour with nprocs = 2 or 3, the speed scaling is far from linear. Normally, with np=2, I would see 2 procs fully occupied and 2 with very low usage. Here, I see that all 4 processors are averaging 50% load with np=2 .

Post Reply