How to save more than one field at a time

How to use openEMS. Discussion on examples, tutorials etc

Moderator: thorsten

thorsten
Posts: 1438
Joined: Mon 27 Jun 2011, 12:26

Re: How to save more than one field at a time

Post by thorsten » Thu 29 Mar 2012, 17:33

Hi,

this looks all very strange...

fist off:

Code: Select all

Operator::CalcTimestep(): Using timestep algorithm by Andreas Rennings, Dissertation @ University Duisburg-Essen, 2008, pp. 76, eq. 4.77 ff.
Why do you see this line, it has been removed a long time ago???

What kind of processors are you using???
6 MC/s is very slow... on my Core-i7 I would expect something around 100MC/s and for such a small structure which fits into L2-Cache even much more....

And a speedup from 6 to 90 just using 4 cpu's is completely impossible... that's a speed up of 15. Normally something below 4 would be expected...

Can you attach the m or xml file you used??

regards
Thorsten

jam
Posts: 18
Joined: Wed 28 Mar 2012, 00:17

Re: How to save more than one field at a time

Post by jam » Thu 29 Mar 2012, 17:59

I was looking for timestep calculation at the time and found it around this line which I just uncommented.

This is the "xml" code for "Parallel_Plate_Waveguide.m" which was used. Note that both fields are requested and if I allowed file dumps, only one is dumped.

Code: Select all

<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<openEMS>
  <FDTD NumberOfTimesteps="10000" endCriteria="0" OverSampling="50">
    <Excitation Type="1" f0="10000000">
    </Excitation>
    <BoundaryCond xmin="PMC" xmax="PMC" ymin="PEC" ymax="PEC" zmin="MUR" zmax="MUR">
    </BoundaryCond>
    <MPI SplitN_Z="1">
    </MPI>
  </FDTD>
  <ContinuousStructure CoordSystem="0">
    <Properties>
      <Electrode Name="excitation">
        <Excitation Type="0" Excit_X="0" Excit_Y="1" Excit_Z="0">
        </Excitation>
        <Primitives>
          <Box Priority="0">
            <P1 X="-10" Y="-10" Z="0">
            </P1>
            <P2 X="10" Y="10" Z="0">
            </P2>
          </Box>
        </Primitives>
      </Electrode>
      <DumpBox Name="Et" DumpMode="0" FileType="0">
        <Primitives>
          <Box Priority="0">
            <P1 X="-10" Y="0" Z="-10">
            </P1>
            <P2 X="10" Y="0" Z="30">
            </P2>
          </Box>
        </Primitives>
      </DumpBox>
      <DumpBox Name="Ht" DumpMode="1" FileType="0">
        <Primitives>
          <Box Priority="0">
            <P1 X="-10" Y="0" Z="-10">
            </P1>
            <P2 X="10" Y="0" Z="30">
            </P2>
          </Box>
        </Primitives>
      </DumpBox>
    </Properties>
    <RectilinearGrid DeltaUnit="1">
      <XLines>-10,-9,-8,-7,-6,-5,-4,-3,-2,-1,0,1,2,3,4,5,6,7,8,9,10</XLines>
      <YLines>-10,-9,-8,-7,-6,-5,-4,-3,-2,-1,0,1,2,3,4,5,6,7,8,9,10</YLines>
      <ZLines>-10,-9,-8,-7,-6,-5,-4,-3,-2,-1,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30</ZLines>
    </RectilinearGrid>
  </ContinuousStructure>
</openEMS>
I just compiled with vtk 5.4 and I still don't get two fields dumps. i.e same as with vtk 5.8.

And this is the my CPU

0.146080] CPU0: AMD Athlon(tm) II X4 640 Processor stepping 03
[ 0.150000] Booting processor 1 APIC 0x1 ip 0x6000
[ 0.030000] Initializing CPU#1
[ 0.030000] CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
[ 0.030000] CPU: L2 Cache: 512K (64 bytes/line)
[ 0.030000] CPU 1/0x1 -> Node 0
[ 0.030000] CPU: Physical Processor ID: 0
[ 0.030000] CPU: Processor Core ID: 1

thorsten
Posts: 1438
Joined: Mon 27 Jun 2011, 12:26

Re: How to save more than one field at a time

Post by thorsten » Thu 29 Mar 2012, 18:10

Hi,

I really have now idea why it is not dumping more than one file.

I just ran your xml file. I see around 10000 files named Et_xxxxxxx.vtz and as many Ht_xxxxxxx.vtr

By the way, with dumps enabled I see a speed around 6MC/s with disabled dumps >60MC/s (with a Core2 Duo 2.66GHz)

I still don't understand why MPI is faster for you... the overhead for this small simulation should kill any speed-up....

regards
Thorsten

jam
Posts: 18
Joined: Wed 28 Mar 2012, 00:17

Re: How to save more than one field at a time

Post by jam » Thu 29 Mar 2012, 18:21

Thank you for checking the dump files.
I am on Ubuntu 10.04

It works for you but not for me :(

thorsten
Posts: 1438
Joined: Mon 27 Jun 2011, 12:26

Re: How to save more than one field at a time

Post by thorsten » Thu 29 Mar 2012, 18:26

This means you have 10k Et_xxxxxx.vtr files but no Ht_xxxxxx.vtr???

Are you using MPI currently? Try without MPI...

jam
Posts: 18
Joined: Wed 28 Mar 2012, 00:17

Re: How to save more than one field at a time

Post by jam » Thu 29 Mar 2012, 18:37

I have tried both; with and wihtout MPI.

Yes, either Et_xxxxxx.vtr or Ht_xxxxxx.vtr depending on which comes first in the file.

thorsten
Posts: 1438
Joined: Mon 27 Jun 2011, 12:26

Re: How to save more than one field at a time

Post by thorsten » Thu 29 Mar 2012, 18:56

Please run openEMS with "-vvv" option to get the full debug infos... and post it here please

jam
Posts: 18
Joined: Wed 28 Mar 2012, 00:17

Re: How to save more than one field at a time

Post by jam » Thu 29 Mar 2012, 19:09

New xml file

Code: Select all

<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<openEMS>
  <FDTD NumberOfTimesteps="100" endCriteria="0" >
    <Excitation Type="1" f0="10000000">
    </Excitation>
    <BoundaryCond xmin="PMC" xmax="PMC" ymin="PEC" ymax="PEC" zmin="MUR" zmax="MUR">
    </BoundaryCond>
  </FDTD>
  <ContinuousStructure CoordSystem="0">
    <Properties>
      <Electrode Name="excitation">
        <Excitation Type="0" Excit_X="0" Excit_Y="1" Excit_Z="0">
        </Excitation>
        <Primitives>
          <Box Priority="0">
            <P1 X="-10" Y="-10" Z="0">
            </P1>
            <P2 X="10" Y="10" Z="0">
            </P2>
          </Box>
        </Primitives>
      </Electrode>
      <DumpBox Name="Et" DumpMode="0" FileType="0">
        <Primitives>
          <Box Priority="0">
            <P1 X="-10" Y="0" Z="-10">
            </P1>
            <P2 X="10" Y="0" Z="30">
            </P2>
          </Box>
        </Primitives>
      </DumpBox>
      <DumpBox Name="Ht" DumpMode="1" FileType="0">
        <Primitives>
          <Box Priority="0">
            <P1 X="-10" Y="0" Z="-10">
            </P1>
            <P2 X="10" Y="0" Z="30">
            </P2>
          </Box>
        </Primitives>
      </DumpBox>
    </Properties>
    <RectilinearGrid DeltaUnit="1">
      <XLines>-10,-9,-8,-7,-6,-5,-4,-3,-2,-1,0,1,2,3,4,5,6,7,8,9,10</XLines>
      <YLines>-10,-9,-8,-7,-6,-5,-4,-3,-2,-1,0,1,2,3,4,5,6,7,8,9,10</YLines>
      <ZLines>-10,-9,-8,-7,-6,-5,-4,-3,-2,-1,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30</ZLines>
    </RectilinearGrid>
  </ContinuousStructure>
</openEMS>
Run with -vvv


Code: Select all

 ---------------------------------------------------------------------- 
../../../openEMS_no_MPI tmp.xml -vvv

 | openEMS 64bit -- version 0.0.27
 | (C) 2010-2012 Thorsten Liebig <thorsten.liebig@gmx.de>  GPL license
 ---------------------------------------------------------------------- 
	Used external libraries:
		CSXCAD -- Version: 0.1.2a
		hdf5   -- Version: 1.8.4
		          compiled against: HDF5 library version: 1.8.4
		tinyxml -- compiled against: 2.6.2
		fparser
		boost  -- compiled against: 1_40
		vtk -- Version: 5.8.0
		       compiled against: 5.8.0

openEMS - verbose level 3
Read openEMS xml file: tmp.xml ...
Read openEMS Settings...
Read Geometry...
-----------------------------------------
 Property #0 Type: "Electrode" Name: "excitation"
 Primitive Count 	: 1
  -- Primitives: --
  Primitive #0 Type: "Box" Priority: 0
  Start: -10,-10,0
  Stop : 10,10,0
 --- Electrode Properties --- 
  Active directions: 1,1,1
  Excitation	: 0, 1, 0
  Weighting	: 1, 1, 1
  Delay		: 0
-----------------------------------------
 Property #1 Type: "Dump-Box" Name: "Et"
 Primitive Count 	: 1
  -- Primitives: --
  Primitive #1 Type: "Box" Priority: 0
  Start: -10,0,-10
  Stop : 10,0,30
 --- Dump Properties --- 
  DumpType: 0  DumpMode: 0 FileType: 0
-----------------------------------------
 Property #2 Type: "Dump-Box" Name: "Ht"
 Primitive Count 	: 1
  -- Primitives: --
  Primitive #2 Type: "Box" Priority: 0
  Start: -10,0,-10
  Stop : 10,0,30
 --- Dump Properties --- 
  DumpType: 0  DumpMode: 1 FileType: 0
Create FDTD operator (compressed SSE + multi-threading)
Multithreaded operator using 4 threads.
Operator::CalcTimestep_Var3: Smallest timestep (1.92583e-09s) found at position: 0 : 0;0;0
Compressing the FDTD operator... this may take a while...
------- Stat: FDTD Operator -------
Dimensions		: 21x21x41 = 18081 Cells (0.018081 MCells)
Size of Operator	: 867888 Byte (0.827682 MiB) 
Size of Field-Data	: 433944 Byte (0.413841 MiB) 
-----------------------------------
Number of PEC edges	: 0
in x direction		: 0
in y direction		: 0
in z direction		: 0
-----------------------------------
Timestep (s)		: 1.92583e-09
Timestep method name	: Rennings_2
Nyquist criteria (TS)	: 25
Nyquist criteria (s)	: 4.81458e-08
-----------------------------------
SSE compression enabled	: yes
Unique SSE operators	: 16
-----------------------------------
-----------------------------------
--- Mur ABC Extension ---
 Active direction	: z at line: 0
--- Mur ABC Extension ---
 Active direction	: z at line: 40
--- Excitation Extension ---
Voltage excitations	: 420	 (0, 420, 0)
Current excitations	: 0	 (0, 0, 0)
Excitation Length (TS)	: 100
Excitation Length (s)	: 1.92583e-07
-----------------------------------
Creation time for operator: 0.066143 s
FDTD simulation size: 21x21x41 --> 18081 FDTD cells 
FDTD timestep is: 1.92583e-09 s; Nyquist rate: 25 timesteps @1.03851e+07 Hz
Excitation signal length is: 100 timesteps (1.92583e-07s)
Max. number of timesteps: 100 ( --> 1 * Excitation signal length)
Create FDTD engine (compressed SSE + multi-threading)
---  Engine::SortExtensionByPriority() ---
 #0: Mur ABC Extension (0)
 #1: Mur ABC Extension (0)
 #2: Excitation Extension (-1000)
Multithreaded engine using 4 threads. Utilization: (6;5;5;5)
Setting up processing...
Running FDTD engine... this may take a while... grab a cup of coffee?!?
Time for 100 iterations with 18081 cells : 0.635854 sec
Speed: 2.84358 MCells/s 
ls

Code: Select all

et                 Et_0000000012.vtr  Et_0000000030.vtr  Et_0000000048.vtr  Et_0000000066.vtr  Et_0000000084.vtr  ht
Et_0000000000.vtr  Et_0000000018.vtr  Et_0000000036.vtr  Et_0000000054.vtr  Et_0000000072.vtr  Et_0000000090.vtr  
Et_0000000006.vtr  Et_0000000024.vtr  Et_0000000042.vtr  Et_0000000060.vtr  Et_0000000078.vtr  Et_0000000096.vtr  tmp.xml
NOTE : replace DumpMode by DumpType in xml file -> same result, only one field dump.
Last edited by jam on Thu 29 Mar 2012, 19:27, edited 2 times in total.

thorsten
Posts: 1438
Joined: Mon 27 Jun 2011, 12:26

Re: How to save more than one field at a time

Post by thorsten » Thu 29 Mar 2012, 19:20

By the way, you are dumping (or trying to) in both cases an electric field. DumpMode 1 means Node-Interpolation

What you want to do is 'DumpType' 1 --> refer to help AddDump

Please try option "--showProbeDiscretization" and post the result.

I tried your xml file. I see all vtr as expected... I have no idea what is going on... Sorry but I can't fix a problem I cannot reconstruct...

It seems you need to dump to hdf5 and use Matlab to convert it to vtk...
Or use the hdf5 output directly.

jam
Posts: 18
Joined: Wed 28 Mar 2012, 00:17

Re: How to save more than one field at a time

Post by jam » Thu 29 Mar 2012, 19:25

../../../openEMS_no_MPI tmp.xml --showProbeDiscretization
----------------------------------------------------------------------
| openEMS 64bit -- version 0.0.27
| (C) 2010-2012 Thorsten Liebig <thorsten.liebig@gmx.de> GPL license
----------------------------------------------------------------------
Used external libraries:
CSXCAD -- Version: 0.1.2a
hdf5 -- Version: 1.8.4
compiled against: HDF5 library version: 1.8.4
tinyxml -- compiled against: 2.6.2
fparser
boost -- compiled against: 1_40
vtk -- Version: 5.8.0
compiled against: 5.8.0

openEMS - showing probe discretization information
Create FDTD operator (compressed SSE + multi-threading)
FDTD simulation size: 21x21x41 --> 18081 FDTD cells
FDTD timestep is: 1.92583e-09 s; Nyquist rate: 25 timesteps @1.03851e+07 Hz
Excitation signal length is: 100 timesteps (1.92583e-07s)
Max. number of timesteps: 100 ( --> 1 * Excitation signal length)
Create FDTD engine (compressed SSE + multi-threading)
Et: snapped primary coords: (-10,0,-10) -> (10,0,30) [0,10,0] -> [20,10,40]
Ht: snapped dual coords: (-9.5,-0.5,-9.5) -> (10.5,-0.5,30.5) [0,9,0] -> [20,9,40]
Running FDTD engine... this may take a while... grab a cup of coffee?!?
Time for 100 iterations with 18081 cells : 1.7562 sec
Speed: 1.02955 MCells/s

Post Reply