Mfe simulation data management
1 / 12

MFE Simulation Data Management - PowerPoint PPT Presentation

  • Uploaded on

MFE Simulation Data Management. SLAC DMW 2004 March 16, 2004 W. W. Lee and S. Klasky Princeton Plasma Physics Laboratory Princeton, NJ. atomic mfp. electron-ion mfp. system size. skin depth. tearing length. ion gyroradius. Debye length. electron gyroradius. Spatial Scales (m). 10 -6.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'MFE Simulation Data Management' - lucius

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Mfe simulation data management

MFE Simulation Data Management


March 16, 2004

W. W. Lee and S. Klasky

Princeton Plasma Physics Laboratory

Princeton, NJ

Mfe simulation data management

atomic mfp

electron-ion mfp

system size

skin depth

tearing length

ion gyroradius

Debye length

electron gyroradius

Spatial Scales (m)






pulse length

current diffusion

Inverse ion plasma frequency

inverse electron plasma frequency


ion gyroperiod

Ion collision

electron gyroperiod

electron collision





Temporal Scales (s)

Spatial & Temporal Scales Present Major Challenge to Theory & Simulations

  • • Huge range of spatial and temporal scales.

  • • Overlap in scales often means strong (simplified) ordering not possible

  • Different codes/theory for different scales.

  • 5+years: Integration of physics into Fusion Simulation Project

Plasma turbulence simulation
Plasma Turbulence Simulation

• Gyrokinetic Particle-In-Cell Simulation

-- Reduced Vlasov-Maxwell Equations

• Simulations on MPP Platforms

-- Cray T3E & IBM SP (NERSC), Cray-X1 (ORNL),

SX6 (Earth Simulator, Japan)

• Simulation of Burning Plasmas

-- International Tokamak Experimental Reactor (ITER)

• Integrated Fusion Simulation Project (MFE)

• Visualization -- turbulence evolution & particle orbits

Gyrokinetic approximation
Gyrokinetic Approximation

  • Gyromotion

  • Polarization provides quasineutrality

[W. W. Lee, PF ‘83; JCP ‘87]

Mfe simulation data management

Earth Simulator




Mfe simulation data management

Ion Temperature Gradient Driven Turbulence

Particle Trajectories

Electrostatic Potential

Data management challenges
Data Management challenges

  • GTC is producing TBs of data

    • Data rates: 80Mbs now, 1.6Gbs 5 years.

    • Need QOS to stream data.

  • This data needs to be post-processed

    • Essential to parallelize the post-processing routines to handle our larger datasets.

    • We need a cluster to post process this data.

      • M (supercomputer processors) x N (cluster processors) problem.

      • QOS becomes more important to sustain this post-processing.

  • The post-processed data needs to be shared among collaborators

    • Different sections of the post-processed data may go to different users .

    • Post-processed data, along with other metadata should be archived into a relational database.

Post processing of gtc data
Post processing of GTC Data.

  • Particle Data

    • No compression possible.

    • Sent to 1 cluster for visualization/analysis.

    • Work being done with K. Ma, U.C. Davis: Visualize a million particles.

    • Gain new insights into the theory.

  • Field Data

    • Geometric/Temporal compression of the data is possible.

    • Data needs to be streamed to a local cluster at PPPL.

    • Reduced subset needs to be sent to PPPL + collaborators.

      • Use Logistic Network. [Beck, UT-K]

      • Data transfer needs to be automatic, and integrated into a dataflow/webflow for use with parallel analysis routines.

    • We desire to see post-processed data during the simulation.

After the analysis
After the analysis

  • Post-processed data needs to be saved into a relational database

    • How do we query this abstract data to compare it with experiments?

    • 3D correlation functions

    • Processing of TBs of data/run now, 100’s of TBs of data/run in 5 years.

    • Data mining techniques will be necessary to understand this data.