1 / 12

MFE Simulation Data Management

MFE Simulation Data Management. SLAC DMW 2004 March 16, 2004 W. W. Lee and S. Klasky Princeton Plasma Physics Laboratory Princeton, NJ. atomic mfp. electron-ion mfp. system size. skin depth. tearing length. ion gyroradius. Debye length. electron gyroradius. Spatial Scales (m). 10 -6.

lucius
Download Presentation

MFE Simulation Data Management

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MFE Simulation Data Management SLAC DMW 2004 March 16, 2004 W. W. Lee and S. Klasky Princeton Plasma Physics Laboratory Princeton, NJ

  2. atomic mfp electron-ion mfp system size skin depth tearing length ion gyroradius Debye length electron gyroradius Spatial Scales (m) 10-6 10-2 10-4 100 102 pulse length current diffusion Inverse ion plasma frequency inverse electron plasma frequency confinement ion gyroperiod Ion collision electron gyroperiod electron collision 105 10-10 100 10-5 Temporal Scales (s) Spatial & Temporal Scales Present Major Challenge to Theory & Simulations • • Huge range of spatial and temporal scales. • • Overlap in scales often means strong (simplified) ordering not possible • Different codes/theory for different scales. • 5+years: Integration of physics into Fusion Simulation Project

  3. Major Fusion Codes

  4. Data Rates of Major Fusion Codes

  5. Plasma Turbulence Simulation • Gyrokinetic Particle-In-Cell Simulation -- Reduced Vlasov-Maxwell Equations • Simulations on MPP Platforms -- Cray T3E & IBM SP (NERSC), Cray-X1 (ORNL), SX6 (Earth Simulator, Japan) • Simulation of Burning Plasmas -- International Tokamak Experimental Reactor (ITER) • Integrated Fusion Simulation Project (MFE) • Visualization -- turbulence evolution & particle orbits

  6. Gyrokinetic Approximation • Gyromotion • Polarization provides quasineutrality [W. W. Lee, PF ‘83; JCP ‘87]

  7. Earth Simulator 18% 10 (Ethier)

  8. Ion Temperature Gradient Driven Turbulence Particle Trajectories Electrostatic Potential

  9. Data Management challenges • GTC is producing TBs of data • Data rates: 80Mbs now, 1.6Gbs 5 years. • Need QOS to stream data. • This data needs to be post-processed • Essential to parallelize the post-processing routines to handle our larger datasets. • We need a cluster to post process this data. • M (supercomputer processors) x N (cluster processors) problem. • QOS becomes more important to sustain this post-processing. • The post-processed data needs to be shared among collaborators • Different sections of the post-processed data may go to different users . • Post-processed data, along with other metadata should be archived into a relational database.

  10. Post processing of GTC Data. • Particle Data • No compression possible. • Sent to 1 cluster for visualization/analysis. • Work being done with K. Ma, U.C. Davis: Visualize a million particles. • Gain new insights into the theory. • Field Data • Geometric/Temporal compression of the data is possible. • Data needs to be streamed to a local cluster at PPPL. • Reduced subset needs to be sent to PPPL + collaborators. • Use Logistic Network. [Beck, UT-K] • Data transfer needs to be automatic, and integrated into a dataflow/webflow for use with parallel analysis routines. • We desire to see post-processed data during the simulation.

  11. After the analysis • Post-processed data needs to be saved into a relational database • How do we query this abstract data to compare it with experiments? • 3D correlation functions • Processing of TBs of data/run now, 100’s of TBs of data/run in 5 years. • Data mining techniques will be necessary to understand this data.

More Related