1 / 27

Outline

Numerical simulations of astrophysical plasmas : status and perspectives of the Saclay/DAPNIA software project E. Audit, D. Pomar è de , R. Teyssier, B. Thooris CEA/DAPNIA - Saclay. Outline. Overview of numerical simulations in Saclay Cosmological structures formation : the RAMSES code

caryn-salas
Download Presentation

Outline

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Numerical simulations of astrophysical plasmas :status and perspectives of the Saclay/DAPNIA software projectE. Audit, D. Pomarède, R. Teyssier, B. ThoorisCEA/DAPNIA - Saclay

  2. Outline • Overview of numerical simulations in Saclay • Cosmological structures formation : the RAMSES code • Dynamics of the interstellar medium : the HERACLES code • Numerical simulations software project • data handling • visualization • Perspectives

  3. Numerical simulations in Saclay • An extensive program of simulations of astrophysical plasmas is conducted at the Saclay/DAPNIA Laboratory • It covers a vast spectrum of challenging problems at various scales and has its foundations in a suite of independent numerical codes : COSMOLOGICAL STRUCTURES FORMATION RAMSES STELLAR EVOLUTION ASH DYNAMICS OF THE ISM HERACLES FORMATION OF PROTO-PLANETARY DISKS FARGO

  4. Numerical simulations in Saclay : overview • Cosmological structures • The formation of the large structures in the Universe • Interplay of the dark matter component and the baryon gas • RAMSES code (R. Teyssier) : N-body and hydrodynamical simulations • The Interstellar Medium • The formation and dynamics of molecular clouds • Radiation Hydrodynamics • HERACLES code (E. Audit et al.) : radiative transfer coupled to hydrodynamics • Stellar evolution • MHD of the Sun • Participation (A.S. Brun) to ASH developments (J. Toomre et al.) • Proto-planetary systems • Study of the disk-planet tidal interactions • FARGO code (F. Masset) : 2D & 3D hydrodynamics

  5. Numerical simulations in Saclay : overview • Computing resources • CCRT (CEA Supercomputing Center) : • HP/OSF1 256 quadri-processors alpha nodes • Linux 128 quadri-processors AMD Opteron • “Horizon Project” resources (cosmology) • IDRIS (CNRS Supercomputing Center) • 1024 processors Power4 cluster • MareNostrum (Barcelona Supercomputing Center) • 2406 dual 64-bit processor nodes @ 2.2GHz, ~42 TeraFlops • the most powerful supercomputer in Europe • Software Project • Developments are now addressed in the context of a software engineering project aiming at the optimization and sharing of expertise for various components : • Algorithms parallelization/optimization • Data Handling • Database • Post-treatment • Visualization

  6. The RAMSES code • A hybrid simulation code • N-body treatment to solve the Dark Matter (DM) dynamics • Hydrodynamical treatment of the baryonic component • Position of the problem : • Objective is to study the structure formation in the Universe with high spatial resolution • DM is believed to be the dominant component in mass of the cosmological density field, with only a small fraction ~10% in baryons • At intermediate scales such as galaxy clusters DM stills dominates but a gaseous component is introduced (constraints from observations of hot intracluster medium) • At smaller scales, gas-cooling and fluid dynamics play a dominant role in the structure of galaxy-size objects • at 1st order, baryons = hydrostatic ionized plasma trapped in DM gravitational potential wells • complex hydrodynamical processes require accurate treatment : • shock heating • atomic radiation cooling • star formation

  7. The RAMSES code • Resolution requirements • a volume of 100 Mpch-1 side with haloes simulated with 10 resolution elements requires a spatial resolution of 10 kpc h-1 dynamical range of 104 • 100 particles/galaxy 2563 particles in total • Algorithms • F90 parallelized with MPI or OpenMP • Based entirely on Adaptive Mesh Refinement (AMR) technique • Tree based data structure allowing recursive refinements on a cell-by-cell basis • DM particles considered as a collisionless N-body system described by the Vlasov-Poisson equations • Hydrodynamical solver based on a 2nd-order Godunov method • With 4.1107 cells a formal resolution of 81923 is reached at the 14th Level • Ref : R. Teyssier, A&A 385, 337-364 (2002)

  8. The RAMSES AMR level 2 level 3 level 5 level 9 level 11 level 14 basic element of AMR structure : group of 2dim sibling cells called “octs”

  9. The RAMSES AMR level 9 to level 14 highest granularity reach 213 =8192 cells in each direction

  10. RAMSES simulations of cosmological structures Recent results obtained with RAMSES : “The History of the Baryon Budget – Cosmic Logistics in a Hierarchical Universe”, Y. Rasera and R. Teyssier, accepted for publication in A&A, astro-ph/0505473 (23 May 2005) gas density zooming sequence : from large scale filamentary structures to spiral discs “Kinematic Dynamos using Constrained Transport with High Order Godunov Schemes and Adaptive Mesh Refinement”, R. Teyssier, S. Fromang, E. Dormy, accepted for publication in Journal of Computa-tional Physics, astro-ph/0601715 (31 Jan 06) extension to the modeling of magnetic field evolution (MHD)

  11. The HERACLES code • A 3-D RHD code written in F90 parallelized with MPI • Implemented in Cartesian, cylindrical, and spherical coordinates • In current implementation only regular mesh grids • Solves the equations of radiative transfer coupled to hydrodynamics : • fluid evolution is determined by the classical conservations equations (mass, momentum, energy) • + source terms characterizing the momentum and energy exchanges between the fluid and the radiation • The transfer equations are solved by a second order Godunov type method and integrated implicitly using iterative solvers • Reference : “HERACLES: a three dimensional radiation hydrodynamics code” ,M. González, E. Audit, and P. Huynh, to be published

  12. HERACLES - simulations of ISM dynamics • Current studies focus on the thermal fragmentation of turbulent flows of interstellar hydrogen • neutral atomic hydrogen (HI) represents more than 50% of the ISM • molecular clouds form through the condensation of a warm neutral phase (WNM) into a cold medium (CNM) • Thermal processes : • heating : photo-electric effect on small grains and poly-aromatic hydrocarbons due to the far-ultraviolet galactic radiation • cooling by molecules and various processes • Initial conditions : • collision between two turbulent streams (converging flow) of WNM triggers the formation of CNM structures • Resolution requirements : 4 different spatial scales are involved • cooling length of the WNM typically ~ 10-20 pc • typical size of CNM fragments ~ 0.1 pc • conduction length in the 10-1-10-3 pc range • size of the shocked layer ~ 10-3 pc dynamic range of 104

  13. HERACLES - simulations of ISM dynamics • First studies conducted with 2D simulations weakly turbulent forcing very turbulent forcing Fragments of cold, high-density gas connected by thin layers of low-density gas. Sharp thermal fronts bound the structures and connect them to the warm surrounding medium. Complex density field. The different phases are highly interwoven with pockets of warm gas embedded in filaments of cooler gas. • Analysis of structure morphology, internal velocity dispersion, fraction of gas in the various states in E. Audit and P. Hennebelle, A&A 443,1-13 (2005)

  14. New, high resolution 3D simulations density field volume of size 15 pc grid 12003

  15. HERACLES simulations of ISM dynamics 2D slice the limited dynamical range leads to under-estimation of the highest density reached during supersonic collisions

  16. Numerical simulations software project • Software developments are managed in the context of an indepen-dent engineering project. The objective is to provide a core of software modules useable by the various simulations tools. This includes data handling, post-treatment, visualization, parallelization optimization,… RAMSES CORE SOFTWARE MODULES I/O, Graphics, Algorithms ASH HERACLES FARGO

  17. Data Handling • A unique format has been chosen for the data produced by all the simulation codes : HDF5, the Hierarchical Data Format developed by the NCSA National Center for Supercomputing Applications. • HDF5 is a general purpose library and file format for storing scientific data • Two primary objects are handled : • Datasets = multidimensional array of data elements • Groups = structures • Efficient storage and I/O : the library is tuned and adapted to read and write data efficiently on parallel computing systems • Free, open source software, including utilities (browser) • Plenty of applications have interfaces to HDF5, in particular in the graphics/analysis area : • IDL, ParaView, MATLAB, …

  18. browser of HDF5 files

  19. Visualization and browsing of HDF5 files in IDL

  20. Visualization • Visualization is performed in the framework of IDL • Baseline is to use the object-oriented programming offered by IDL’s Object Graphics • The visualization tool is a graphical widget • users act on the objects to modify their attributes through menus, droplist, sliders, buttons & dialog fields • Input data : scalar and vector fields on regular grids and on AMR tree, particles • Once the data are loaded in memory, various implementations of the visualization objects are proposed based on different classes of the IDL library : • IDLgrPolygon : set of polygons and vertices, to hold an iso-surface • IDLgrVolume : mapping from a 3D array of data to a 3D array of voxel colors projected to two dimensions • IDLgrPolyline : useful for hedgehog displays of vector fields • IDLgrSurface, IDLgrImage, IDLgrPlot, IDLgrSymbol • Basic graphics objects are inserted in Models that can be transformed (rotated, scaled and/or translated) : this forms the basis for spatial navigability around the 3D objects on display. • IDLgrLight objects represent sources of illumination : ambient, positional, directional, spotlight • Two methods of rendering objects : • hardware accelerator using OpenGL, highly efficient on a local machine • software rendering applies when running on a distant computer

  21. the graphical widget • Select : • density • pressure • velocity • radiative energy • radiative flux • particles • custom variable navigation surface object menus sliders buttons droplist dialog fields axis object profile plot object image object

  22. Visualization of regular grids and AMR data Max grid : regular 3D 1200x1200x1200 = 1.728 109 cells regular 2D 10000x10000 AMR up to level 10 projected in grid 1024x1024x1024 image of 2D 10000x10000 HERACLES simulation surface of a density slice on a restricted region of the RAMSES AMR (level 7 to level 11)

  23. Visualization of iso-surfaces Simulation of ISM hydrogen turbulences density field - volume of size 15 pc grid 12003 Interactive setting of the contour value on the density histogram

  24. Visualization of AMR data Volume Projection Box of size 100 h-1 Mpc

  25. Visualization of 3D vector fields Hedgehog display of hydrodynamical velocity field Blue semi-transparent volume = iso-surface of the density RAMSES cosmological simulation box of size 100 h-1 Mpc HERACLES simulation of ISM turbulences

  26. Visualization of particles particle cloud display of a dark matter sample yellow semi-transparent volume = iso-surface of the hydrodynami-cal density field DM haloes

  27. Conclusions and outlooks • An extensive program of simulations of astrophysical plasmas is pursued • New developments are managed in the framework of a software engineering project • Solutions to current limitations will be studied : • HERACLES : implementation of multiple grid algorithm to reach the effective 104 mesh resolution in 3D simulations • RAMSES : algorithm optimization to improve load balancing • Visualization : • optimization of memory management to access larger data sets • parallelization to improve speed (fastDL/mpiDL solutions from RSI) • implementation of multiple grid • New projects : • Extend the use of the core software modules to FARGO and ASH • Database : development of a Virtual Observatory for cosmological simulations (Horizon project) • Openings towards other domains : • HERACLES application to laser fusion and plasma physics

More Related