phy.ornl/tsi/

1. TeraScale Supernova Initiative http://www.phy.ornl.gov/tsi/

2. Investigator Team • Cross-Cutting Team • Long-Term Collaborations • Structured like SciDAC TOPS • Linear System/Eigenvalue Problem Solution Algorithms for Radiation Transport and Nuclear Structure Computation • Dongarra (UT, ORNL) • Saied (UIUC, NCSA) • Saylor (UIUC, NCSA) • Radiation Transport/ • Radiation Hydrodynamics • Blondin (NC State) • Bruenn (FAU) • Hayes (UCSD) • Mezzacappa (ORNL) • Swesty (SUNYSB) • Supernova Science • Blondin • Bruenn • Fuller • Haxton • Hayes • Lattimer • Meyer (Clemson) • Mezzacappa • Swesty TOPS CCA PERC TSTT • Nuclear Structure Computations • for EOS and Neutrino-Nucleus/ • Nucleon Interactions • Dean (ORNL, UT) • Fuller (UCSD) • Haxton (INT, Washington) • Lattimer (SUNYSB) • Prakash (SUNYSB) • Strayer (ORNL, UT) SDM • Visualization • Baker (NCSA) • Toedte (ORNL)

3. Goal • Ascertain the explosion mechanism(s). • Reproduce supernova phenomenology (element synthesis; neutrino, gravitational wave, and gamma ray signatures; neutron star kicks; gamma ray burst connection) • Relevance • Dominant source of many elements in the Universe. • Given sufficiently well developed models, serve as laboratories for fundamental nuclear and particle physics that cannot be explored in terrestrial laboratories. • Driving application in computational science (radiation transport, hydrodynamics, nuclear physics, applied mathematics, computer science, visualization). • Paradigm • Result from stellar core collapse and bounce in massive stars. • Radiatively driven (perhaps some are • MHD driven, or both).

4. Convection • Need Boltzmann Solution • Need Angular Distribution • Need Spectrum • “Gray” Schemes Inadequate • Spectrum Imposed • Limited Angular Information • (Few Moments) • Parameterized • (No First Principle Solution) • The bar is high! (10% effects can make or break explosions.)

5. 0D 0D 1D 1D Neutrino Energy Lightbulb D FLD MGFLD MGBT Space Burrows, Hayes, and Fryxell Janka and Mueller 1D Mezzacappa et al. Herant et al. TSI Year 1 TSI Year 2 2D Swesty Fryer and Heger Past Transport in 2D Models D: Diffusion FLD: Flux-Limited Diffusion MGFLD: Multigroup FLD MGBT: Boltzmann Transport TSI Year 2 TSI Year 3 3D Gray Models

6. Latest TSI 2D/3D Models: • Hydrodynamics only. • Focused on understanding 2D/3D flow and its • coupling to shock wave. • Convectively stable. • 2D model exhibits bipolar explosion(due to nonlinear flow-shock interaction). • 3D model exhibits similar “long-wavelength” • behavior. Key finding. • New “rolling” flows identified. AAS Meeting; Ap.J. Submitted 2D Model 3D Model

7. Explosion Mechanism: Open Questions • What is theRecipe for Explosion? Neutrino Heating Convection General Relativity Rotation MagneticFields • Are there multiple mechanisms? • Neutrino-driven supernovae • MHD-driven supernovae • Supernovae driven by both neutrinos and MHD effects • One mechanism for a class of stars? • Is the mechanism tailored to the individual star?

8. Nuclear and Weak Interaction Physics Needs High-Density EoS Nuclear MatterOpacities Thomas Fermi (Classical) Classical treatment of many-body problem. Ensembles Hartree-Fock Density of States Lowest order solution to the quantum mechanical many-body problem. e-capture n-nucleus Shell Model Monte Carlo b-decay Shell Model Diagonalization Time Advanced solutions to the many body problem. Bloch-Horowitz Solve “exact” many-body problem.

9. Supernova Nucleosynthesis R-Process Breakthrough • r-process can occur in “symmetric” environment (equal numbers of protons and • neutrons) under certain conditions (high entropy, fast expansion). • Meyer, PRL Submitted

10. Supernova Science Hydrodynamics Explicit Differencing Reactive Flows Newtonian General Relativistic Nuclear, Weak Interaction Physics Thermodynamics (Composition), Neutrino Sources and Interactions Radiation Transport Implicit Differencing MGFLD Preconditioners Sparse System Solvers MGBT Preconditioners Sparse System Solvers (Matrix Free)

11. Integration of Technologies Generation 3 High-Resolution 3D MGFLD with Full Integration of Components (Ensemble of Nuclei, State of the Art Neutrino-Matter Interactions, ...) Increasing Integration Generation 2 Inclusion of state of the art neutrino interactions in “Generation 1” MGBT/ MGFLD Simulations Generation 1 Increasing Integration 2D MGFLD Simulation with Naive Neutrino Interactions and Single-Nucleus Equation of State Computation of State of the Art Neutrino-Matter Interactions

12. Supernova Simulation Timeline Year 0 Year 1 Year 2 Year 3 Year 4 Year 5 2D MGFLD Models (3D) 3D MGFLD Models w/ AMR (4D) 3D MGFLD Models (4D) 2D Boltzmann Models (5D) 1D Boltzmann Models (3D) 3D Boltzmann Models (6D)

13. ISIC Collaborations: TOPS • Nonlinear Algebraic Equations • Linearize • Solve via Multi-D Newton-Raphson Method • Large Sparse Linear Systems Boltzmann Equation nonlinear integro-PDE • Implicit Time Differencing • Extremely Short Neutrino-Matter Coupling Time Scales • Neutrino-Matter Equilibration • Neutrino Transport Time Scales Memory Requirements (assuming matrix-free methods): 10s Gb up to 1/2 Tb Progress: Sparse Approximate Inverses for 2D MGFLD (Saylor, Smolarski, Swesty; J. Comp. Phys.) ADI-Like Preconditioner for Boltzmann Transport (D’Azevedo et al.; Precond 2001, NLAA) AGILE-BOLTZRAN, V2D codes turned over to TOPS for analysis and development.

14. ISIC Collaborations: CCTTSS TSI Code: • F90 + MPI Code • Object-Oriented Design for Interoperability and Reuse • Application Framework: IBEAM = Interoperability Based Environment for Adaptive Meshes • NASA HPCC-Funded Project (PI: Swesty) • AMR: PARAMESH • Goal: Develop our framework to be CCA-compliant. • Initiated discussions with ANL, LLNL, and ORNL members of CCTTSS.

15. ISIC Collaborations: PERC • Assess Code Performance on Parallel Platforms • Identify Code Optimizations to Increase Performance • TSI Code Suite • Hydrodynamics: • VH-1 (PPM) • ZEPHYR (Finite Difference) • Neutrino Transport: • AGILE-BOLTZTRAN: 1D General Relativistic Adaptive Mesh • Hydrodynamics with 1D Boltzmann Transport • V2D: 2D MGFLD Transport Code • V3D: 3D MGFLD Transport Code (Under Development) • 2D/3D Boltzmann Code (Under Development) VH-1 numerical hydrodynamics algorithm scales well. Results for VH-1

16. ISIC Collaborations: SDM • Use PROBE environment for staging data between simulation platforms and • end-user visualization platforms. • Develop new data analysis techniques/tools tailored to our application, allowing • (a) data reduction and (b) discovery potential. • Use of agent technology for distributed data analysis (data analysis must be • done in parallel to achieve reasonable throughputs).

17. ISIC Collaborations: TSTT Adaptive Quadratures (Direction Cosines) for Multidimensional Radiation Transport • Greatest challenge to completing 3D Boltzmann • simulations is memory. • Minimize number of quadratures to minimize • memory needs while maintaining physical • resolution. (Also important for 1D/2D MGBT.) • Optimization Problem Results for 1D Boltzmann Transport on Milne Problem (D’Azevedo): Extended Core Compact Core

18. Collaboration with Supporting Base Projects: Networking • Identify Optimal Paths in Our Collaborative Visualization Server-Client Model • Maximize Bandwidth along these Paths (Not Achieved Using Current Protocols) • Participated in ORNL Workshop on DoE High-Performance Network R&D and Applications • Convey TSI Needs to Networking Team • Participate in White Paper to Define and Develop Interface between Efforts