Seismic Simulations of Explosions and Earthquakes Computing Grand Challenge Symposium

1. Arthur Rodgers Chemistry, Materials, Earth and Life Sciences Directorate Seismic Simulations of Explosions and Earthquakes Computing Grand Challenge Symposium Lawrence Livermore National Laboratory, P. O. Box 808, Livermore, CA 94551 This work performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344 IM #358050

2. Our Effort • This team effort involved several people • Kathleen McCandless • Computer Application and Research, Computations • Anders Petersson, Bjorn Sjogreen, Stefan Nilsson • Center for Applied Scientific Computing, Computations • Jeff Wagoner, Phil Harben • Atmospheric, Earth and Energy Division, CMELS • Rich Cook, Liam Krauss, Becky Springmeyer • IM and Graphics Group,Computations • Bill Walter and Steve Myers • Geophysical Monitoring Program, AEED & NP-Div • Dave McCallen • NP-Division, Global Security, PD • We had an allocation of 50,000 CPU-hours/week on Thunder • Also, several DAT weekends

3. Summary • This allocation enabled progress in seismic simulations in several areas • Nuclear explosion monitoring (NNSA/NA-22) • Oct. 9, 2006 North Korean nuclear test • Hydroacoustic wave reflection/conversion • Earthquake ground motion (USGS) • San Francisco Bay Area 3D model • Hayward Fault scenario earthquakes • It also enabled development of advanced features of the WPP elastic wave propagation code • LDRD project 05-ERD-079 • We attracted support from NNSA/NA-22 for a demonstration calculation on BlueGene/L

4. Seismic simulations are computationally intensive • Numerical (finite difference or element) algorithms: • discretize a 3D volume of the earth into grid points • require a certain number of grid points/wavelength fmax = vmin/min and min = n*h • step through time explicitly, time step t • high frequencies require small h and high velocities require small t. • We typically want to model • large volumes (many wavelengths) • high-resolution (frequency) • We’ve used two codes: • FD (WPP, LLNL) • SEM (SPECFEM3D, Caltech) h

5. We are striving for ever larger domains and higher resolution (frequency)

6. Nuclear Explosion Monitoring (NNSA/NA-22) • NEM requires analysis of signals resulting from wave propagation phenomena • Seismic • waves in the solid earth • Hydroacoustic • waves in the ocean (SOFAR channel) • The physics of these phenomena are generally well understood and can be modeled. • However, we do not know the material properties of the earth to the scale-length required to model the full bandwidth of observations

7. We modeled seismograms from the 9 October, 2006 North Korean Nuclear Test Seismograms at Beijing (BJT) showed: signals at BJT (1100 km) are weak large amplitude surface waves energy on transverse component - possibly due to sympathetic earthquake surface waves We wanted to know if model(s) of 3D structure, including sedimentary basins, can predict the observed wavefield.

8. 3D model predicts the observed energy partitioning - consistent with explosion source data simulation Explosion source in 1D model predicts no energy on transverse component Explosion source in 3D model predicts refracted energy on transverse component

9. The sparse hydroacoustic network requires maximum information be extracted reflections However, reflections have lower amplitudes Hydrophones in the Indian Ocean reflection direct Reflections can help locate events, or may be provide the only detection when direct wave is blocked.

10. We modeled the hydroacoustic reflection from the Seychelles Plateau Incoming wave is reflected by bathymetry land 450 million points h=50 m 125 x 100 x 4.25 km Ran in ~ 5,000 CPU-hours ocean ocean

11. Earthquake modeling in the SF Bay Area • We have been modeling earthquakes in the Bay Area with the USGS (Menlo Park) since 2005 • Evaluation of a 3D geologic/seismic model • In press at BSSA (Rodgers et al., 2008) • October 31, 2007 Alum Rock earthquake • Did you feel it? • Simulations of the 1906 SF earthquake • In press at BSSA (Aagaard et al., 2008) • Currently, working on simulations of a M 7.0 Hayward Fault earthquake • Presented at AGU Fall 2007

12. We evaluated the USGS 3D seismic model of the San Francisco Bay Area USGS 3D Model We compared simulated and observed seismograms for moderate (M ~ 4-5) earthquakes, 2000-4000 CPU-hours/run Simulated seismograms are late (t < 0) relative to observed. Model is too fast! However, it’s being fixed

13. October 31, 2007 Alum Rock Earthquake MD WC Oak Tri-Valley LLNL SF Bay East West Diablo Range SC Valley South

14. October 30, 2007 (M 5.6) Alum Earthquake was the largest since 1989 (M 6.9) Loma Prieta Recordings at Wente Shaking in Livermore was ~ 1% g Large earthquake (M>6.5) expect > 50% g LLNL code and USGS model can accurately predict ground motions, including future large earthquakes

15. 1906 SF simulation, f ≤0.5 Hz68,000 CPU-hours 200 km 550 km 40 km deep

16. We’re working with the USGS on Hayward Fault scenario earthquakes - the mostly likely next EQ One “ShakeMap” for M 7 Simulation of the 1995 Kobe, Japan earthquake on the Hayward Fault Mesh refinement reduces effort to ~8000 CPU-hours Slip distribution along fault

17. Conclusions and Future Directions • This allocation allowed us to advance and demonstrate seismic modeling capabilities for a number of applications • The results from the last twelve months have • enabled important new science that was not possible with routine LC access • attracted interest from sponsors • Future directions • BlueGene/L port & demo • Improve seismic velocity models • Including waveform methods • Perform higher resolution simulations • Use suites of simulations to bound uncertainty