SCEC Community Modeling Environment (SCEC/CME): Cyberinfrastructure for Earthquake Science

1. SCEC Community Modeling Environment (SCEC/CME):Cyberinfrastructure for Earthquake Science Philip Maechling Southern California Earthquake Center University of Southern California SCEC/UseIT Intern Program June 6, 2005

2. People on the SCEC/CME Project

3. SCEC/CME Researchers • Principal Investigators: • Tom Jordan (USC) • Bernard Minster (Scripps Institution of Oceanography) • Carl Kesselman (USC/ISI) • Reagan Moore (San Diego Supercomputer Center) • Research Leads: • Ned Field (USGS) -- Jacobo Bielak (CMU) • Kim Olsen (SDSU) -- Dave O’Hallaron (CMU) • Steve Day (SDSU) -- Ralph Archuleta (UCSB) • Tim Ahern (IRIS) -- Hans Chalupsky (ISI) • Yolanda Gil (ISI) • Project Manager: • Phil Maechling (USC)

4. SCEC/CME Project Goal:To develop a cyberinfrastructure that can support system-level earthquake science – the SCEC Community Modeling Environment (CME) Support:5-yr project funded by the NSF/ITR program under the CISE and Geoscience Directorates Start date: Oct 1, 2001 NSF CISE GEO SCEC/ITR Project ISI USGS Information Science Earth Science SDSC IRIS SCEC Institutions www.scec.org/cme

5. (http://geohazards.cr.usgs.gov/eq/) Seismic Hazard Analysis Definition: Specification of the maximum intensity of shaking expected at a site during a fixed time interval Example: National seismic hazard maps • Intensity measure: peak ground acceleration (PGA) • Interval: 50 years • Probability of exceedance: 2%

6. (http://geohazards.cr.usgs.gov/eq/) Seismic Hazard Analysis Definition: Specification of the maximum intensity of shaking expected at a site during a fixed time interval Example: National seismic hazard maps • Intensity measure: peak ground acceleration (PGA) • Interval: 50 years • Probability of exceedance: 2%

7. Faulting, shaking, landsliding, liquifaction Extent & density of built environment Structural fragility Risk Analysis: A System-Level Problem Risk = Probable Loss (lives & dollars) = Hazard  Exposure  Fragility

8. The FEMA 366 Report “HAZUS’99 Estimates of Annual Earthquake Losses for the United States”, September, 2000 • U.S. annualized earthquake loss (AEL) is about $4.4 billion/yr. • For 25 states, AEL > $10 million/yr • 74% of the total is concentrated in California • 25% is in Los Angeles County alone

9. 0 - 12 12 - 24 24 - 36 36 - 48 48 - 60 60 - 72 Pathway 1: Puente Hills M 7.1 Scenario Peak Ground Acceleration (% g) Downtown LA Los Angeles County

10. Climate System Atmosphere Hydrosphere Cryosphere Biosphere Lithosphere Asthenosphere Deep Mantle Outer Core Inner Core Mantle Convection Three Global Geosystems Core Dynamo

11. Improvement of models Physics-based simulations Empirical relationships SHA Computational Pathways 1 Standardized Seismic Hazard Analysis Ground motion simulation Physics-based earthquake forecasting Ground-motion inverse problem 2 3 Other Data Geology Geodesy 4 Unified Structural Representation Invert 4 Faults Motions Stresses Anelastic model Ground Motions AWM SRM FSM RDM 3 2 Earthquake Forecast Model Attenuation Relationship Intensity Measures 1 FSM = Fault System Model RDM = Rupture Dynamics Model AWP = Anelastic Wave Propagation SRM = Site Response Model

12. SCEC Community Modeling EnvironmentA collaboratory for system-level earthquake science KNOWLEDGE REPRESENTATION & REASONING Knowledge Server Knowledge base access, Inference Translation Services Syntactic & semantic translation Knowledge Base Ontologies Curated taxonomies, Relations & constraints Pathway Models Pathway templates, Models of simulation codes DIGITAL LIBRARIES Navigation & Queries Versioning, Replication Mediated Collections Federated access KNOWLEDGE ACQUISITION Acquisition Interfaces Dialog planning, Pathway construction strategies Pathway Assembly Template instantiation, Resource selection, Constraint checking Code Repositories FSM RDM AWM SRM Users Data & Simulation Products Data Collections GRID Pathway Execution Policy, Data ingest, Repository access Grid Services Compute & storage management, Security Pathway Instantiations Storage Computing

13. SCEC/CME Computational Pathway Construction A major SCEC/CME objective is the ability to construct and run complex computational pathways for SHA Lat/Long/Amp (xyz file) with 3000 datapoints (100Kb) Define Scenario Earthquake ERF Definition Calculate Hazard Curves Extract IMR Value Plot Hazard Map 9000 Hazard Curve files (9000 x 0.5 Mb = 4.5Gb) IMR Definition GMT Map Configuration Parameters Gridded Region Definition Probability of Exceedence and IMR Definition Pathway 1 example

14. Example Application of Pathway 1:Scenarios for M 7.4 Southern San Andreas Rupture Without soil & basin effects With soil & basin effects Courtesy of Ned Field, USGS, Pasadena

15. SCEC Collaboratory for system-level earthquake science

16. Pathway Comparisons SCEC/CME computational testbed was used to generate PGV Hazard Maps utilizing Pathway 1 and Pathway 2 data sets and SCSN observed data. Pathway 1 (All Firm Soil) Pathway 1 (SCEC CVM 3.0) Pathway 2 (Olsen AWM) PGV data for Northridge from SCSN System (Pathway 0)

17. SCEC IT Challenges • Many geophysical models, computational programs, and data sets and data types. • Large scale simulations, high performance computing and large-scale data management are required in a physics-based approach to earthquake modeling at high spatial-temporal resolution requires. • Communication tools, distributed model development, and computer resource sharing are required by the distributed SCEC collaboration.

18. SCEC/CME Research Areas • Geoscience Research Areas: • Probabilistic Seismic Hazard Analysis • Anelastic Wave Propagation Modeling • Rupture Dynamics Modeling • Data Inversion • IT Research Areas: • High Performance Computing • Grid • Digital Library • Knowledge Representation and Reasoning • 4D Data Visualization • Creation of Computational Pathways • Web Services • Data Integration • Data Standards • Community Computational Models • Outreach and Education: • Undergraduate and Graduate Research Opportunities • Access to non-scientific Users (Emergency Management, Public)

19. Composition Analysis Tool (CAT) Interface User building a pathway specification from library of components Errors and fixes generated by ErrorScan algorithm

20. The SCEC Earthquake Simulation SCEC ITR Collaboration

21. Major Earthquakes on the San Andreas Fault, 1680-present 1906 M 7.8 1857 M 7.9 ~1680 M 7.7

22. TeraShake Simulation Area

23. 33 researchers, 8 Institutions Southern California Earthquake Center San Diego Supercomputer Center Information Sciences Institute Institute of Geophysics and Planetary Physics (UC) University of Southern California San Diego State University University of California, Santa Barbara Carnegie-Mellon University EXonMobil

24. TeraShake Peak Ground Velocity Maps NW to SE rupture SE to NW rupture

25. almaak.usc.edu • - SUN Sunfire 15K • 64 CPUs • 256GB RAM • condor.usc.edu • Condor pool • A collection of 320 • SUN workstations • epi.usc.edu • SUN E3800 • 8CPUs- 8GB RAM • horizon.sdsc.edu • IBM Blue Horizon • 1152 CPUs • 576GB RAM • gravity.usc.edu • Linux • 4 CPUs- 4GB RAM • hpc.usc.edu • IBM Linux cluster • 640 CPUs • 320GB RAM • giis.scec.org/ • scec-giis.isi.edu • Linux • 2CPUs, 1GHz • 1GB RAM • pinto.isi.edu • Linux • 2CPUs,500MHz • 380MB RAM • sidecar.psc.edu • Linux • 1CPU- 1GB RAM SCEC/CME Grid Infrastructure SCEC/CME has established grid-based connectivity, job-scheduling, and user and host authentication between SCEC, USC, ISI, SDSC, and PSC. SCEC Grid Testbed SCEC USC SDSC PSC ISI Current SCEC Grid configuration

26. SCEC Community Library • Data grid architecture using SDSC Storage Resource Broker • Supports user customizable portals • Maintains associations between data and metadata • Current collections contain 1.6 million files (10 terabytes) • 3D ground motion for LA Basin (36 scenarios) • Rupture Dynamics 4D Wavefield • http://www.sdsc.edu/SCEC

27. SCEC UseIT Undergraduate Intern Program

28. LA3D GeoWall Software

29. Group Interaction and Collaboration Tools • Applying communication tools to help with this distributed collaboration. • Web Sites • Software Configuration Management Tools • Document Management Tools • Bug Tracking Tools • Data File and Metadata Tools • Email Lists

30. Community Fault Model (CFM-A) Community Velocity Model (CVM.3.0) Community Crustal Motion Map (CMM.3.0.1) Hosting of SCEC Community Models • Provide access to SCEC Community Models, possibly alternative models, and utilities for working with the models.

31. Validation Exercises for Simulation Codes Comparisons for 09/03/02 Yorba Linda EarthquakeData in black, SCEC CVM (FD) in blue, Harvard model (SEM) in red Comparison of Dynamic Rupture Models Rupture Test Case Contours

32. Hosting of SCEC Community Codes • Provide access to SCEC geophysical codes • Pathway 2: • Olsen AWM • CMU Hercules AWM • Pathway 1: • OpenSHA • Pathway 4: • Synthetic Seismograms • Fréchet Kernels

33. Supporting and Running Large Scale Simulations 8 Processors (in 2002) 240 Processors (in 2004)

34. almaak.usc.edu • - SUN Sunfire 15K • 64 CPUs • 256GB RAM • condor.usc.edu • Condor pool • A collection of 320 • SUN workstations • epi.usc.edu • SUN E3800 • 8CPUs- 8GB RAM • horizon.sdsc.edu • IBM DataStar • 1152 CPUs • 576GB RAM • horizon.sdsc.edu • IBM Blue Horizon • 1152 CPUs • 576GB RAM • gravity.usc.edu • Linux • 4 CPUs- 4GB RAM • hpc.usc.edu • IBM Linux cluster • 640 CPUs • 320GB RAM • giis.scec.org/ • scec-giis.isi.edu • Linux • 2CPUs, 1GHz • 1GB RAM • pinto.isi.edu • Linux • 2CPUs,500MHz • 380MB RAM • sidecar.psc.edu • Linux • 1CPU- 1GB RAM Establishment of SCEC Grid Infrastructure SCEC/CME has established grid-based connectivity, job-scheduling, and user and host authentication between SCEC, USC, ISI, SDSC, PSC, and TeraGrid sites. SCEC Grid Testbed SCEC USC SDSC PSC TeraGrid ISI

35. Select Receiver (Lat/Lon) Output Time History Seismograms Select Scenario Fault Model Source Model SCEC Community Library SCEC Digital Library - Providing Data Management Capabilities • Storage Resource Broker based Digital Library Collection now includes SCEC/PEER Scenario Ground Motion data collection, USC Green Tensors data collection (40TB+ Storage), TeraShake Simulations (40 TB+), and Puente Hills Simulation.

36. SDSC Data Visualization

37. ISI Data Visualization

38. Example SCEC and SCEC/CME IT-oriented Activities • UseIT Intern Program • Unified Structural Representation • Ground Motion Prediction • Communication, Education, Outreach • OpenSHA Seismic Hazard Analysis • Earthquake Forecasting • Unified Structural Representation • Ground Motion Prediction • Communication, Education, Outreach

39. Example SCEC and SCEC/CME IT-oriented Activities • CyberShake Waveform-based Seismic Hazard Analysis • Unified Structural Representation • Earthquake Rupture Dynamics • Ground Motion Prediction • Communication, Education, Outreach • TeraShake 2 Dynamic Rupture-based Simulations • Unified Structural Representation • Earthquake Rupture Dynamics • Ground Motion Prediction • Communication, Education, Outreach

40. CyberShake ProjectUsing 3D Synthetic Seismic Waveforms In Seismic Hazard Analysis

41. Various IMR types (subclasses) Attenuation Relationships Gaussian dist. is assumed; mean and std. from various parameters IMT, IML(s) Multi-Site IMRs compute joint prob. of exceeding IML(s) at multiple sites (e.g., Wesson & Perkins, 2002) Rupture Site(s) Intensity-Measure Relationship List of Supported IMTs List of Site-Related Ind. Params Vector IMRs compute joint prob. of exceeding multiple IMTs (Bazzurro & Cornell, 2002) Simulation IMRs exceed. prob. computed using a suite of synthetic seismograms

42. Ruptures in ERF within 200KM of USC

43. CyberShake Computational Elements • Large (TeraShake Scale) forward calculations for each site. • Requires calculation of 100,000+ seismogram for each site. • SCEC/CME Grid-based scientific workflow system required to work at this scale. • Access to distributed computing resources • Large scale file management • High performance and high throughput computing. • TeraGrid allocation awarded for effort • 145K SU (TG-BCS050001N)

44. End