The USC High Performance Computing Center’s Infrastructure for Earthquake Engineer Research using the Globus Toolkit

1. Abstract 4. USCGrid User: Southern California Earthquake Center USC Information Services Division (ISD) provides various information technologies for the University community. In conjunction with USC High Performance Computing Center (HPCC) and Information Sciences Institute (ISI), the emerging technology group within ISD has created and maintains USCGrid, a production installation of the Globus Toolkit and related Grid software components including GPT, Condor-G, KX.509, and Ganglia with an extension integrating it with the Globus Toolkit. USCGrid has been in operation since July 2002. It has been used by earthquake engineers to run computationally intensive physics-based earthquake simulations. The integration of the Globus Toolkit and other grid software with USC campus IT infrastructure makes it possible for users (e.g. geophysicists) to easily and seamlessly submit their jobs to the USCGrid computing facilities, and soon other Grids (e.g. Teragrid) in the future. KNOWLEDGE REPRESENTATION & REASONING Knowledge Server Knowledge base access, Inference Translation Services Syntactic & semantic translation The Southern California Earthquake Center (SCEC), headquartered at the University of Southern California, is a regionally focused organization with a mission to gather new information about earthquakes in Southern California and integrate that knowledge into a comprehensive and predictive understanding of earthquake phenomena. Knowledge Base Ontologies Curated taxonomies, Relations & constraints Pathway Models Pathway templates, Models of simulation codes DIGITAL LIBRARIES Navigation & Queries Versioning, Topic maps Mediated Collections Federated access KNOWLEDGE ACQUISITION Acquisition Interfaces Dialog planning, Pathway construction strategies Pathway Assembly Template instantiation, Resource selection, Constraint checking With recent advances in geophysics, SCEC has embarked on an ambitious program to develop physics-based models of earthquake processes and integrate these models into a new scientific framework for seismic hazard analysis and risk management. To achieve these objectives, the environment must provide a means for describing, configuring, instantiating, and executing complex computational pathways that result from the composition of various earthquake simulation models. Grid technologies, as part of SCEC-ITR multidisciplinary collaboration, will enable SCEC researchers access to distributed simulation codes and resources for the timely execution of simulation scenarios, specifically by integrating high-performance computing resources into the execution environment. The USCGrid, as a part of a shared computational testbed for SCEC applications, has been successfully used to calculate synthetic seismograms using anelastic wave-propagation simulation. Code Repositories FSM RDM AWM SRM Users Data & Simulation Products Data Collections GRID Pathway Execution Policy, Data ingest, Repository access Grid Services Compute & storage management, Security Pathway Instantiations Storage Computing 1. USCGrid USC ISD provides the University community with information technology support in the area of networking, library services, academic computing and telecommunications. ISD is always in search for, experimenting with and maintaining state-of-the-art emerging technologies that can be used to facilitate users’ operations on its maintained computing and storage resources. Recent advances in grid technologies enable the ability to pull geographically distributed resources into a single integrated computation. In addition, grid technologies can be used to facilitate multi-organization collaborative effort which we envision will be common in the future. Therefore, in conjunction with USC High Performance Computing Center (HPCC) and Information Sciences Institute (ISI), the emerging technology group within ISD has created and maintained USCGrid, a production installation of the Globus Tooklit and other Grid-related software components. 5. SCEC Execution Environment: Physics-based Wave Propagation Simulation The current execution environment involves a user (e.g. a SCEC scientist) submitting a job request to a job submission manager on his or her unix workstation at USC. In the future, this request can be made by a workflow manager which given an input computational pathway instantiation, automatically generates an execution plan (e.g. a sequence of execution steps) and submits it to the testbed. Structural Representation Faults Motions StressesAnelastic model Ground Motions AWM SRM AWM = Anelastic Wave Model SRM = Site Response Model almaak.usc.edu User’s proxy Certificate it is used for user authentication in order to access grid services This figure shows how we may obtain a wave simulation via the testbed. Here, a SCEC scientist (almaak.usc.edu) will request wave propagation code (Olsen, 2001) to be run on a USCGrid machine (hpc.usc.edu). (1) Create user’s proxy certificate v via KX.509 Earthquake Forecast Model Attenuation Relationship Intensity Measures • USCGrid deploys a production installation of the following software from the NSF Middleware Initiative Release 2 (http://www.nsf-middleware.org) • The Globus Toolkit: provides mechanisms for the discovery, access, and management of distributed computation and storage resources (i.e. grid services). • Condor-G: a computation management agent for the Grid. It is used an a front-end to a computational grid. • KX.509: built upon the Kerberos authentication infrastructure to allow easy access to the grid components for anyone with a USC unix login ID, while maintaining authentication and authorization controls to enforce restrictions on research compute facilities. Pathway 2: Ground motion simulation (physics-based simulations) (2) Submit requests CondorG A SCEC scientist • Run wave propagation model on hpc.usc.edu • Transfer the output from hpc.usc.edu to almaak.usc.edu (5) User plots the wave propagation output or (5) User runs visualization tool on the output • Fundamental services provided by USCGrid infrastructure • Security infrastructure: provides single sign-on access to all accessible resources. Integrated with USC campus wide authentication system. • Information services: for discovering available resources and their characteristics e.g. the amount of disk space available. • Uniform resource management: enables allocation, monitoring and control of a variety of different resource types, including computers, storage systems and networks. • GridFTP: provides high performance data transfer for data-intensive applications. 2. USCGrid Software Suites hpc.usc.edu Grid services (4) Submit request for file transfer (3) Jobs are submitted to remote servers GRAM GridFTP 6. Future Plan (3.1) Jobs are submitted to local scheduler 3. USCGrid Testbed (4.1) Output files Are transferred to pinto.isi.edu via GridFTP USC ISD emerging technology group is working to maintain and extend the Globus-Toolkit-based USCGrid as fully operational and production-grade. The Grid software components have been integrated with campus IT infrastructure to provide seamless services to the user community. In addition, the group is preparing to connect the USCGrid to other universities, research institutions, and Grid based projects that are also using the Globus Toolkit to create an even larger computational and data grid. Regarding the cross-organization Grid interoperability, ISD emerging technology group has started the process of defining trust relationships and a broadly acceptable PKI certificate Policy for its KCA and KX.509 installation. This mission will enable members of the USCGrid to exploit computing resources around the world. It will also facilitate collaborative research involving extensive computing resources owned and maintained by different administrative organizations. Support for cross certification with the DOE Science Grid, NPACI, and another prominent southern California based university is underway. Job submission manager (condor-G, Globus client) KX.509 certificate Local resource scheduler Grid services by theGlobus Toolkit (3.2) Jobs get executed USCGrid testbed consists of the following high performance and high throughput computing facilities: Compute Resource Management (GRAM) Data Resource Management (GridFTP) Information Services (MDS) Compute Resources Data Storage Local resource scheduler Acknowledgements: David Okaya (USC), Kim Olsen (UCSB) Garrick Staples (ISD at USC) More information: www.scec.org www.globus.org www.cs.wisc.edu/condor/condorg • hpc.usc.edu • IBM Linux cluster • 1646 CPUs • 560GB RAM • 13.8TB Disk • almaak.usc.edu • - SUN Sunfire 15K • 64 CPUs • 256GB RAM • epi.usc.edu • SUN E3800 • 8CPUs- 8GB RAM • HPCcondor.usc.edu • Condor pool • A cluster of 320 SUN workstations Compute resources Data Repository Note: GRAM, GridFTP, MDS are provided by the Globus Toolkit. Grid Security Infrastructure (GSI) for user/resource authentication and authorization are implemented by all Globus components. Volume Rendering: Cross-Eye Stereo Pairs P-wave velocity of rocks within volume (0-8131.3 meters/sec) Focus on a point in front of this stereo pair. Shelley Henderson*, Hongsuda Tangmunarunkit**, John Mcgee** *Center for High Performance Computing and Communication, USC, ** Center for Grid Technologies, USC/ISI http://www.usc.edu/isd/services/uscgrid The USC High Performance Computing Center’s Infrastructure for Earthquake Engineer Research using the Globus Toolkit