1 / 12

Development of an Integrated Earth System Climate Modeling Environment

Development of an Integrated Earth System Climate Modeling Environment. Purdue University/RCAC NCAR/ESG NOAA/CIRES. A Great Team!. Purdue/RCAC Carol X. Song, Lan Zhao, Chris Thompson, Robert Campbell , Jungha Woo, Han Zhang, Madhavan Lakshminarayanan NCAR

mahlah
Download Presentation

Development of an Integrated Earth System Climate Modeling Environment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Development of an Integrated Earth System Climate Modeling Environment Purdue University/RCAC NCAR/ESG NOAA/CIRES

  2. A Great Team! Purdue/RCAC Carol X. Song, Lan Zhao, Chris Thompson, Robert Campbell, Jungha Woo, Han Zhang, Madhavan Lakshminarayanan NCAR Don Middleton, Nathan Wilhelmi, Eric Nienhouse, Michael Burek, Rick Brownrigg NOAA/CIRES Kathy Saint (SGI), Cecelia DeLuca, Sylvia Murphy “Developing an integrated end-to-end TeraGrid climate modeling environment”, TeraGrid 2011 conference, Salt Lake City, UT, July 2011. XSEDE Science Gateway

  3. Background • Motivation for a climate modeling environment • CCSM/CESM • Resource • Expertise • A collaborative online environment • Existing synergistic community efforts • Earth System Grid, Earth System Curator, ESMF, METAFOR Common Information Model and Purdue CCSM3 portal • Project goals • Provide an integrated environment for climate modeling, data/metadata archival, visualization and analysis • Support research and education, lower barriers to using climate models XSEDE Science Gateway

  4. System Architecture CESM portal ESG gateway Account DB User requests TG MyProxy Token Mgr CESM Web Services SOAP and Restful XML/JSON Create Case Configure Case Submit Case Track Status Data/ Metadata Authentication/ Authorization Transfer Files Job Management Debugging Post-process Publish Data Publish Metadata XD community account/Individual account Output Output Jobs ESG Data Publisher Scratch Storage iRODS XSEDE Science Gateway

  5. CESM Portal - Model Workflow • Insulate users from cluster • Hide command line details • Integrate CESM 1.0 + metadata capabilities + ESG publishing + NCL • Web services (SOAP, Restful XML/JSON) • Manage user’s experience • Maintain job status and metrics • End-to-end modeling from simulation to data/metadata publishing • Bind independent processes into unified workflow CESM-WS/ gateway Create case CESM Model Create create_case Configure case Configure ./configure CESM Portal SOAP Compile <case>.compile JBOSS Compile case Job Scripts Publish Scripts Execute <case>.submit Query Status Execute case Data Configure post-process Post-processing Post-processing Engines Run post-process AMWG NCL scripts ESG Publication workflow … XSEDE Science Gateway

  6. ESMF Attributes and Self-describing Models • Metadata is increasingly important in Earth science modeling • Understand multi-component models with many options and parameters • Distinguish simulations from one another • Archive simulations with full descriptions for preservation and defensibility • Enable run reproducibility • ESMF Attributes store, aggregate, and output metadata • Structured as name-value pairs • Arranged in packages by purpose and convention:International Standards Organization (ISO), Climate and Forecast (CF), METAFOR CommonInformation Model (CIM), etc. • Output in XML and other formats • ESMF Attributes in CESM • CESM components (land, atm, ice, ocean, sea ice) and driver are being instrumented with Attributes • Information captured includes component level, field level, andsystem level metadata XSEDE Science Gateway

  7. CESM Metadata in the ESG Gateway Screenshot of CESM metadata Implemented in the model by ESMF and Curator. Exported by CESM at run time. Uploaded to Purdue’s atom feed. Ingested and displayed by ESG. XSEDE Science Gateway

  8. Data Discovery, Access and Analysis • ESG: Science Gateway for Earth System Data Access • Data discovery and access across federation of data nodes. • Identity services and project specific access control. • Data publishing services. • Versioning and provenance support. • Bulk data download, wget scripts. • Data use tracking and metrics. • Model documentation metadata browsing. • ATOM CIM feed and ESG ingest. • NCL Backend to Product Services • Integrated 64-bit NCL visualization engine. • Big data capable analysis functions. • ParNCL as a drop-in replacement, for parallel computation. • Plug-in component to NOAA/PMEL LAS Product Server. XSEDE Science Gateway

  9. Demo XSEDE Science Gateway

  10. Demo XSEDE Science Gateway

  11. Demo XSEDE Science Gateway

  12. Next Steps • Transition into XD • Release CESM portal • Work with faculty and instructors to use in teaching • Need help in connecting with users • Federated authentication/identity between CESM and ESG gateways • Making a rich set of tools for analysis and visualization available to CESM gateway users • Add other resources to backend for better user experience XSEDE Science Gateway

More Related