Earth System Modeling Framework Capabilities

1. Earth System Modeling Framework Capabilities NASA GSFC GSI MITgcm Cecelia DeLuca cdeluca@ucar.du SCD Users Forum May 18, 2005 http://www.esmf.ucar.edu GFDL FMS Suite NCAR/LANL CCSM NCEP Forecast GMAO Seasonal Forecast

2. ESMF Tutorial at the SCD Users Forum 30+ participants Goals of the Tutorial: • To give future ESMF users an understanding of the background, goals, and scope of the ESMF project • To review the status of the ESMF software implementation and current application adoption efforts • To outline the overall design and principles underlying the ESMF software • To describe the major classes and functions of ESMF in sufficient detail to give future users an understanding of how ESMF could be utilized in their own codes • To describe in steps how a user code prepares for using ESMF, incorporates ESMF, and runs under ESMF • To identify ESMF resources available to users such as documentation, mailing lists, and support staff • To examine and work with code examples in order to demonstrate ESMF adoption and use

3. What is ESMF? • ESMF provides tools for turning model codes into componentswith standard interfaces and standard drivers. • ESMF provides data structures and common utilities that components use for routine services such as data communications, regridding, time management and message logging. ESMF GOALS • Increase scientific productivity by making model components much easier to build, combine, and exchange, and by enabling modelers to take full advantage of high-end computers. • Promote new scientific opportunities and services through community building and increased interoperability of codes (impacts in collaboration, code validation and tuning, teaching, migration from research to operations)

4. ESMF Application Example: GEOS-5 AGCM Each box is an ESMF-compliant component with a standard interface • The ESMF software provides tools for building modular, component-based applications • The ESMF community offers a pool of technically compatible geophysical components

5. Growing ESMF Customer Base • New applications coming in during FY05 through the newly funded, ESMF-based DoD Battlespace Environments Institute (BEI): DoD Navy HYCOM oceanDoD Navy NOGAPS atmosphereDoD Navy COAMPS coupled atm-ocean DoD Air Force GAIM ionosphereDoD Air Force HAF solar windDoD Army ERDC WASH123 watershed • More new applications will begin adopting ESMF during FY06 through the ESMF-based NASA Modeling Analysis and Prediction (MAP) Climate Variability and Change program. • Further growth of the customer base is anticipated through development of an ESMF-based Space Weather computational environment. • Original ESMF applications:NOAA GFDL atmospheresNOAA GFDL MOM4 oceanNOAA NCEP atmospheres, analysesNASA GMAO models and GEOS-5NASA/COLA Poseidon oceanLANL POP oceanNCAR WRFNCAR CCSMMITgcm atmosphere and ocean • Other groups using ESMF: NASA GISSUCLACSUNASA Land Information Systems (LIS) projectNOAA Integrated Dynamics in Earth’s Atmosphere (IDEA) project, more…

6. ESMF Impacts ESMF impacts a very broad set of research and operational areas that require high performance, multi-component modeling and data assimilation systems, including: • Climate prediction • Weather forecasting • Seasonal prediction • Basic Earth and planetary system research at various time and spatial scales • Emergency response • Ecosystem modeling • Battlespace simulation and integrated Earth/space forecasting • Space weather (through coordination with related space weather frameworks) • Other HPC domains, through migration of non-domain specific capabilities from ESMF – facilitated by ESMF interoperability with generic frameworks such as CCA

7. Adoption Status for Original NASA Collaborators adoptiondue summer 2005

8. DoD Battlespace Environments Institute To develop a DoD-wide whole-earth environment which interoperates with that from other agencies: • Migrate core DoD models to ESMF • Navy (e.g., NCOM, HYCOM, SWAN, COAMPS™) • Air Force (Kinematic Solar Wind and GAIM ) • Army (e.g., ADCIRC, WASH123) • Development of tools and applications (e.g., extend ESMF to support unstructured grids and nesting) • Coupled applications: Air/ocean, air/ocean/ice, air/ocean/groundwater, air/space-weather • Thorough testing, prototyping, documentation of all components • $11M over six years of funding, starting FY05

9. NASA Modeling Analysis and Prediction Program • ESMF-based computational environment for researchers investigating climate variability and change • New set of climate, chemistry, and related applications coming in to ESMF • Awards to be announced May 2005

10. ESMF Development Status • Overall architecture is well-defined • Components and low-level communications stable • Concurrent or sequential execution, single executable • Serial or parallel execution (mpiuni stub library) • Logically rectangular grids with regular and arbitrary distributions can be represented and regridded • On-line parallel regridding (bilinear, 1st order conservative) completed and optimized • Other parallel methods, e.g. halo, redistribution, low-level comms implemented • Utilities such as time manager, logging, and configuration manager usable and adding features • Virtual machine with hooks for load balancing implemented • Fortran interfaces and complete documentation, some C++ interfaces

11. ESMF Platform Support • IBM AIX (32 and 64 bit addressing) • SGI IRIX64 (32 and 64 bit addressing) • SGI Altix (64 bit addressing) • Cray X1 (64 bit addressing) • Compaq OSF1 (64 bit addressing) • Linux Intel (32 and 64 bit addressing, with mpich and lam) • Linux PGI (32 and 64 bit addressing, with mpich) • Linux NAG (32 bit addressing, with mpich) • Linux Absoft (32 bit addressing, with mpich) • Linux Lahey (32 bit addressing, with mpich) • Mac OS X with xlf (32 bit addressing, with lam) • Mac OS X with absoft (32 bit addressing, with lam) • Mac OS X with NAG (32 bit addressing, with lam) • User-contributed g95 support

12. Planned ESMF Extensions • Looser couplings: support for multiple executable and Grid-enabled versions of ESMF • Support for representing, partitioning, communicating with, and regridding unstructured grids and semi-structured grids • Support for advanced I/O, including I/O support for both logically rectangular, semi-structured, and unstructured grids, asynchronous I/O, checkpoint/restart, and multiple archival mechanisms (e.g. NetCDF, HDF5, binary, etc.) • Advanced support for data assimilation systems, including data structures for observational data and adjoints for ESMF methods • Support for nested, moving grids and adaptive grids • Support for regridding in three dimensions and between different coordinate systems • Advanced optimization and load balancing

13. Open Source Development • Open source license (GPL) • Open source environment (SourceForge) • Open repositories: web-browsable CVS repositories accessible from the ESMF website http://www.esmf.ucar.edu • for source code • for contributions (currently porting contributions and performance testing) • Open development priorities and schedule: priorities set by users (previously during meetings and telecons, transition to new Change Review Board), web-browsable task lists • Open communication: frequent telecons and mailing list discussions • Open testing: ~1300 unit tests, ~17 system tests, ~40 examples are bundled with the ESMF distribution and can be run by users • Open port status: results of nightly tests on many platforms are web-browsable • Open metrics: test coverage, lines of code, requirements status are updated regularly and are web-browsable

14. ESMF Component Overhead • Measures overhead of ESMF superstructure in NCEP Spectral Statistical Analysis (SSI), ~1% overall • Run on NCAR IBM • Runs done by JPL staff, confirmed by NCEP developers

15. ESMF Current Challenges • Process and techniques for software requirements analysis, design and interface review with a VERY large distributed community • Installation of commercial software package (DOORS) for requirements archival and tracking, ties to testing and tasking • Implementation of process and format for collaborative design reviews earlier this year • Transition to multi-agency organizational structure • Development and review of a 5-year ESMF Project Plan, creation of an Interagency Working Group, Change Review Board and other bodies • Design strategy for advanced grids and regridding • Science and math hires to implement new grids and regridding methods • Clear, complete, carefully edited documentation and training program materials • Ongoing training program and continuing documentation efforts • New collaborations for technical and domain extension • ESMF on the GRID workshop this summer, interactions with CCA, PRISM and others

16. Learning More ESMF Website http://www.esmf.ucar.edu See this site for downloads, documentation, references, repositories, meeting schedules, test archives, and just about anything else you need to know about ESMF. References to ESMF source code and documentation in this tutorial correspond to ESMF Version 2.1.0. 4th ESMF Annual Community Meeting ESMF on the GRID Workshop ESMF Tutorial MIT Campus July 20-22, 2005 See the ESMF Website for meeting information and registration.