1 / 46

GEOS–CHEM, Transition to ESMF, and GEOS-5 met fields

GEOS–CHEM, Transition to ESMF, and GEOS-5 met fields. Bob Yantosca Software Engineer Atmospheric Chemistry Modeling Group Harvard University Group Meeting / Telecon 16 Nov 2005. Topics. Quick GEOS–CHEM Status Report ESMF Overview ESMF Design Principles ESMF Installation on Your Platform

arne
Download Presentation

GEOS–CHEM, Transition to ESMF, and GEOS-5 met fields

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GEOS–CHEM,Transition to ESMF,and GEOS-5 met fields Bob Yantosca Software Engineer Atmospheric Chemistry Modeling Group Harvard University Group Meeting / Telecon 16 Nov 2005

  2. Topics • Quick GEOS–CHEM Status Report • ESMF Overview • ESMF Design Principles • ESMF Installation on Your Platform • GEOS–5 Met Fields: A Quick Look NOTES: • ESMF = Earth System Model Framework • Some slides in this presentation were taken from Introduction to ESMF Tutorial and Introduction to the Earth System Modeling Framework from www.esmf.ucar.edu GEOS-CHEM, ESMF and GEOS-5

  3. Part 1 GEOS–CHEM Status Report GEOS-CHEM, ESMF and GEOS-5

  4. GEOS–CHEM Status • GEOS–CHEM v7–03–06 is the latest public release • v7–03–06 is now available for download from G–C web site. The manual pages have been updated. • Several additions, fixes, improvements have been incorporated into G–C from the last public version (v7–02–04, April 2005) • Here follows a laundry list of updates… GEOS-CHEM, ESMF and GEOS-5

  5. GEOS–CHEM Status • Met field updates in v7–03–06 • Now supports GISS met fields for GCAP simulations (S. Wu) • Added code to read GEOS–3 "XTRA" met fields (M. Fu) • i.e. PARDF, PARDR, SNOW • Snow depth is now used in GEOS–3 for dust emissions • New simulations in v7–03–06 • CO2 (P. Suntharalingam) • HCN / CH3CN (Y. Xiao) GEOS-CHEM, ESMF and GEOS-5

  6. GEOS–CHEM Status • Convection updates in v7–03–06 • Turn off scavenging in shallow convection for GCAP (S.Wu) • Diagnostic updates in v7–03–06 • Improvements to plane-flight diagnostic • Archives trop column AOD at location of aircraft • Archives column AOD from the surface to the aircraft location • Improvements to timeseries diagnostics • Minor fixes for AOD output in ND49 diagnostic • Timeseries diagnostics now can archive grid box height GEOS-CHEM, ESMF and GEOS-5

  7. GEOS–CHEM Status • Emissions updates in v7–03–06 • EMEP (anthro) European Emissions (M. Auvray, B. Field) • MEGAN (biogenic) emissions (M. Fu) • Rescaling of Lightning NOx AND • LNOx, DMS, sea-salt emissions now turned off @ poles(R. Hudman, R. Yantosca, B. Alexander) • Uses Nightingale et al 2000b formula for sea-air exchange (S. Wu) • For DMS, Acetone, CH3I, sea-salt, etc. emissions GEOS-CHEM, ESMF and GEOS-5

  8. GEOS–CHEM Status • Aerosol updates in v7–03–06 • Now uses ISORROPIA package to compute ATE (B. Alexander) • Added SO4s, NITs tracers into std simulation (R. Park, B. Alexander) • New oxidant fields for offline aerosol simulations (C. Heald, R. Yantosca) • From 1-yr benchmark v7–02–03 • Now allows for hygroscopic growth of aerosols in drydep(R. Park, B. Alexander) GEOS-CHEM, ESMF and GEOS-5

  9. GEOS–CHEM Status • Technical updates in v7–03–06 (R.Yantosca, B.Field) • Now modified for Intel Fortran Compiler “ifort” v9.0 • Can now to regrid emissions from GEOS 1x1 grid to: • 1x1 GEOS nested grids • 2x25 GEOS grid • 4x5 GEOS grid • 4x5 GCAP grid • Removed obsolete SLOW–J code • Removed obsolete CO–OH parameterization code • Removed several obsolete common block variables • G–C now writes diaginfo.dat, tracerinfo.dat files for GAMAP • You no longer need to manually edit these files anymore! GEOS-CHEM, ESMF and GEOS-5

  10. GEOS–CHEM Status • Benefits of v7–03–06 • Intel v9 compiler optimizes much better on the Altix • 1-month 4x5 fullchem run is ½ hr faster than Intel v7 • MEGAN biogenic inventory will eventually supplant GEIA • But for the time being we are keeping both • Housekeeping • Please migrate to v7–03–06 when possible! • Also be sure to download new data directories • From geos.as.harvard.edu GEOS-CHEM, ESMF and GEOS-5

  11. Part 2 ESMF Overview GEOS-CHEM, ESMF and GEOS-5

  12. What is ESMF? What is the Earth Systems Modeling Framework (ESMF)? • The ESMF is a focused community effort to tame the complexity of models and the computing environment. It leverages, unifies and extends existing software frameworks, creating new opportunities for scientific contribution and collaboration. Goals of ESMF: • Increase scientific productivity by making modeling and analysis software components much easier to build, combine, and exchange, and by enabling modelers to take full advantage of high-end computers. • Unify the national and international Earth system modeling community through a common modeling paradigm and regular interactions at all levels. GEOS-CHEM, ESMF and GEOS-5

  13. What is ESMF? • ESMF provides tools for turning model codes into componentswith standard interfaces and standard drivers • ESMF provides data structures and common utilities that components use • to organize codes • to improve performance portability • for common services such as data communications, regridding, time mgmt and message logging GEOS-CHEM, ESMF and GEOS-5

  14. With ESMF, you can reduce a CTM to a collection of “pluggable”black box “Components” with standard inputs & outputs What is ESMF? Diagnostics GEOS-CHEM, ESMF and GEOS-5

  15. ESMF is a Community Effort • Collaborators and customers include: • NSF NCAR • NOAA GFDL, NOAA NCEP • DOE LANL, DOE ANL • NASA GMAO, NASA Land Information Systems, NASA GISS • DoD Navy, Air Force, and Army • University of Michigan, UCLA, MIT • Development & testing is open-source • Website: www.esmf.ucar.edu • Email: esmf_support@ucar.edu GEOS-CHEM, ESMF and GEOS-5

  16. Why Should I Adopt ESMF if I already have a working model? • There is an emerging pool of other ESMF-based science components that you will be able to interoperate with to create applications - ESMF has a broad customer base. • It will reduce the amount of infrastructure code that you need to maintain and write, and allow you to focus more resources on science development. • ESMF provides solutions to two of the hardest problems in model development: structuring large, multi-component applications so that they are easy to use and extend, and achieving performance portability on a wide variety of parallel architectures. • It may be better software (?!) than the infrastructure software that you are currently using. • Community development and use means that the ESMF software is widely reviewed and tested, and that you can leverage contributions from other groups. GEOS-CHEM, ESMF and GEOS-5

  17. Why Should I Adopt ESMF if I already have a working model? • NASA now “requires” that all of its Earth sciences modeling assets conform to the Earth System Modeling Framework • This is being done so that “pieces” of GEOS–CHEM (or other models) can be inserted into larger Earth System Models • Future funding may be dependent on the ability to make models ESMF-compliant GEOS-CHEM, ESMF and GEOS-5

  18. ESMF Overview Summary • What is ESMF? • It’s a software framework for earth science models that NASA et al is espousing • What ESMF will do: • It will make it easier to connect “pieces” of the same model model together • It will make it easier to take “pieces” of one model and mix them with “pieces” from other models. • It will probably eventually reduce model development time (however, some up-front development will be needed) • What ESMF will not do: • It will NOT improve the quality of the science of a model GEOS-CHEM, ESMF and GEOS-5

  19. Part 3 ESMF Design Principles GEOS-CHEM, ESMF and GEOS-5

  20. Classes and Objects in ESMF • The ESMF Application Programming Interface (API) is based on the object-oriented programming notion of a class. • A class is a software construct that’s used for grouping a set of related variables together with the subroutines and functions that operate on them. • A particular instance of a class is called an object. • Think of a class as being a “data type” and an object as being a specific variable of that particular data type. • Our goal: convert existing model code into ESMF Components by using the ESMF class structure. GEOS-CHEM, ESMF and GEOS-5

  21. Classes and Objects in ESMF • There are two software layers in ESMF • Infrastructure • Superstructure • Infrastructure classes define the data types which are ESMF’s “building blocks” • Array, Field, Bundle, etc … • Superstructure classes define the overall function of ESMF Components • Gridded Component, Coupler Components, “main” GEOS-CHEM, ESMF and GEOS-5

  22. ESMF Infrastructure Data Classes ESMF Array = a data container (similar to an F90 array) It can contain data and may be up to 7 dimensions. ESMF Array ESMF Grid = definition of a physical grid(latitude, longitude, and vertical dimensions) ESMF Grid An ESMF Field is the combination of an ESMF array with an ESMF grid. Therefore, an ESMF field is like a “smart” array. It can tell you not only about its data but also about what kind of grid it is defined on. ESMF Fields can be used to store physical quantities (i.e. Pressure, Temperature, Winds, Humidity, etc.) ESMF Array ESMF Grid ESMF Field GEOS-CHEM, ESMF and GEOS-5

  23. ESMF Infrastructure Data Classes An ESMF Bundle is the combination of more than one ESMF Field. Bundles make it easy to group fields together in a logical fashion ESMF Field #1 ESMF Field #2 ESMF Field #3 . . . ESMF Bundle An ESMF State is the combination of one or more bundles. States are passed between ESMF Gridded Components. Each Gridded Component takes one Input State and one Output State as arguments. ESMF Bundle #1 ESMF Bundle #2 ESMF Bundle #3 . . . ESMF State GEOS-CHEM, ESMF and GEOS-5

  24. ESMF Infrastructure Data Classes • Other Infrastructure utilities: • Time Manager and Calendar Functions • Configuration Attributes (replaces namelists) • Message logging • Communication libraries • Regridding library (parallelized, on-line SCRIP) • IO (barely implemented) • Performance profiling GEOS-CHEM, ESMF and GEOS-5

  25. ESMF Superstructure Classes Schematic of an ESMF Gridded Component Input State Output State ESMF Clock Return code Each ESMF Gridded Component must have init, run, finalize routines. Init takes data from the Input state and allocates all internal data storage Run is used to invoke the user source code Finalize deallocates all internal storage. ESMF Superstructure Layer User Code Init Routine User Code Run Routine User Code Finalize Routine ESMF library function calls Internal F90 code routines Internal F90 code routines Internal F90 code routines ESMF Infrastructure Layer GEOS-CHEM, ESMF and GEOS-5

  26. Design Strategy: Modularity Gridded Components don’t have access to the internals of other Gridded Components, and don’t store any coupling information. Gridded Components pass their States to other components through their argument list. Since components are not hard-wired into particular configurations and do not carry coupling information, components can be used more easily in multiple contexts. NWP application NOTE: the same ESMF Component can be “plugged” into several different types of simulations Seasonal prediction Standalone for basic research atm_comp No more global variables! GEOS-CHEM, ESMF and GEOS-5

  27. ESMF Superstructure Classes Schematic of an ESMF Coupler Component ESMF Coupler Component P Comp #1 P, T Comp #3 T In the Coupler, the programmer is responsible for connecting data (e.g. P,T) from the output states of one or more Components with the input state of another Component (and vice versa). Comp #2 Input States Output States Internal Linkages GEOS-CHEM, ESMF and GEOS-5

  28. Design Strategy: Hierarchical Applications Since each ESMF application is also a Gridded Component, entire ESMF applications can be nested within larger applications. This strategy can be used to systematically compose very large, multi-component codes. Example: GEOS–5 assimilation system. GEOS-CHEM, ESMF and GEOS-5

  29. ESMF Class Structure GridComp Land, ocean, atm, … model CplComp Xfers between GridComps State Data imported or exported Superstructure Infrastructure Regrid Computes interp weights Bundle Collection of fields Field Physical field, e.g. pressure Grid LogRect, Unstruct, etc. PhysGrid Math description DistGrid Grid decomposition F90 Array Hybrid F90/C++ arrays DELayout Communications Route Stores comm paths C++ Utilities Virtual Machine, TimeMgr, LogErr, IO, ConfigAttr, Base etc. Communications Data GEOS-CHEM, ESMF and GEOS-5

  30. Things to be happy about • GEOS–CHEM is in good shape for ESMF conversion • Most new G–C code is in F90 module form • Each F90 module has INIT and CLEANUP routines • G–C is highly “methodized” • Many quantities are accessible via function calls • Complexity is moved to functions rather in main code • G–C has consistent file I/O and error handling • G–C code is very well documented • We may be able to leverage other efforts • GMI is also converting to the ESMF framework • Tom Clune is writing an ESMF replacement for SMVGEAR GEOS-CHEM, ESMF and GEOS-5

  31. Things to be aware of • ESMF conversion will require time and effort • ESMF library function calls are very “wordy” • You have to use a lot of function calls to do stuff • G–C’s legacy code routines will have to be rewritten: • Full-chemistry emissions • Drydep ? • FAST–J photolysis (this is 3rd party code) • Diagnostics • Conversion of G–C to ESMF will have to be in parallel with: • Normal G–C user support • GEOS–5 met field implementation • We are looking to hire a 2nd FTE for programming support GEOS-CHEM, ESMF and GEOS-5

  32. Part 4 ESMF Installation GEOS-CHEM, ESMF and GEOS-5

  33. ESMF Platform Support System requirements include: • A Fortran 90 or later compiler • C and C++ compilers (proprietary or gcc/g++) • a MPI implementation compatible with these compilers OR an MPI-bypass library (which is bundled with the ESMF source distribution) • The gmake utility (for building the code) • The tar and gzip utilities (for unpacking data files) • The Perl programming language, for running test scripts • Latex, latex2html, and the Unix/Linux dvipdf utility are needed to build the ESMF documentation (not required to build code) GEOS-CHEM, ESMF and GEOS-5

  34. ESMF Platform Support • IBM AIX (32 and 64 bit) • SGI IRIX64 (32 and 64 bit) • SGI Altix (64 bit) • Cray X1 (64 bit) • Compaq OSF1 (64 bit) • Linux Intel (32 and 64 bit) • Linux PGI (32 bit) • Linux NAG (32 bit) • Linux Absoft (32 bit) • Linux Lahey (32 bit) • Mac OS X with xlf (32 bit) GEOS-CHEM, ESMF and GEOS-5

  35. ESMF Platform Support CAVEATS: • Only the most recent compilers may be supported • SGI: Probably need at least SGI MIPS 7.4 compiler • Intel: Need at least Intel v8; it works w/ Intel v9 • *** BE PREPARED TO UPDATE YOUR COMPILERS *** • Installation is heavily weighted towards proprietary C and C++ compilers • Bob Y. had difficulties trying to install with the “free” gcc/g++ compilers at Harvard • If your system is unsupported, you’re out of luck • Complain to esmf_support@ucar.edu GEOS-CHEM, ESMF and GEOS-5

  36. ESMF Platform Support • My ESMF Installation story – version 2.2 • I couldn’t install on SGI with v7.2.1.3 compiler • So I gave up on that platform • I couldn’t install on Altix with Intel v7.1 compiler • Intel v7.1 compiler was “too old” for ESMF • I bugged Jack to install Intel Compiler v9.0 • Intel 9.0 compiler installation took a while (licensing issues) • My 1st attempt to install w/ Intel v9 didn’t work • ESMF source code had bugs • ESMF issued a software patch, I recompiled w/ that • Finally, success!!!! GEOS-CHEM, ESMF and GEOS-5

  37. Part 5 GEOS–5: A Quick Look GEOS-CHEM, ESMF and GEOS-5

  38. GEOS–5 Quick Look • GEOS–5 is the next met field product from GMAO • Slated to become operational after INTEX-B mission • Resolution will be significantly higher • 0.5 (lat) x 0.625 (lon) x 72 vertical layers • Grid will be in Lagrangian control volume coords • Different than GEOS–4 • You don’t use a formula to compute P(I,J,L) • You carry around a 3-D pressure field to compute P(I,J,L) GEOS-CHEM, ESMF and GEOS-5

  39. GEOS–5 Quick Look • Concerns • GEOS–5 “raw” data volume per day is huge!! • We may have to get an account on NASA machines for data processing • LCV coordinates are not compatible with existing G–C code • LCV coordinates are not compatible with existing TPCORE • May need to modify TPCORE • S–J Lin is out of the TPCORE business • Not sure what the ramifications for mass-conservation are when we switch from ETA to LCV coordinates • It would be easier for us if GMAO made GEOS–5 data on the same grid as GEOS–4 • This may be possible if we squawk loud enough  GEOS-CHEM, ESMF and GEOS-5

  40. GEOS–5 Quick Look GEOS-5 A-3 (3hr time-avg'd) MET FIELDS REQUIRED FOR GEOS-CHEM =============================================================================== Name Description Units Where it's used ---- ----------- ----- --------------- ALBEDO Visible surface albedo [unitless] drydep + as a snow/ice flag CLDTOT 2-D column cloud fraction [unitless] drydep GWETTOP Topsoil wetness [unitless] dust mobilization HFLUX Sensible heat flux [W/m2 ] drydep LAI GMAO leaf area index [m2/m2 ] for comparison purposes PARDF Diffuse PAR [W/m2 ] new isoprene emission code PARDR Direct PAR [W/m2 ] new isoprene emission code PBLH PBL heights [m ] BL mixing, drydep, etc. PRECTOT Total accum. precip [kg/m2/s ] wetdep PRECCON Total conv. precip [kg/m2/s ] wetdep RADLWG Net upward LW rad @ ground [W/m2 ] dust mobilization RADSWG New downwd SW rad @ ground [W/m2 ] dust mobilization SNOWMAS Snow depth -- H2O equiv. [kg/m2 ] dust mobilization T2M Temp @ 2m altitude [K ] proxy for surface air. temp TSKIN Ground or Sea Surf. Temp. [K ] new isoprene emission code U10M U-wind @ 10m altitude [m/s ] drydep, NOx em, acetone em. USTAR Friction velocity [m/s ] drydep V10M U-wind @ 10m altitude [m/s ] drydep, NOx em, acetone em. Z0M Roughness Height [m ] drydep GEOS-CHEM, ESMF and GEOS-5

  41. GEOS–5 Quick Look GEOS-5 A-6 (6hr time-avg'd) FIELDS REQUIRED FOR GEOS-CHEM =============================================================================== Name Description Units Where it's used ---- ----------- ----- --------------- CMFMC Cloud mass flux [kg/m2/s ] convection DTRAIN Cloud detrainment flux [kg/m2/s ] convection CLOUD 3-D cloud fraction [unitless] diagnostic output DQVDTMST Tendency in sp. humidity [g/kg/day] wetdep OPTDEP Total grid box opt. depth [unitless] photolysis (J-values) QV Specific Humidity [kg/kg ] full chemistry + wetdep T Temperature [K ] everywhere U Zonal winds [m/s ] Transport V Meridional winds [m/s ] Transport We will probably also pull OMEGA (vertical velocity) Pa/s GEOS-CHEM, ESMF and GEOS-5

  42. GEOS–5 Quick Look GEOS-5 I-6 (6-hr inst) FIELDS REQUIRED FOR GEOS-CHEM =============================================================================== Name Description Units Where it's used ---- ----------- ----- --------------- LWI Land-water flags [unitless] to denote water/land/ice PS Surface pressure [Pa] used everywhere TROPP Tropopause pressure [Pa] for trop height SLP Sea-level pressure [Pa] for diagnostics GEOS-CHEM, ESMF and GEOS-5

  43. Conclusions • v7–03–06 is now the latest std code release • Please upgrade!!!! • GEOS–CHEM will be converted to ESMF • This will make things easier in the long run • Conversion process will take time • GEOS–5 will be the operational data product in 2006 • However there are several questions we still have • Not sure to what extent G–C may need to be modified • A thorough validation will be necessary GEOS-CHEM, ESMF and GEOS-5

  44. Extra Slides GEOS-CHEM, ESMF and GEOS-5

  45. Design Strategy:Intracomponent Communication All communication in ESMF is handled within components. This allows the architecture of the framework to be independent of the communication strategy. The result is that there is flexibility in implementation of communications and component drivers are straightforward. climate_comp As a consequence, Coupler Components must be defined on the union of all processors and all the Gridded Components that they couple. In this example, in order to send data from the atm Component (blue) to the ocean Component (green), the atm2ocn_coupler mediates the send. atm2ocn _coupler atm_comp ocn_comp phys2dyn_coupler atm_phys atm_dyn GEOS-CHEM, ESMF and GEOS-5

  46. Design Principle: Scalable Applications Since each ESMF application is also a Component, entire ESMF applications can be treated as Gridded Components and nested within larger applications. climate_comp Example: atmospheric application (in blue) itself composed of multiple Components may be run standalone, or nested within a larger climate application ocn2atm_coupler ocn_comp atm_comp phys2dyn_coupler atm_phys atm_dyn GEOS-CHEM, ESMF and GEOS-5

More Related