1 / 48

GEOS–CHEM: Code Management and Benchmarking

GEOS–CHEM: Code Management and Benchmarking. Bob Yantosca Software Engineer Atmospheric Chemistry Modeling Group Harvard University GEOS-CHEM Users’ Meeting 04 April 2005. Topics. Recent history of GEOS–CHEM versions Procedure for updating GEOS–CHEM Rules and Responsibilities

raja
Download Presentation

GEOS–CHEM: Code Management and Benchmarking

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GEOS–CHEM:Code Management and Benchmarking Bob Yantosca Software Engineer Atmospheric Chemistry Modeling Group Harvard University GEOS-CHEM Users’ Meeting 04 April 2005

  2. Topics • Recent history of GEOS–CHEM versions • Procedure for updating GEOS–CHEM • Rules and Responsibilities • GEOS–CHEM 1–Year Benchmarks • Future directions • Appendix: Downloading GEOS–CHEM GEOS-CHEM: Code Management and Benchmarking

  3. Part 1 GEOS–CHEM Recent History GEOS-CHEM: Code Management and Benchmarking

  4. Recent History • GEOS–CHEM versions are named vX–YY–ZZ • X is the major version number • This is changed when a new GEOS–CHEM version contains an update that is so revolutionary that it is incompatible with all previous versions. • YY is the the minor version number • This is changed each time the model is released publicly on the GEOS–CHEM website. • ZZ is the release number • This is changed for each “internal release”. Very often when we have many updates to add to GEOS–CHEM; these will be added in several stages so that each internal release can be tested separately. Then after all internal releases have been tested, a public release will be made. GEOS-CHEM: Code Management and Benchmarking

  5. Recent History • GEOS–CHEM version numbers throughout history: • v1–YY, v2–YY, and v3–YY (all prior to 2000) • Ancient history • v4–YY (2000 – 2003) • Denotes the ability to run with GEOS–3 met fields • v5–YY–ZZ (2003) • Denotes the installation of SMVGEAR II • v6–YY–ZZ (2004) • Denotes the ability to run with GEOS–4 met fields • v7–YY–ZZ (Oct 2004 – present) • Denotes new and much improved user interface, updated diagnostics, new emissions, etc… GEOS-CHEM: Code Management and Benchmarking

  6. Recent History • In the last year, GEOS–CHEM development has been particularly rapid. This was mostly driven by: • The need to make G–C compatible with GEOS–4 met fields • The need to add science into G–C for the Near-Real-Time simulations that we would produce for ICARTT • GEOS–4 met fields: • Are produced w/ a different GCM than prior GEOS versions • Are on a hybrid grid and not a pure-sigma grid • Have a different time-stamping scheme than prior GEOS v’s • Required substantial recoding of GEOS–CHEM GEOS-CHEM: Code Management and Benchmarking

  7. Recent History • Updates required for GEOS–4 meteorology • Installed new fvDAS TPCORE transport code(B. Field) • Mass-flux diagnostics had to be totally rewritten • Installed LLNL pressure-fixer for transport (B. Field) • P-fixer “modifies” U & V so that the surface pressure computed from the winds matches the met field pressure read from disk • Installed new convection code from NCAR(S. Wu) • GEOS–4 convection fields are different than GEOS–3 • New scheme (from MATCH) includes updrafts & downdrafts • GEOS–4 also required coding changes in several places… GEOS-CHEM: Code Management and Benchmarking

  8. Recent History • Important Scientific Updates to GEOS–CHEM • Addition of aerosol tracers for full-chemistry & offline runs • Hydrophilic & hydrophobic BC & OC (R. Park) • Secondary organic aerosols (H. Liao & S. Chung, Caltech) • Sea salt aerosols (B. Alexander) • Dust aerosols (D. Fairlie) • New types of emissions • EPA/NEI99 inventory for North America (R. Park & R. Hudman) • NOx, CO, ALK4, ACET, MEK, PRPE, C2H6, C3H8, CH2O, SO2, and SO4 • SO2 emissions from ship exhaust (B. Alexander) • AVHRR LAI can be turned on or off (M. Fu) GEOS-CHEM: Code Management and Benchmarking

  9. Recent History • Coding updates and improvements to GEOS–CHEM • Totally new user interface in v7-yy-zz • All GEOS–CHEM switches are now set in one input file • Timeseries diagnostics updated for ICARTT • Rewritten for consistency • Parallelized with OpenMP for optimal execution • GEOS–CHEM was ported to the SGI/Altix platform • Wetdep and cloud convection now parallelize on Altix • Other improvements • Production & loss diagnostics were cleaned up • Use more allocatable arrays to save memory • Removing LOTS of “historical baggage” GEOS-CHEM: Code Management and Benchmarking

  10. Recent History • Ongoing GEOS–CHEM development … • Mercury simulation (w/ U. Wash) • Recoding for MPI parallelization (w/ JPL) • Creation of a GEOS–CHEM adjoint (w/ U. Toronto, Caltech) • Aerosol microphysics modules (w/ CMU) • Interface with CMAQ model (w/ U. Houston, U. Tenn) • Online data assimilation (w/ GMAO) • GCAP: Interface with GISS–GCM winds (w/ GISS) • And many other projects!! • This requires that we have a clearly-defined procedure for updating GEOS–CHEM!! GEOS-CHEM: Code Management and Benchmarking

  11. Part 2 Updating GEOS–CHEM GEOS-CHEM: Code Management and Benchmarking

  12. Updating GEOS–CHEM • GEOS–CHEM “standard code” is kept at Harvard • Bob Y. is the “Model Programmer” and maintains the “standard” GEOS–CHEM code • Version control is done with a CVS source code repository • Changes from the last released version are documented • All changes to the “standard code” must go through Bob Y. • This prevents divergent versions from floating around • This also ensures the code will be computationally optimized • Each “standard code” release is benchmarked with a 1-month full-chemistry simulation (more on this later) • For some versions we also run a 1-year benchmark simulation GEOS-CHEM: Code Management and Benchmarking

  13. Updating GEOS–CHEM GEOS–CHEM modification process: • Users start by downloading a “standard code” version • Available via GEOS–CHEM web site (see Appendix) • Users add new science into GEOS–CHEM (and test it!) • Please test for 2x2.5, 4x5 (and maybe 1x1) grids • Please test for different met fields (e.g. GEOS–3, GEOS–4) • When revisions are mature, users submit updates to Bob Y. for inclusion in the next “standard code” release • Do not submit unless code is 100% ready to go • Please also include any relevant documentation and required data or other input files GEOS-CHEM: Code Management and Benchmarking

  14. Updating GEOS–CHEM GEOS–CHEM modification process: • Bob Y. will add users’ revisions into the “standard code” • Bob Y. will test the “standard code” with a 1–month full-chemistry benchmark simulation • Benchmark month is 4 x 5 GEOS–3 for July 2001 • Daniel Jacob must approve the 1-month benchmark results • Bob Y. will release the “standard code” and update the documentation on the web site • Notification will be made via GEOS–CHEM mailing list GEOS-CHEM: Code Management and Benchmarking

  15. Updating GEOS–CHEM • Full chemistry benchmark (July 2001) • GEOS-3 meteorology w/ 30 layers @ 4x5 • The following types of plots are created: • Tracer ratios w/ previous version (surface, 500hPa) • Frequency distribution of tracer and AOD ratios • J-Value ratios w/ previous version (surface, 500hPa) • Tracer concentrations (surface, 500hPa) and zonal means • Budgets for Ox and CO; CH3CCl3 lifetime • Emissions table (comparison to previous version) • Aerosol optical depths at surface • Aerosol optical depth ratios (comparison to previous version) • Results are compared to the previously released version GEOS-CHEM: Code Management and Benchmarking

  16. Part 3 Rules and Responsibilities GEOS-CHEM: Code Management and Benchmarking

  17. Rules and Responsibilities • GEOS–CHEM user responsibilities • Maintain communication with other users • Update to new “standard code” versions when available • Do not distribute non-standard code versions • Give credit to developers in publications • Consult documentation pages on the G–C website • Become familiar with debugging techniques • Be resourceful! Try to isolate locations in the code where bugs or errors are encountered before reporting to Bob Y. GEOS-CHEM: Code Management and Benchmarking

  18. Rules and Responsibilities • GEOS–CHEM developer responsibilities • Start from the latest available “standard code” release • Provide new code for future “standard code” releases • Code should adhere to GEOS–CHEM style guide • Provide related data and/or input files: • For 1x1, 2x25 and 4x5 horizontal grids • For GEOS–1, GEOS–STRAT, GEOS–3, and GEOS–4 vert. grids • Also provide any IDL routines etc. used to create these data • Provide PROFUSE documentation for new code & data files! • Your humble “Model Programmer” isn’t a chemist  • Think of future users when writing documentation • Journal references are very helpful to include in source code! GEOS-CHEM: Code Management and Benchmarking

  19. Rules and Responsibilities • GEOS–CHEM users and developers should familiarize themselves with the following: • Fortran 90 • GEOS–CHEM is written in this language • IDL • Visualization tools (e.g. GAMAP) are written in IDL • netCDF, HDF, HDF–EOS files can be read with IDL • Some shell scripting (e.g. tcsh) • Advanced users: Learn Perl! GEOS-CHEM: Code Management and Benchmarking

  20. Rules and Responsibilities • Why is GEOS–CHEM written in Fortran? • Fortran is the de-facto standard for scientific computing • Most existing scientific code is written in Fortran • Fortran is much easier to learn than C, C++, Java, etc. • C can be cryptic; Fortran is more English-like • Fortran was designed for scientific computation • Fortran has more math functions & data types than C • Fortran code optimizes better than C code • Most Fortran compilers are OpenMP compatible • Fortran executes faster than IDL • Fortran 90 brings many improvements over Fortran 77 • You can write structured code with modules GEOS-CHEM: Code Management and Benchmarking

  21. Part 4 1–Year Benchmarks GEOS-CHEM: Code Management and Benchmarking

  22. 1–Year Benchmarks • 1–month GEOS–CHEM benchmarks are useful for: • Tracking changes from one revision to the next • Discovering coding errors • Ensuring code is ported correctly to new platforms • Shortcomings of 1–month benchmarks • 1 month is often not long enough for certain species • It is a model-to-model comparison; not model-to-data GEOS-CHEM: Code Management and Benchmarking

  23. 1–Year Benchmarks • At the last GEOS–CHEM meeting it was decided to also incorporate a 1–year GEOS–CHEM benchmark • GEOS–CHEM versions are compared against observed data • 1–year benchmarks are performed when: • A new version of GEOS–CHEM contains a significant scientific change from a previous version • A new set of GMAO met fields is released and we need to compare how GEOS–CHEM performs w/r/t the previous met fields (e.g. GEOS–4 vs. GEOS–3) GEOS-CHEM: Code Management and Benchmarking

  24. 1–Year Benchmarks • Description of 1–year benchmark simulations: • Coupled NOx–Ox–HC–aerosol simulation • Simulation year: 2001 (with 2000 for spin-up) • Horizontal Resolution: 4° latitude x 5° longitude • Vertical Resolution: • GEOS–3: 30 level pure-sigma grid (lumped from 48 levels) • GEOS–4: 30 level hybrid grid (lumped from 55 levels) • Gas-phase chemical tracers: • NOx, Ox, PAN, CO, Lumped >=C4 Alkanes, ISOP, HNO3, H2O2, MEK, Acetaldehyde, Lumped Aldehyde >=C3, MVK, Methacrolein, PMN, PPN, Lumped Alkyl Nitrate, >=C3 Alkenes, C3H8, HCHO, C2H6, N2O5, HNO4, Methyl Hydro Peroxide GEOS-CHEM: Code Management and Benchmarking

  25. 1–Year Benchmarks • Description of 1–year benchmark simulations: • Aerosol tracers: • DMS, SO2, SO4, MSA, NH3, NH4, NIT, Hydrophilic BC, Hydrophilic OC, Hydrophobic BC, Hydrophobic OC, Dust Aerosol (4 size bins), Sea Salt Aerosol (2 size bins) • Secondary Organic Aerosols: ALPH, LIMO, ALCO, SOA1, SOA2, SOA3, SOG1, SOG2, SOG3 • Chemical Mechanism: • SMVGEAR II (Jacobson et al, 1997), using ~90 chemical species, ~300 kinetic reactions (including rapid recycling of peroxides as well as H2SO4—HNO3—NH3 chemistry), and 51 photolysis reactions • Photolysis Mechanism: • FAST–J algorithm (Wild et al, 2000) GEOS-CHEM: Code Management and Benchmarking

  26. 1–Year Benchmarks • Description of 1–year benchmark simulations: • Operations: • Transport (Lin & Rood, 1996, 1997) • Cloud Convection • GEOS–3: (Lin, 1996) • GEOS–4: (Rasch, 2001) • Boundary Layer Mixing (Allen, 1996) • Emissions • Dry Deposition (Jacob et al, 1995) • Chemistry • SMVGEAR II + aerosol chemistry schemes • Wet Deposition (Jacob et al, 2000) GEOS-CHEM: Code Management and Benchmarking

  27. 1–Year Benchmarks • Description of 1–year benchmark simulations: • Diagnostic output (saved as monthly means): • Optical Depths and Noontime J(O3), J(NO2) • Anthropogenic, biomass, and biogenic emission fluxes • Dry deposition fluxes and velocities • Sources of CO, NOx, and Acetone • Concentrations of chemically produced OH, NO, NO2, HO2 • Tracer concentrations • Chemical production & loss of Ox and CO • Lifetime of Methylchloroform (CH3CCl3) • Selected meteorological quantities (U, V, P, T, etc.) • Air masses, grid box heights GEOS-CHEM: Code Management and Benchmarking

  28. 1–Year Benchmarks • Description of 1–year benchmark simulations: • The following plots are created: • Model vs. aircraft data (several missions) for: • C2H6, C3H8, CO, H2O2, HNO3, NO, O3, PAN • Model CO vs. MOZAIC CO observations • Model CO vs. ship track data • Model column CO from vs. observations • Model CO vs. CMDL CO • Model O3 vs. MOZAIC O3 observations • Model O3 vs. ozonesonde data • Model O3 vs. surface O3 observations GEOS-CHEM: Code Management and Benchmarking

  29. 1–Year Benchmarks • Results from different 1–year benchmark simulations are usually plotted together against observations • Jennifer Logan is in charge of examining 1–year benchmark results • Jennifer will speak about our most recent 1–year benchmark simulations later today. GEOS-CHEM: Code Management and Benchmarking

  30. Part 5 Future Directions GEOS-CHEM: Code Management and Benchmarking

  31. Future Directions • v7–02–04 will be released shortly after this mtg • Option to use EPA / NEI 1999 emissions over USA • Mercury simulation with online ocean fluxes • PBL mixing has been centralized into a single module • Planeflight diag. can now read daily files (a la ICARTT) • Various bug fixes / improvements • GAMAP v2–03 will also be released shortly • Can now read/write to COARDS-compliant netCDF files • Other various improvements and minor fixes GEOS-CHEM: Code Management and Benchmarking

  32. Future Directions • Earth System Model Framework (ESMF) • Is a package of libraries which define “couplings” between models and/or model components • If two models are ESMF compliant then they can “talk” to each other using a common interface • Individual components in the same model may also connect to each other via ESMF interfaces • GEOS–CHEM is amenable to ESMF compliance • It is not ESMF compliant at this moment but… • We will move towards ESMF when the need arises • GEOS–CHEM coding style does not preclude ESMF GEOS-CHEM: Code Management and Benchmarking

  33. Part 6 Appendix GEOS-CHEM: Code Management and Benchmarking

  34. Downloading GEOS–CHEM • We distribute GEOS–CHEM as a complete software package, with the following components: • Source code and Makefiles • Run directories containing user-modifiable input files • Data directories containing emissions etc. files • Directories containing meteorological field files • Web-based documentation • Visualization tools (GAMAP) • These components are available via the Internet • GEOS–CHEM website • FTP server (geos.as.harvard.edu) GEOS-CHEM: Code Management and Benchmarking

  35. Downloading GEOS–CHEM • Source Code • Is available from the GEOS-CHEM Website • http://www-as.harvard.edu/chemistry/trop/geos/ • Click on Model Versions / Get Code and Data in menu • You must supply the proper password • Click on “1-month full chemistry benchmark run package” • This will download GEOS-CHEM.v7-xx-yy.stdrun.tar.gz • Uncompress the file with: • gunzip GEOS-CHEM.v7-xx-yy.stdrun.tar.gz • Extract the contents with: • tar –xvf GEOS-CHEM.v7-xx-yy.stdrun.tar GEOS-CHEM: Code Management and Benchmarking

  36. Downloading GEOS–CHEM • GEOS-CHEM.v7-xx-yy.stdrun.tar contains • Code.v7-xx-yy: Dir w/ GEOS–CHEM source code • IDL:Dir w/ IDL routines used to print benchmark results • run.fullchem.geos3:Run directory with input files and 1-month benchmark results (more on this later!) • GEOS—CHEM supported compilers • SGI MipsPro • Linux: Intel (both PC and Altix versions) and PGI • HP/Compaq • IBM/AIX • Sun/SPARC GEOS-CHEM: Code Management and Benchmarking

  37. Downloading GEOS–CHEM • Run directories • Are also available from the GEOS-CHEM Website • http://www-as.harvard.edu/chemistry/trop/geos/ • Click on Model Versions / Get Code and Data in menu • Supply the user password • Click on “4x5 run directories” or “2x25 run directories” • GEOS-CHEM.v7-xx-yy.rundir4x5.tar.gz • GEOS-CHEM.v7-xx-yy.rundir2x25.tar.gz • Uncompress the files with: • gunzip GEOS-CHEM.v7-xx-yy.rundir4x5.tar.gz, etc… • Extract the contents with: • tar –xvf GEOS-CHEM.v7-xx-yy.rundir4x5.tar, etc… GEOS-CHEM: Code Management and Benchmarking

  38. Downloading GEOS–CHEM • GEOS-CHEM.v7-xx-yy.rundir4x5.tar contains: • run.geos1.4x5: Dir w/ input files for 4x5 GEOS–1 • run.geoss.4x5: Dir w/ input files for 4x5 GEOS–STRAT • run.geos3.4x5: Dir w/ input files for 4x5 GEOS–3 • run.geos4.4x5: Dir w/ input files for 4x5 GEOS–4 • GEOS-CHEM.v7-xx-yy.rundir2x25.tar contains • run.geos1.2x25: Dir w/ input files for 2x2.5 GEOS–1 • run.geoss.2x25: Dir w/ input files for 2x2.5 GEOS–S • run.geos3.2x25: Dir w/ input files for 2x2.5 GEOS–3 • run.geos4.2x25: Dir w/ input files for 2x2.5 GEOS–4 GEOS-CHEM: Code Management and Benchmarking

  39. Downloading GEOS–CHEM • Important input files contained in the GEOS–CHEM run directories: • File for setting GEOS–CHEM simulation options: • input.geos • For SMVGEAR chemistry mechanism: • globchem.dat, mglob.dat, chemga.dat • For FAST-J photolysis: • ratj.d, jv_spec.dat, jv_atms.dat • Files containing initial concentrations • restart.YYYYMMDDHH (e.g. restart.2001010100) GEOS-CHEM: Code Management and Benchmarking

  40. Downloading GEOS–CHEM • GEOS–CHEM data directories • Available via FTP download from geos.as.harvard.edu • Contact Jack Yatteau (jhy@io.harvard.edu) for access • These directories contain “shared” data files (e.g. emissions, etc.) which are too large to fit in individual users’ space • Data directory structure on geos.as.harvard.edu: • /data/GEOS_4x5/: Root-level dir for 4x5 simulation • /data/GEOS_2x2.5/: Root-level dir for 2x2.5 simulation • /data/GEOS_1x1_CH/: Root-level dir for 1x1 China sim. • /data/GEOS_1x1_NA/: Root-level dir for 1x1 N. Am. sim. • Data files are grouped into subdirectories by category GEOS-CHEM: Code Management and Benchmarking

  41. Downloading GEOS–CHEM • GEOS–CHEM met field dirs on geos.as.harvard.edu • 4x5 met field directories: • /data/GEOS_4x5/GEOS_1/YYYY/MM/ • /data/GEOS_4x5/GEOS_S/YYYY/MM/ • /data/GEOS_4x5/GEOS_3/YYYY/MM/ • /data/GEOS_4x5/GEOS_4_v4/YYYY/MM/ • 2 x 2.5 met field directories: • /data/GEOS_4x5/GEOS_1/YYYY/MM/ • /data/GEOS_4x5/GEOS_S/YYYY/MM/ • /data/GEOS_4x5/GEOS_3/YYYY/MM/ • /data/GEOS_4x5/GEOS_4_v4/YYYY/MM/ GEOS-CHEM: Code Management and Benchmarking

  42. Downloading GEOS–CHEM • GEOS–CHEM met field dirs on geos.as.harvard.edu • 1x1 Nested Grid met field directories: • /data/GEOS_1x1_CH/GEOS_3/YYYY/MM/ • /data/GEOS_1x1_NA/GEOS_3/YYYY/MM/ • NOTES: • GEOS–1, GEOS–STRAT, GEOS–3 data files are gzipped • Each month of GEOS–4 data files are stored as a gzipped TAR file in order to save disk space GEOS-CHEM: Code Management and Benchmarking

  43. Downloading GEOS–CHEM • Visualization tools • GAMAP: Global Atmospheric Model Analysis Package • Suite of routines written in IDL • Requires IDL version 5.3 or higher • GAMAP routines can be downloaded via the web • http://www-as.harvard.edu/chemistry/trop/gamap/ • Online documentation is also available at this website • Advantages of using GAMAP • Tested on both PC’s and Unix machines; is very stable • You can use GAMAP routines with other IDL code • IDL has very powerful mapping/imaging capabilities • Routines are provided for reading HDF, HDF–EOS, netCDF files GEOS-CHEM: Code Management and Benchmarking

  44. Downloading GEOS–CHEM • Advantages of using GAMAP • GAMAP contains a user-friendly menu-driven interface • GAMAP has been tested on PC’s and Unix machines • You can combine GAMAP routines with other IDL code • Advantages of using IDL • IDL has very powerful mapping/imaging capabilities • IDL has routines to read HDF, HDF–EOS, netCDF files • You can develop an interactive web interface for GAMAP with IDL–ION software (sold separately!) • See our near-real-time web page (L. Jaegle) GEOS-CHEM: Code Management and Benchmarking

  45. Downloading GEOS–CHEM • Web-based documentation • Available through the GEOS–CHEM website • http://www-as.harvard.edu/chemistry/trop/geos/ • Click on the Documentation section of the main menu • Types of documentation • GEOS–CHEM Frequently-Asked Questions • GEOS–CHEM Online Manual • GEOS–CHEM Style Guide GEOS-CHEM: Code Management and Benchmarking

  46. Downloading GEOS–CHEM • GEOS–CHEM Frequently-Asked Questions • What’s new in GEOS–CHEM? • What kinds of simulations can G–C perform? • What is the horizontal and vertical resolution of G–C? • What kinds of met fields can be used with G–C? • What diagnostic quantities can be archived w/ G–C? • How is the chemistry mechanism defined in G–C? GEOS-CHEM: Code Management and Benchmarking

  47. Downloading GEOS–CHEM • GEOS–CHEM Online Manual • Introduction • Installation and compilation • Coding: Practice and style • F90 modules • Parallelization • Files in the shared data directories • Short description of files and their creators • References • Files in the run directories • Debugging GEOS-CHEM: Code Management and Benchmarking

  48. Downloading GEOS–CHEM • GEOS–CHEM Style Guide • Guidelines for writing readable, well-documented code • Headers, declarations, indentations, use of white space • New language features of Fortran 90 (including modules) • Fixed-format vs. free-format • Numeric and character data types used in GEOS–CHEM • Converting from number to character and vice versa • Math optimizations • DO Loops GEOS-CHEM: Code Management and Benchmarking

More Related