An update on obs4MIPs earthsystemcog /projects/obs4mips

1. An update on obs4MIPs www.earthsystemcog.org/projects/obs4mips WDAC Observations for Model Evaluation Task Team P. Gleckler and D. Waliser (WDAC task team co-chairs) with substantial input from all other team members (especially K. Taylor, R. Ferraro and V. Eyring) • A very brief review and status report of CMIP (a WGCM activity) • Progress with obs4MIPs since WDAC2 • Critical infrastructure (standards, conventions and technologies) • obs4MIPs-CMIP6 meeting (last week at NASA HQ) • The WDAC task team and next steps . . .

2. … CMIP6 Timeline • 2014 • 2015 • 2016 • 2017 • 2018 • 2019 • 2020 • Diagnostic, Evaluation and Characterization CMIP DECK with standardized metrics & assessment Model Version 1 Model Version 2 Model Version 4 Model Version 3 Diagnostic, Evaluation and Characterization of Klima(DECK) MIP1 MIP2 MIP1 CMIP6 Endorsed MIPs MIP3 MIP2 MIP4 Finalize experiment design (WGCM) Forcing data: harmonization, emissions to concentrations Scenario MIP studies , MIP matrix, pattern scaling, scenario pairs Run and analyze scenario simulations from matrix Future projection runs Community input on CMIP6 design Possible IPCC AR6 Preliminary ESM/AOGCM runs with new scenarios Formulate scenarios to be run by AOGCMs and ESMs • Nominal Simulation Period of CMIP6

3. Many are contributing to obs4MIPs Obs4MIPs initiated by JPL (R. Ferraro, J. Teixeira and D. Waliser) and PCMDI (P. Gleckler, K. Taylor) with oversight and support provided by NASA (T. Lee) and U.S. DOE (R. Joseph) Several dozen scientists (thus far especially in NASA and CFMIP-OBS) doing substantial work to contribute datasets Encouragement at WDAC2 to “internationalize” with a task team (now in place) Participation is broadening, and the hope is sponsorship will also Jet Propulsion Laboratory California Institute of Technology

4. obs4MIPs: The 4 Commandments Use the CMIP5 simulation protocol (Taylor et al. 2009) as guideline for selecting observations. Matching variable required Observations to be formatted/structured the same as CMIP Model output(e.g. NetCDF files, CF Convention) Include a Technical Note for each variable describing observation and use for model evaluation (at graduate student level). Hosted side by side on the ESGFwith CMIP model output. Observation Experts Modelers Target Quantities Model Output Variables Analysis Community Satellite Retrieval Variables Initial Target Community

5. obs4MIPs “Technical Note” Content (5-8 pages) • Intent of the Document • Data Field Description • Data Origin • Validation and Uncertainty Estimate • Considerations for use in Model Evaluation • Instrument Overview • References • Revision History • Point of contact

6. obs4MIPs: Current Set of Observations CFMIP-OBS Provided ana4MIPs Provided Reanalysis Initial in-situ Example

7. Critical Infrastructure for CMIP, obs4MIPs and related activities

8. Why data standards? Standards facilitate discovery and use of data • MIP standardization has increased steadily over more than 2 decades • User community expanded from 100’s to 10,000 Standardization requires • Conventions and controlled vocabularies • Tools enforcing or facilitating conformance Standardization enables: • ESG federated data archive • Uniform methods of reading and interpreting data • Automated methods and “smart” software to analyze data efficiently

9. What standards is obs4MIPs building on? netCDF – an API for reading and writing certain types of HDF formatted data (www.unidata.ucar.edu/software/netcdf/) CF Conventions – providing for standardized description of data contained in a file (cf-convention.github.io) Data Reference Syntax (DRS) – defining vocabulary used in uniquely identifying MIP datasets and specifying file and directory names (cmip-pcmdi.llnl.gov/cmip5/output_req.html). CMIP output requirements – specifying the data structure and metadata requirements for CMIP (cmip-pcmdi.llnl.gov/cmip5/output_req.html) CMIP “standard output” list (cmip-pcmdi.llnl.gov/cmip5/output_req.html) Climate Model Output Rewriter (CMOR): used to produce CF-compliant netCDF files that fulfill requirements of standard model experiments. Much of the metadata written to the output files is defined in MIP-specific tables. CMOR relies on these tables to provide much of the needed metadata.

10. Earth System Grid Federation (ESGF) esgf.org The Earth System Grid Federation (ESGF) Peer-to-Peer (P2P) enterprise system is a collaboration that develops, deploys and maintains software infrastructure for the management, dissemination, and analysis of model output and observational data. The US DOE has funded development of this system for over a decade

11. The WCRP supports adoption and extension of the CMIP standards for all its projects/activities Working Group on Coupled Modelling (WGCM) mandates use of established infrastructure for all MIPs • Standards and conventions • Earth System Grid Federation (ESGF) The WGCM has established the WGCM Infrastructure Panel (WIP) to govern evolution of standards, including: • CF metadata standards • Specifications beyond CF guaranteeing fully self-describing and easy-to-use datasets (e.g., CMIP requirements for output) • Catalog and software interface standards ensuring remote access to data, independent of local format (e.g., OPeNDAP, THREDDS) • Node management and data publication protocols • Defined dataset description schemes and controlled vocabularies (e.g., the DRS) • Standards governing model and experiment documentation (e.g., CIM)

12. Earth System Commodity Governance (CoG) This is NEW! earthsystemcog.org CoGenables users to create project workspaces, connect projects, share information, and seamlessly link to tools for data archival. CoG is integrated with the ESGF data. Easy to tailor to project specific needs. The primary interface to CMIP and related MIPs is likely to migrate to COG in the next few months All obs4MIPs information is now hosted on the COG, including access to data

13. Immediate infrastructural needs of obs4MIPs CMOR needs to be generalized to better handle observational data. • CMOR was developed to meet modeling needs • Some of the attributes written by CMOR don’t apply to observations (e.g., model name, experiment name) Modifications and extensions are needed for the DRS (should be proposed by the WDAC obs4MIPs Task Team to the WIP) A streamlined “recipe” for preparation/hosting obs4MIPs datasets CoG further refined to meet obs4MIPs requirements.

14. Obs4MIPs / CMIP6 Planning meeting • Invite only, held at NASA HQ (Washington DC), April 29– May 1 • ~ 55 attendees: a very diverse mix of data experts (mostly satellite), modelers, and agency representatives from NASA, DOE, NOAA, ESA, EUMETSAT, and JAXA • Focused on identifying opportunities to improve the use of existing satellite datasets in CMIP (model evaluation and research)

15. Obs4MIPs / CMIP6 Planning meeting • Day 1 • Background presentations • Atmospheric Composition & Radiation • Atmospheric physics • Moderated discussion topics: • - CMIP6 forcing data sets • - High frequency observations for CMIP6 • - High spatial resolution for CMIP6 • - Geostationary data? • - Going beyond satellite data? • Day 2 • Broadening involvement: Agency views Perspectives, Reanalysis • Terrestrial Water & Energy, Land Cover/use • Carbon cycle • Oceanography & Cryosphere • Moderated discussion topics: • Satellite simulators • Reanalysis (relationship to ana4MIPs) • Day 3 (morning) • Rapporteur summaries • General discussion of future directions • A brief post meeting gathering of the WDAC task team

16. Great discussion, lots of thoughtful feedback & recommendations • Example topics • Some specific recommendations (what we were after!) • More data sets – which ones, priority? • Higher frequency (strong interest in this) • Process & Model Development focus – how to? • Satellite Simulator/Observation Proxy priorities? • Relaxing the “model-equivalent” criteria – how far? • Better characterization of obs uncertainty needed • Optimizing connections to ana4MIPs/reanalysis • Use of averaging kernels – how far? • Geostationary priorities/guidance • Gridded In-Situ data sets • In-Situ – where to start, how far to go? WDAC & Task Team obs4MIPs obs4MIPs obs4MIPs obs4MIPs obs4MIPs

17. Meeting open forum: mixed views on several key issues • Should obs4MIPs continue is focus on “enabling” CMIP research, or should it strive to lead it? • Should obs4MIPs filter datasets based on quality? What about “obsolete” datasets? • How many different datasets of the same observable should be included? Do we want 10 different SST products? • Should obs4MIPs expand beyond focusing on “near globally gridded data sets”, and if so, how quickly and how much? • The WDAC task team & WDAC needs to take a position on these issues

18. A range of possibilities for obs4MIPs guidelines/requirements : Should we construct a “Maturity Matrix” or Model Evaluation Readiness Level (MERL)?

19. WDAC task team discussions (90min, post obs4MIPs meeting) • Team members: P. Gleckler (co-chair; PCMDI), D. Waliser (co-chair; JPL), S. Bony (IPSL), M. Boslovich (GSFC), H. Chepfer (IPSL), V. Eyring (DLR), R. Ferraro (JPL), R. Saunders (MOHC), J. Schulz (EUMETSAT), K. Taylor (PCMDI), J-N Thépaut, ECMWF • Task team should oversee the connection between obs4MIPs and ana4MIPs, but not ana4MIPs itself • Connecting with GEWEX assessments • Review existing “Maturity Matrix” applications and consider building on them to create a Model Evaluation Readiness Level (MERL)? • Considering ex-officio members from agencies contributing or planning to contribute (ESA, NASA, etc.)… liaisons with other MIPs may also be necessary • Augmenting CMOR to better handle observations is urgently needed • For now, webX sessions being considered on a ~ bimonthly basis

20. Some closing thoughts on how all this works Standards and infrastructure fundamentally rely on community consensus • Must serve the needs of multiple projects • Must meet the the diverse set of needs of a broad spectrum of users • Requires substantial coordination • Demands community input and the oversight relying on the expertise of a diversity of scientists and data specialists The current system is fragile in that many of the components are funded by individual projects that could disappear, impairing viability of the entire infrastructure Every funder is essential to supporting this collaborative effort where • Leadership and recognition is shared across multiple projects • Integration of the “whole” imposes external requirements on the individual components.