1 / 31

WOUDC SAG Ozone / UV Update

Explore the latest developments in the WOUDC Ozone/UV Data Centre operations, including enhancements, future work, and the progress towards full NDACC data integration.

sorrell
Download Presentation

WOUDC SAG Ozone / UV Update

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. WOUDC SAG Ozone / UV Update Tom Kralidis Geospatial and Open Data Systems Meteorological Service of Canada https://woudc.org WMO GAW SAG Ozone/UV Meeting Geneva, Switzerland Wearable technology, developed with funding from the NSERC Discovery Grants program Source: Western University 08 May 2019

  2. WOUDC SAG Ozone / UV Update • Data Centre Status • 2018 by the Numbers • 2018 SAG Actions/Recommendations • 2018 Enhancements/Activities • Future work

  3. Data Centre Status • Renewal operational 2015 • Current operations • Contributor support • User support • Continuous data improvement • Data ingest • Metadata quality assessment / correction • Enhancements

  4. Data Centre Status • Organizational Overview • Meteorological Service of Canada • Data Management • Data Products and Services • Geospatial and Open Data • Collaboration with ECCC Science and Technology Branch • Participation in WMO Expert Team on World Data Centres (ET-WDC) • https://wmo-cop.github.io/et-wdc/

  5. GAW Implementation Plan – Data Management • Federation, interoperability, data access, discovery • Open access, metadata • WIGOS • Data archiving, analysis centres • Known quality, documentation • Near real-time delivery (GTS/WIS) • Data submission and data use • DOIs

  6. Number of Files by Dataset (current)

  7. Processing Runs 2018

  8. 2018 Downloads

  9. Website Visits

  10. Top Hits by Visitor Domain

  11. Website Summary • Bulk download (vs search) via the flat file archive continues to increase (vs search, totalozone master file, dataset snapshots)

  12. New Contributors • Portugal: IPMA (back) • Taiwan: CWBT (back) • Greece: Athens Academy • Hungary: HMS (back) • Kenya: KMD (back) • Spain: INTA (back) • Thank you Lorena Moreira (WMO) !

  13. New Stations • Graciosa (Portugal) • Athens Academy (Greece) • Kirchbichl (Austria)

  14. 2018 Contributor Contact Validation • Implemented September 2018 • Contributor Verification emails sent • 149 • Positive response/update: 25 • Unknown / no response / bounced: 124 • Next run: September 2019

  15. SAG Actions • Action 18: Link WOUDC to the European UV database (EUVDB) • Available via https://woudc.org/resources/links.php • WOUDC recommendation: investigate data indexing approach (similar to NDACC and Eubrewnet pattern)

  16. SAG Recommendation • A DOI for individual stations should be implemented at WOUDC • Best implemented via GAWSIS/OSCAR • GAWSIS creates DOIs • WOUDC integrates them on website • Being discussed among WMO Expert Team for World Data Centres (ET-WDC)

  17. WOUDC DOI Implementation • https://woudc.org/about/data-policy.php#dois • Levels of granularity • First order (Ozone, UV) • Dataset level • TODO: Stations (from GAWSIS/OSCAR)

  18. SAG Recommendation • The WOUDC terminology be changed to replace "level" with "version" number, as in general "level" usually refers to the degree of processing from raw data • Extended CSV already has a ‘version’ property as part of data metadata • May cause confusion

  19. WOUDC Metadata Level/Version • CONTENT.Level: The Level refers to the data product. Acceptable values are “1” for data that has been formatted into WOUDC extCSV format (and therefore ready for submission to WOUDC), or “2” for data that has also been interpolated, re-gridded or otherwise processed. (Note that level is not the same as Version as described in 3.2.1.2 – Version is used to indicate the “revision” of a file, where Level is used to indicate the “processing stage” of a file. (There may be several versions of both level-1 and level-2 files) • https://guide.woudc.org/en/#3211-content • DATA_GENERATION.Version: Data version specified by the submitting Agency. These versions have the form “major.minor” (e.g., 3.2) where major values are incremented with a./ changes to data file metadata or b./ the processing algorithm. Minor values are incremented when the characterisation or calibration values have changed. Note: minor values are reset to zero with changes to the processing algorithm • https://guide.woudc.org/en/#3212-data-generation

  20. SAG Recommendation • The WOUDC should implement a full mirroring of the NDACC data archive, which would include NDACC data being available in WOUDC format and listed in WOUDC reports

  21. NDACC Integration • Discussions with NDACC (June/July 2018) • Co-presentation with Jeannette Wild (NDACC DHF Curator) at the NDACC SC Meeting (September 2018) • History of (numerous) efforts (since 2002) • Pros/cons of previous approaches

  22. NDACC Integration • Mirroring • Consistent data formats • Contributes to WOUDC metrics / reporting • Station lookup variations • WOUDC standardized on GAW/OSCAR • Derivations, calculations, interpretations, PI involvement • Managing updates to data (versionitis) • Distributed Search • Data managed closer to data centre source • Reduces data centre data duplication • Inconsistent data formats • Integration effort still on the user

  23. NDACC Integration • Data Centre Interoperability project (DCIO) • Since 2008 • Harmonized dataset metadata • Concept of peering • Data discovery • WOUDC/NDACC believe the most sustainable way forward is to support distributed searching • Evolution of DCIO project • Reduce problems associated with data duplication • Authoritative single source

  24. NDACC Integration • Follow up meetings with NDACC (November 2018) • NDACC renewal • Producing data index • WOUDC could provide HDF output in alignment with NDACC (pilot) • TODO: expand • Expand metrics to include non-WOUDC data

  25. Eubrewnet Integration • Eubrewnetmakes available filelist of all data • WOUDC processes filelist(periodically) and create file index • Eubrewnetdata is discoverable via federated search • Eubrewnetdata available for download from Eubrewnet • Data are not part of metrics • Work resumed (April 2019)

  26. Extended CSV Format Updates • Proposal (Jonathan Davies, ECCC): add data verification method: • DATA_GENERATION.Verification: the means or process used to verify or check the data generation. Possible values (case sensitive) are: MANUAL, AUTO. No default value is assumed • Optional (backward compatible)

  27. Website Updates • Station profile pages now provide a direct link for a given station/instrument • Search/Download Bug fixes • Instrument list reset when changing country filter • Link updates • Added link to EUVDB (per SAG Action)

  28. Centralized processing for Umkehr at WOUDC • History • Prior to WOUDC restructuring (2012) Umkehr data were centrally processed with Mateer and DeLuisi, 1992 algorithm (Fortran code), and ozone profiles were regularly archived. Other Umkehr data versions (including Petropavlovskikh et al, 2005) were also archived and shared with user community. • After WOUDC restructuring, processing of the new data stopped due to software compatibility issues. Umkehr data records processed with UMK92 were made available up to 2010. • Limited processing (6 stations) was continued by Irina to update long-term Umkehr records for trend analyses (i.e. SPARC/WMO LOTUS and UNEP/WMO Ozone assessments) • Since no Umkehr data were processed in the recent years Dobson operators questioned the WMO interest in Umkehr observations. As the result, some stations (i.e. Japan) have recently abandoned Umkehr measurements and interrupted long-term ozone recordsthat are valuable source for WMO objective to guide the ozone recovery

  29. Centralized processing for Umkehr at WOUDC • Recent Ozone assessment found discrepancies in stratospheric ozone trends derived from combined satellite records. Also, discrepancies in trends from collocated ground-based instruments (i.e. MLO, Lauder, EU) were also found. The LOTUS phase 2 is focusing on re-evaluation of long-term GB records (i.e. after recent homogenization) that is impacted by instrumental artifacts, different retrieval algorithms used for the same instruments and conversion between units (mixing ratio on pressure levels vs density on altitude grid). Co-located GB measurements are of great value to evaluate instrumental artifacts, and provide comprehensive validation of combined satellite records. • Archival of the Umkehr observations (aka N-values) continues up-to-date (## stations?) • WOUDC requires that all core software is written in Python • University of Saskatchewan provided support for converting the Umkehr code from Fortran to Python (funded by NOAA) • https://github.com/woudc/woudc-umkehr • WOUDC has resumed central processing Umkehr data (new and archived) • NOAA is working to homogenize its Umkehr records (similar to Evans et al, 2017 paper) • Centralized processing removes artifact of differences in individualized software and auxiliary information (i.e. temperature, ozone cross sections, etc).

  30. SHADOZ updates • Data submission conversion / processing • Hilo (2017), Samoa (1998-2017) • Using pyshadoz • https://tropo.gsfc.nasa.gov/shadoz/Links.html • New submissions coming

  31. Thank you • Website: https://woudc.org • Guidebook: https://guide.woudc.org • Code: https://github.com/woudc • Wiki: https://github.com/woudc/woudc/wiki

More Related