1 / 18

OPeNDAP QC Implementation Use Case

OPeNDAP QC Implementation Use Case. SECOORA/SEACOOS Activities UNC, USC, USF, UMiami, SKIO Sara Haines, UNC Wednesday, 27 February 2008. Implementing QC for Regional Data Sharing. SECOORA/SEACOOS Activities UNC, USC, USF, UMiami, SKIO Sara Haines, UNC Wednesday, 27 February 2008.

euclid
Download Presentation

OPeNDAP QC Implementation Use Case

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. OPeNDAP QC ImplementationUse Case SECOORA/SEACOOS Activities UNC, USC, USF, UMiami, SKIO Sara Haines, UNC Wednesday, 27 February 2008

  2. Implementing QC for Regional Data Sharing SECOORA/SEACOOS Activities UNC, USC, USF, UMiami, SKIO Sara Haines, UNC Wednesday, 27 February 2008

  3. What is SEACOOS? • Southeast Atlantic Coastal Ocean Observing System • Kick-off meeting October 2002 • Observing, modeling, outreach, and data management • Partnership of only a few • http://www.seacoos.org What is SECOORA? • Southeast Coastal Ocean Observing Regional Association • Fall 2006 • Many of the same people and ideas are now part of SECOORA • Stakeholders and established governance, more diverse • http://www.secoora.org

  4. SEACOOS Data Management • Share data from a wide range of instruments (CTD, HFRADAR, ADCP, Met-packages) • From many different platforms (ships, aircraft, buoys, moorings, satellite, AUVs, drifters) • Initial parameters addressed were winds, temperature, salinity, water level and currents • We needed standards but there were none • Created a Common Data Language (CDL) • Established a Data Dictionary (included by MMI)

  5. Data Flow using OPeNDAP

  6. Best of both worlds

  7. SEACOOS CDL v2.0 • Specifies required and recommended metadata fields and attributes for • Provides several canonical formats modeled on how measurement taken (e.g. ship-mounted ADCP, or moored string of CTDs) • Initially intended for netCDF, but could be used for other formats such as HDF, XML

  8. SEACOOS CDL v2.0Examples • Fixed point • Fixed profile • Moving profile • Fixed map • Moving 2D map • Moving 3D point

  9. SECOORA QA/QC Workshop March 9-10, 2006 Data Sharing and Data Quality of Ocean Currents and Temperature towards an Operational Data Management System • Regional Association (RA) Responsibilities • Aggregator of data and/or access to distributed data • QC of aggregated data • Assuring data available (Is a cataloged sensor/data there for time desired?) • Catalogue of assets • Ratify and “support” a few well chosen standards • Make it easy as possible to submit data

  10. SECOORA QA/QC Workshop March 9-10, 2006 • List of QC Tests and Who is Responsible: • Data Availability DP (or RA) • Range checks DP (or RA) • Time Continuity DP (or RA) • Rate of Change DP (or RA) • Data Format DP (or RA) • Instrument uncertainties DP • Nearest Neighbor RA (or DP) • Climatology RA (or DP) • Model Comparison RA (or DP) • Based on QARTOD recommended tests

  11. Implementation Sprint for QC July 11-14, 2006 • Focus on QC or the step that supports the output of high quality data • Assume QA addressed by those responsible for proper operation of instrumentation and assessment of normal operation • Draw on QARTOD findings and IOOS DMAC recommendations • We needed more standards • Extended the Common Data Language (draft SEACOOS CDL v3.0) which defined required information • Specified Tests • Specified Flags (specific and aggregate)

  12. Draft SEACOOS CDL with QC (v3.0) • Wind, SST, Current • For each variable of interest, and for each instance of data for that variable, there will be multiple QA/QC tests and multiple QA/QC flags. • Test Flags • Aggregate Flag • Example SST

  13. Stub Out • Specifications for data providers (required and recommended) on OPeNDAP-servers (Draft CDL v3.0) • Specifications for regional provider (Draft CDL v3.0) • Regional search and database schema to accept QC flags and info (beta) • GUI Control (beta)

  14. Stub out • Lack funds and resources for data managers to do QA/QC work of implementing data model • SECOORA/SEACOOS only lacked individual institutions to output QC Flags and info on OPeNDAP-servers using this data model

  15. Next Steps • Add variables for testing (currents, waves) • Define and implement analysis of aggregated QA/QC effectiveness. • Nearest neighbor, add model comparison • Uncertainty values • Sensor specific assessment, tests and flags • Revisit draft CDL for currents and temperature with QARTOD 4 findings • Continue monitoring IOOS DMAC and QARTOD as new QA/QC recommendations, parameters and technologies emerge

More Related