1 / 24

Once upon a time ….

Once upon a time …. Implementing QA/QC into OGC/SWE. Janet Fredericks Woods Hole Oceanographic Institution OOSSIW – UCAR/Boulder, CO November 18-20, 2008. Q2O – http://q2o.whoi.edu. What is Q2O … QARTOD to OGC? What is the Q2O goal? What has to be done? What does it enable?

curry
Download Presentation

Once upon a time ….

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Once upon a time ….

  2. Implementing QA/QC into OGC/SWE Janet Fredericks Woods Hole Oceanographic Institution OOSSIW – UCAR/Boulder, CO November 18-20, 2008

  3. Q2O – http://q2o.whoi.edu What is Q2O … QARTOD to OGC? What is the Q2O goal? What has to be done? What does it enable? How can it be used? What next?

  4. QARTOD is a multi-organization effort to address the Quality Assurance and Quality Control issues of the Integrated Ocean Observing System (IOOS). • Motivation • Expected increase in the number of observations, observing systems, and users • Need simple, accurate and consistent quality descriptions for the real-time data to convey expected level of data quality to users Engaging the community to develop standards. Courtesy Julie Bosch

  5. Activity initiated by NOAA NDBC and CO-OPS in 2003 • Series of Workshops • Workshop format with plenary and breakout sessions • Approach to tackling the QA, QC, and metadata issues evolved • From separate breakout sessions for QA, QC, and metadata for all observations combined • To breakout sessions by observation focus group (covering QA, QC, and metadata) • Each observation focus group addresses the same guidance questions • Participation • Approximately 80 participants per workshop • Primarily observing system operators and data managers • Representation from federal agencies, oceanographic institutions, universities, and instrument manufacturers Courtesy Julie Bosch

  6. The Workshops • QARTOD I: December 3-5, 2003 NDBC, Stennis Space Center, MS • Task to develop minimum standards for QA/QC methods and metadata • Considering existing observing systems and existing practices • QARTOD II: February 28-March 2, 2005 CO-OPS, Norfolk, VA • Focus on wave, in situ current and remote current (HF Radar) measurements • Identify unique calibration, metadata and QA/QC needs • QARTOD III: November 2-4, 2005 SIO, La Jolla, CA • Continued waves, in situ currents and remote currents work • Added CTD focus group • Primarily addressed QC with some metadata emphasis • QARTOD IV: June 21-23, 2006 WHOI, Woods Hole, MA • Continued wave, in situ current and CTD work • Added DO focus group • Primarily addressed QA; increased emphasis on capturing metadata requirements • International considerations Courtesy Julie Bosch

  7. Why OGC?

  8. OOSTETHYS (MVCO in Fall 2006)

  9. What is Q2O? Funded by NOAA CSC/IOOS (January 2008 – December 2010) Deliverables: • Implement the QARTOD recommendations into the OGC Sensor Web Enablement framework providing SensorML profiles for QARTOD tests and • Documenting results by providing a tutorial and • Test the deliverables by implementing services at participating data centers Methodology: • Bring together IT specialists with domain experts (for waves, in situ currents, CTD observations and Dissolved Oxygen) • Partner with community building projects such as OOSTethys and MMI Today we will demonstrate the real-time waves offerings implemented for the MVCO.

  10. SWE-SOS? SWE-SOS QC-flags QC-flags NDBC info & processing Data Server info & processing Manufacturer’s info & processing SWE-SOS NDBC MVCO QC-flags Or Space Time Toolkit, or OOSTethys or NWS or science user (Matlab) or NCDDC …. OR SENSOR ALERT or NOTIFCATION!!!

  11. Q2O – http://q2o.whoi.edu What is Q2O … QARTOD to OGC? What is the Q2O goal? What has to be done? What does it enable? How can it be used? What next?

  12. QARTOD - WAVES - QC RECOMMENDED TESTS What has to be done by Q2O?

  13. Take QARTOD tests and flags and code them into OGC/SWE instances. Each test is defined as a process that is described with input and output E.g., RangeTest has inputs (min, max) and outputs test results (flagged data) SpikeTest has inputs (tolerance, number of iterations)

  14. In your terms, describe your sensor, its deployment, document provenance, related events … AND the processing that was done and on-demand processing steps that could be done What has to be done by data providers?

  15. Research existing vocabularies and SWE profiles to see what already exists that you can use … look for standards!! • Develop vocabularies that specifically define your observations and processing chains … using MMI tools introduced at this workshop What has to be done by a data provider ?

  16. How and with What do we convey that information? • Get Capabilities • lists available data (properties) • uses SOS, Observation Offering • Describe Sensor • Provides sensore characteristics, deployment characteristics and processing methods • Uses SensorML • Get Observation • Provides the data • Provides test results • Points to file with processing/test info • Uses O&M • What information can we provide to data users or systems (OOSTethys)? • What sensors we have available as a service • Description of the sensor • Description of where / how / when it is deployed • List of the processing methods used on the data • List of the QC tests applied • The criteria used in the QC tests • The results of the QC tests • The data • … What do we have (know) to start with? A sensor (wave buoy or ADCP) with certain characteristics A sensor history QA info associated with a sensor Deployment characteristics Methods to process the data QC Tests to apply to the data … Show the data model (tests, criteria), dictionary, and where all the pieces and parts are located Shown as snapshots of information on multiple web Pages, docs, etc Show info conveyed in the SWE xml files Julie Bosch

  17. Register your results so they can be discovered!! What has to be done?

  18. Anatomy of a SensorML Process Chain for MVCO ADCP

  19. Q2O – http://q2o.whoi.edu What is Q2O … QARTOD to OGC? What is the Q2O goal? What has to be done? What does it enable? SEMANTIC INTEROPERABILITY!! How can it be used? What next?

  20. To describe a sensor, its deployment, etc To describe related events (calibration, maintenance, concurrent observations) To describe processing … What was done? what parameters were used? To describe the observation … did the observation pass a given test? How can it be used?

  21. “Provides an API for managing deployed sensors and retrieving sensor data” • Key here is access through a service • Three mandatory “core” operations GetObservation DescribeSensor GetCapabilities (Demonstration) Sensor Observation Service

  22. Q2O – http://q2o.whoi.edu What is Q2O … QARTOD to OGC? What is the Q2O goal? What has to be done? What does it enable? How can it be used? What next?

  23. Q2O – get offerings online; complete tutorial; continue with QARTOD currents, CTD and DO. Demonstrate computation of waves parameters from triplet processing IOOS? Ingest SWE observations with QA/QC descriptions and flags; evaluate and apply additional tests; document in SWE offerings? OOSTethys - utilize QA/QC flags SUGGESTIONS?? What next?

More Related