1 / 18

Data Delivery

This article discusses the data delivery process for various use cases including science users, marine operators, educators, and the general public. It covers topics such as search, download, ancillary data retrieval, data product generation, and configuration changes.

medford
Download Presentation

Data Delivery

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Data Delivery

  2. Agenda- Data Delivery Topics covered: Science/Operator/Education/Public  Use Cases , User requests and Data Product Generation , Operator Use Cases (mobile, uncabled and cabled), Command & Control Addresses: C?#2= How well will the data management plans and processes serve the OOI users as defined by the presented use cases ? Are there any gaps identified, critical weaknesses or recommendations for improvement? 1

  3. Bottom Line of ‘Data Delivery’ • “Get good data to people” – • ‘good’= quality (covered below); • ‘Get data to people’= this section • Physical and functional/logical design just presented • We followed the basic engineering V approach. • Now that we have all the ‘front-end’ items, it’s the user and their needs that determine success or failure • How Users Get the Data- • OOI is, to the science user, a website • How do we judge? 2

  4. How Is OOI Judged? • Initial; then Matured Use Cases • Science user • Marine Operator • Educator • Requirements Verification • Beta Testing • Validation Testing • User Community 3

  5. Science use case summary • Search, Locate and download OOI data • Search, Locate and download OOI documents, reports • Add or retrieve ancillary data related to an OOI deployment/recovery cruise • Subscribe to alerts based on user-defined criteria • Examine a data product for consistency and quality • External PI wishes to alter the configuration of an instrument or platform 4

  6. User Request and Data Product Generation User Interface Database- Postgres Product Catalog Product Generation 5

  7. Search, Locate and download OOI data • Most fundamental task • Pick area, find product 6

  8. Search, Locate and download OOI data • Most fundamental task • Pick time, examine/look at, download • Download formats include NetCDF, CSV, Matlab, HTML, PNG (map plot), PDF • Download URL accessible from external tools, e.g. Matlab using DAP protocol 7

  9. Search, Locate and download OOI documents, reports • Data is only part of the need • Pick array- find document, read/download •Coastal Glider, June 2011 ◦Glider specifications •Open Ocean Glider, October 2011 ◦Glider specifications •Global Hybrid Profiler Mooring, May 2012 ◦Top-level drawings •Global Mesoscale Flanking Mooring, May 2012 ◦Top-level drawings •Global Surface Mooring, July 2012 ◦Top-level drawings •Coastal Surface Mooring, July 2012 ◦Top-level drawings •Coastal Profiler Mooring, July 2012 ◦Top-level drawings •Endurance Array Benthic Experiment Package, May 2012 ◦Top-level drawing •Operations and Management Component, January 2013 •AUV/AUV Dock System, February 2013 •Uncabled Coastal Surface Piercing Profiler, Sept 2013 •Cabled Coastal Surface Piercing Profiler, Mar 2014 8

  10. Add or retrieve ancillary data related to an OOI deployment/recovery cruise All sites have pages • For a summary of the July 2013 Station Papa Deployment, click here » Click to download a PDF of all Infrastructure Tables Click the below links for individual Instrument Tables (A) Hybrid Profiler Mooring(B) Flanking Mooring A(C) Flanking Mooring BMobile Assets – Gliders Technical Drawings » Click here for the online Technical Data Package Click the below links for High-Level Technical Drawings (A) Hybrid Profiler Mooring(B, C) Flanking Mooring 9

  11. Subscribe to alerts 12 predefined “events” can be subscribed 10

  12. Examine a data product for consistency and quality Check the Metadata 11

  13. External PI wishes to alter the configuration of an instrument or platform • This is largely a management level use case with a triad of actors- NSF, OOI and an external PI • It then becomes part of routine operations • Steps: • The PI and the O&M Lead for the Endurance Array conduct planning meetings to confirm the configuration(s) required to address the approved science objectives of the PI’s work, and the planned frequency of configuration changes to OOI instruments and gliders within the scope of the approved work. • The Endurance O&M Lead provides the PI with the information and possibly structure required for submitting configuration changes (sampling rates, glider mission plans). • PI edits the instrument configuration and glider mission plan templates and submits the changes and their likely frequency within the approved 2-year study interval, to the Endurance O&M Lead. • The Endurance O&M Lead reviews those requests with the PI to assure that OOI safety and performance guidelines will not be compromised. • Observatory Director submits approved change(s) to UNOLS for scheduling of implementation. • UNOLS evaluates, with iteration with Observatory Director and external PI, when approved configuration should be implemented (other approved projects may be in the queue). Work is scheduled for implementation within the next calendar year. • Observatory Director includes this approved work in the next Annual Work Plan and schedule for the next calendar year. • External PI begins interactions with Endurance O&M Lead to implement his/her specific instrument and glider configurations upon the approved change dates (as dictated by the work approved by NSF and scheduled via UNOLS). • The Endurance O&M Lead sustains the PI configuration(s) per the approved award. 12

  14. Marine Operator use case summary • PERFORM COMMAND AND CONTROL OF DEPLOYED ASSETS • MONITOR SYSTEM HEALTH AND STATUS AND ACT PER PROCEDURES • SUBMIT DATA TO NATIONAL ARCHIVE(S) • PERFORM HW AND/OR SW UPDATES • OPERATOR SYNCHRONIZES OBSERVATORY CONFIGURATION ACROSS DATA SYSTEM AND DOCUMENT SYSTEM • MANAGE OOI HOMEPAGE (SYSTEM INFORMATION PORTAL) AND TROUBLE REPORTS • ESTABLISH / ALTER / MAINTAIN THE CONFIGURATION OF OOI • PROVIDE EXTERNAL PI SUPPORT 13

  15. Educational Community use case summary 14

  16. Output Data Product Variables • Each L1 and L2 product has the following variables (i.e., columns in the time series): • Time • <measurement>_L1a (e.g., Conductivity_L1a) • <measurement>_L1b_Post_Deployment_Cal • <measurement>_L1b_Post_Recovery_Cal • <measurement>_L1c • QC_Flag_GlobalRange • QC_Flag_LocalRange • <additional QC flags> • Single “Parsed”(Combined) product per instrument, with all variables for applicable L1 and L2 products, additional time stamps

  17. Output Data Product Metadata • In the metadata (i.e., ‘Metadata’ link from ERDDAP page, AND metadata on Data Product facepage on OOINet UI): • Calibration coefficients (as a comma separated list) • QC Look Up Table (as a url) • Data Product Algorithm code (as a url) • DPS for Data Product Algorithm (as a url) • QC Algorithms (as urls) • DPS’s for QC Algorithms (as urls)

  18. Data Delivery Summary • DM is not a noun, verb, thing nor separate effort • But rather a perspective that includes all the other groups and actions • Fundamentally it is the timely implementation of the necessary requirements to deliver data to users • Each piece, say UI, metadata or algorithms or archive policies, is just one piece of the chain that has to be defined, built, integrated and supported 17

More Related