MOOS SSDS Data Access Features

1. MOOS SSDSData Access Features A Discussion with MBARI’s Science Data Users

2. Today’s Goals • Provide overview of SSDS in MOOS context • Review user interface (UI) products and concepts • Show you UI options, and let you guide our way • Offer opinions on good and bad approaches • Identify long-term goals and ideas • Note: We are not presenting a released product • It will be a while (6-12 months) before the system is solid • Just getting early feedback on concepts from users • Part of iterative development process

3. Today’s Sequence of Events • ‘Throwaway Demo’ (“Look, it works, really!”) • Introduce MOOS and SSDS • Demonstrate Our Concept of Operations • Show a simulated example using real SSDS software. Knowledge increases! Data is stored! • Demonstrate Prototype User Interfaces • Tour of (MOOS) HOOVES • You Tell Us: What Are the Most Wanted MOOS Data Products • Fill out the survey, join ssdsusers, just talk to us… • We’ll keep improving (read: changing) these products for a while, but we’ll try to give fair warning.

4. Demonstrating Real Data • Interface to MOOS Test Mooring data • Put together quickly, using assumptions • This interface is simple because data is still simple • Data was collected through latest MTM run • As data sets get more complicated, this kind of interface breaks down • Subsetting, varied record types, deployment changes all make the data rather messy • At end of presentation you can tell us what you need to work ‘in the real world’

5. Archiving 101110 110 234 999 110011 Data Presentation Data line 1 more data last data MOOS (Shows Data Flow) Applications/ Interfaces User Communications             Portal Deployed Platform Data Tracking Applications Devices Shore Side Data System (User Tools) Ocean Side Shore Side

6. Shore Side Data SystemRequirements • Ingest data in any described format and save it • Capture, publish data descriptions (via metadata) • Provide standards-based access to data • Raw data, and other common digital formats • APIs for use by common visualization and analysis tools • User-oriented web interfaces, with quick-look plots • Merge data from different sources & time intervals • Allow straightforward visualization of data • Simplify data processing and product archiving • Provide for automated generic quality checks • Provide data access security as needed

7. John’s Top 10 List: Why Use SSDS? 10. All MOOS packets come via ISI. We’ll format and put the raw data back together for you. 9. All the annoying stuff—backups, permissions, storage, naming—is handled by us (for free!). 8. You can point other people to your data. 7. You can see other people’s data. 6. You can actually search for data (by name!). 5. You can easily get basic (+) plots of your data. 4. You won’t ever have to remember what’s in your data again, it will all be described. Forever. 3. It will plug data in to your favorite application. 2. Every data access tool we build works for you. 1. Everybody else will be doing it.

8. 1 2 3 4 How Does It Work? • First, you describe what’s in your data records. (This is the metadata—more about it later. • That description is stored with (near) your instrument, and sent to SSDS before any data from your instrument. SSDS tracks this info…. • When you send a data record, identify its type. • SSDS automatically routes data of each type to its own ‘data bucket’. • SSDS automagically knows all about your data records, because you described them. Now it can do all sorts of useful things with your data: • Plot Print Search Merge • Format (on request) Describe (in files & headers) • Send to Applications Point to by variable name

9. Archiving 101110 110 234 999 110011 Data Presentation Data line 1 more data last data Recap: MOOS Data Flow Applications/ Interfaces User Communications             Portal Deployed Platform 2c Data Tracking Applications 4 3 2b 5a 2a 1 5b Devices Shore Side Data System (User Tools) Ocean Side Shore Side

10. Science Data Users:What Do You Need (First)? • Raw data via device ID pages? (sort of limited) • Standard plots the same as OASIS quality control? • Standard plots with multiple plots per page? • Access data from applications via a DODS URLs? • Matlab, Ingrid, Live Access Server, Excel, IDV, Ferret • And hopefully, Ocean Data View • Access data via shipped data files (e.g., ASCII CSV w/headers) opened within desktop applications? • Excel, ArcView, Ocean Data View • Delivery of data directly into an application? • Ability to subset data, for example by time window? • Ability to merge data from different data sets?

11. Summary • Good data descriptions of structured data records will make many things possible • Many capabilities will be possible—need your input to determine priorities • This project is iterative, so we can fix, improve, and add high priority features • We’ll present some examples as food for thought, then open it up for discussion

12. What Data is Available? Metadata “Explained” How To Access the Data? How to Display the Data? MOOS Data Architecture How to Access Instrument How Data Access Works How Easy to Use is It? Are we sure the data is OK? What if there’s a problem? Is data distributable? Is data secure? What aren’t you getting? References

13. Example of System In Action • [Developer] Get Device ID • [Developer] Define instrument using XML • [Instrument] Start up, send metadata into system • [SSDS] Receive metadata, update instrument info • [Instrument] Send data record(s) into system • [SSDS] Receive data, save, make available to users • [User/Operator] Access data