Steven Worley NCAR 19 August 2005 Objectives What is IOOS? What is DMAC?

1. Briefing on the Integrated Ocean Observing System (IOOS) and the Data Management and Communication System Steven Worley NCAR 19 August 2005 Objectives • What is IOOS? • What is DMAC? • How might OHH participate?

2. The Integrated Ocean Observing System (IOOS)Oceans & Coasts Component of the U.S. Integrated Earth Observation System (IEOS) & the International GEOSS An Interagency Collaborationfor the Public Good NOAA Navy NASA NSF USACE USGS MMS EPA USCG DOE http://www.ocean.us

3. IOOS GoalsIntegration for Broad Benefit • Detect and Predict Change • Mitigate effects of natural hazards • Improve safety and efficiency of marine ops • Improve national security • Reduce public health risks • Protect and restore marine ecosystems • Sustain marine resources Society oriented goals

4. E.G. Florida Fish and Wildlife Research InstituteRed tides off southwest Florida, Aug. 17 Reported system effects: • Expanding area since last week • Benthic mortalities • Fish kills • Discolored water • Turtle mortalities • Human respiratory irritation http://www.floridamarine.org/

5. E.G. Florida Fish and Wildlife Research InstituteRed tides off southwest Florida, Aug. 17 Actions: • Airplane over flights – not good weather – more flights needed • Ocean sampling cruise 10-12 August

6. E.G. Florida Fish and Wildlife Research InstituteRed tides off southwest Florida, Aug. 17 Observation and what ifs? • Respiratory irritation greater with westerly wind • What if Hi-res wind predictions were available? • What if satellite sensed data, not impacted by weather, was easily used? • What if real-time ocean circulation models were run to analyze and predict? • What if ocean models with T, O2, nutrients, red tide samples, were easy to run? Contention: We have the ability to do these estimates IOOS strives to enable them and address the goals previously shown

7. The Political Environment • Commission on Ocean Policy • Report • Executive Order • Ocean Action Plan • Legislation • Senate (S. 361 - recently passed) • House (H.R. 1489, 1584)

8. IOOSTwo Interdependent Components Global Ocean Component Coastal Ocean Component Regional Coastal Observing Systems National Backbone

9. IOOS Coastal Ocean Component • National Backbone • Largely federal • Satellite remote sensing • Reference, fixed-site stations • Links to global component • Fewer variables • Regional Coastal Systems • Largely non-federal (may be • federally funded) • Regional priorities & needs • Greater resolution • More variables Both use IOOS data standards and exchange protocols

10. The Global ComponentAn International Collaboration IOOS is the U.S. contribution to international GOOSand the Oceans & Coasts component of IEOS & GEOSS

11. IOOS Three Subsystems • Measurements • Remote and in situ observations • Telemetry • Data Management and Communications (DMAC) • Integration • Data analysis and modeling • Data synthesis, evaluation • Change detection and prediction

12. IOOS Education Component • Ocean.US Education Workshop, 22-24 March 2004, Charleston, SC • Result: Workshop Report • Promoting Lifelong Ocean Education: Using the Integrated Ocean Observing System (IOOS) to Shape Tomorrow’s Earth Stewards and Science and Technology Work Force. • Muriel Cole Ocean.US liaison to OHH • Hear more at IOOS Human Health Workshop, January 2006

13. DMAC is the component that enables the diverse sources of data to be accessed in a consistent manner. DMAC is composed of data standards, communications protocols, software tools, archiving and community agreements.

14. Why so much focus on DMAC? • DMAC is the “I” in IOOS Why is DMAC the top priority? • The weak link in the chain • Huge payoff from investment

15. Data Supplier Data User Multi-beam Geo-chemical Satellite DMAC Three Guiding Principles Existing Protocol DMAC • Do no harm • Expand access Data Suppliers • Increase efficiency

16. The DMAC Plan is a framework for interoperability among independent, heterogeneous systems, large and small • Organizations will manage their data independently • Standards – enable interoperability • Data discovery and delivery to applications • New - “Web services” standards • communication between applications • E.g. Browsers that initiate Google searches

17. The heart of integration isStandardsExample:Surface Marine Buoys • “Work horse” for coastal and open ocean observations • International, National and Regional use • Surface , near-surface and subsurface observations

18. What “Simple Standards” buy you today… • Meeting present “standards” • A “ticket” onto GTS • Real-time provides utility for national forecasts & warnings operations • “Accepted” QA /QC • Immediate payoffs • Large user base • Improved local storm detection, intensity and track forecasts • Beach rip tide, boating safety and trip planning, & others

19. But today’s standards are too narrow. Inefficiencies and lost opportunities … • E.g. Delayed mode not standardized • Data not uniformly • archived • QCed • Data not uniformly available for • climate assessment • commercial applications • research (esp. interdisciplinary) • Commonalities are not exploited (e.g. a time series) • Variables: winds, waves, temperature, salinity, biota, chemicals, fish landings • Platforms: surface marine buoys, terrestrial stream gauges, tide gauges • Products: forecasts, satellite products, coral bleaching (paleo), El Nino index

20. Ships Hand Measurements Satellites Floats Moorings Primary DataAssembly & QC RegionalData ManagementSystems Products InternationalData ManagementSystems Users Maps Forecasts Terrestrial and AtmosphericData ManagementSystems On-line Browse Archive Centers Modeling IOOS Data Management andCommunications Subsystem Metadata, Data Discoveryand Data TransportStandards and Protocols

21. Ships Hand Measurements Satellites Floats IOOS Data Management andCommunications Subsystem Primary DataAssembly & QC Moorings RegionalData ManagementSystems Products Users InternationalData ManagementSystems Maps Forecasts Terrestrial and AtmosphericData ManagementSystems Metadata, Data Discoveryand Data TransportStandards and Protocols On-line Browse Archive Centers Modeling

22. The DMAC Plan – a community effort • Brought together • 6 Federal agencies • 6 Universities • 4 Regional/State agencies • 3 Private sector orgs. • Steering Committee,2 Outreach Teams,4 Expert Teams • Data Facilities Outreach • User Outreach • Data Discovery & Metadata • Data Transport • Data Archive & Access • Applications & Products

23. The DMAC Plan – a community effort • Four levels of review • Selected specialists • Public workshop • Two public reviews • 150 reviewers • 6 Federal Agencies • 22 Universities & Institutes • 13 Private Sector • 13 Regional/State agencies • 5 International • Presented at numerous meetings • ASLO • AGU • AMS • JCOMM, WMO • IODE

24. The DMAC Plan – What does it contain? • Articulates the vision, requirements,and technological approaches. • Recommends next steps, estimates costs and timeframes • A roadmap, not a specification. No attempt to map tasks to specific organizations

25. The DMAC Plan – first order guidance • COOS in RAs advise about IOOS NOW • DMAC Steering Committee designated initial guidance • Report standards and standards processes presently used to Ocean.US • Use FGDC (ISO 19115) metadata standards if starting out • Use community accepted standards otherwise • Create metadata in XML-schema with a style sheet • Gridded data – install server and provide access through OPeNDAP • Use RDBMS • OPeNDAP enabled • Enterprise GIS, OGC compliant • Data providers – ensure irreplaceable data are permanently archived • Archive Centers – structure archives to be accessible and DMAC compliant • Security – until IOOS has data and network security guidelines use those prescribed by like federal centers • Use NDBC services for mooring data • Many more, and will be updated, see http://www.ocean.us

26. Next Steps: Work on DMAC Plan Recommendations • New DMAC-Steering Team • Formed in April 2005 • Open, fair, balance representation • About 25 members • Objective: Minimally functioning DMAC framework – in the near-term

27. Advancing the DMAC PlanWorking Structure under the Steering Team **Expert Teams: Metadata & Discovery Standards Process Archive Transport & Access Caucuses: Private Sector International Modeling Working Groups: IT Security On-Line Browse Systems Engineering Status: All but two WGs have SOWs and ETs are being formed. **Possible areas of OHH Participation

28. More on participation Expert Teams: Metadata & Discovery Standards Process Archive Transport & Access How to participate • Contact, rosalind.e.cohen@noaa.gov • Fit into an ET • Review SOW, online soon

29. E.G. SOW for the Archive Expert Team Statement of need brief, • Forge cooperation between multiple centers to insure: • Data flow integrity • Publish data for discovery and access • Stewardship and long-term curation • DMAC metadata and data transport methods are critical – cross communication with other ETs • Preserve data through life cycles of storage systems

30. SOW for Archive ET, continued Membership: • All US agencies with IOOS data responsibilities • International counterparts • Representative from National Archives (NARA) • Long term strategies and preservation policies • Members must have knowledge and have authority to advance ET goals at the organization level • Approx. 10 members

31. SOW for Archive ET, continued Short term Tasks (2005-2006): • Review and refine DMAC Plan with report to DMAC ST • Define archive critical metadata – provide to Metadata ET • Assess, inventory, and report on IOOS relevant archives • Access status • Metadata status • Identify preservation weakness • Identify gaps in irreplaceable data archives

32. SOW for Archive ET, continued Long term Tasks (2007-2008): • Demonstrate, test bed project, merging metadata from multiple ACs into DMAC data discovery portal • Use DMAC meta data standards and transport protocols for two-way data flow • Receipt and delivery of data • Establish comprehensive metrics – system wide • Composite data polices from all agencies • Participate with governing committees to form a IOOS data policy • Many other goals outlined in DMAC plan!

33. SOW for Archive ET, continued Schedule: • Form, to extent possible the Archive ET, October 2005. Current members: • Bob Keeley (MEDS/Canada) • John Lever (Navy/CNMOC) • Landry Bernard (NOAA/NDBC) • Steven Worley (NCAR/NSF) • Validate membership before DMAC ST, Nov. 2005 • Hold first AET meeting, early 2006 • Use, web forums, email, and twice annual meetings to address Short Term tasks

34. Conclusions: • IOOS and DMAC have begun • Much growing to do • Must keep up with rapid developments at COOS and RAs • Fostering broad participation is the challenge • Technology is tracking the need well • Our success will be measured by how well we fulfill the societal needs

35. Questions? http://www.ocean.us