1 / 40

The IOOS DMAC Metadata Expert Team: Challenges and Status

The IOOS DMAC Metadata Expert Team: Challenges and Status MMI Sensor Interoperability Metadata Workshop October 19, 2006. Presentation. Brief overview of IOOS and DMAC Overview of metadata expert team Recommendations and next steps Challenges. Integrated Ocean Observing System (IOOS).

haruki
Download Presentation

The IOOS DMAC Metadata Expert Team: Challenges and Status

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The IOOS DMAC Metadata Expert Team: Challenges and Status MMI Sensor Interoperability Metadata Workshop October 19, 2006

  2. Presentation • Brief overview of IOOS and DMAC • Overview of metadata expert team • Recommendations and next steps • Challenges

  3. Integrated Ocean Observing System (IOOS) U.S. Commission on Ocean Policy called for an integrated ocean observing system with 7 societal goals: • Improve predictions ofclimate change and weatherand their effects on coastal communities and the nation • Improve the safety and efficiency ofmaritime operations • More effectively mitigate the effects ofnatural hazards • Improve national andhomeland security • Reduce public health risks • More effectively protect and restore healthycoastal ecosystems • Enable the sustained use ofocean and coastal resources

  4. IOOS Subsystems 3 linked subsystems: • Observing system • Data Management and Communications (DMAC) • Data Analysis and Modeling (DAM)

  5. Data Management and Communications (DMAC) Goal: Deliver real-time and delayed-mode observations to a wide variety of users Accomplish through use of standards • Adopt • Adapt • Create only as a last resort

  6. DMAC Functions

  7. DMAC Standards Process Steering Team Expert Teams Metadata and Data Discovery ** Archive Transport and Access Caucuses Private sector International Modeling Regional Working Group Systems Engineering Interagency Oversight Working Group

  8. Metadata Expert Team Provide recommendations on standards for use in IOOS DMAC Starting point: “DMAC Plan” -- Data Management and Communications Plan for Research and Operational Integrated Ocean Observing Systems, March 2005

  9. Metadata and Data Discovery Expert Team Current Status: • Organized in November, 2005 • Teleconference calls every other week • Provided 5 recommendations to DMAC Steering Team in May, 2006 • Currently planning recommendations for next DMAC Steering Team meeting in November, 2006

  10. Metadata Expert Team Members • Selected to represent various groups of interest to IOOS • Approved by the DMAC Steering Team • Federal agencies • Academia • Private sector • Metadata organizations (MMI, FGDC, OBIS, ISO 19115, GCMD) • Related projects (ORION, GEOSS) • IOOS Regional Associations

  11. Metadata Expert Team Members

  12. Metadata Definition The information needed to identify assess use access exchange transport archive data for IOOS

  13. Metadata Aspects Metadata Expert Team divided metadata issues into the following: • Content • Format specification • Vocabularies • Discovery

  14. Team’s Approach Identify existing practices • What currently exists that IOOS can readily use/adopt Evaluate capabilities • What does the use of a particular practice buy the IOOS community Determine IOOS requirements • How and What DMAC and IOOS participants need to do in order to utilize these “systems”

  15. Recommendation - Content Initial minimum list of elements needed to support functions: • Consumer use • Data management • Discovery • Access • Transport • Archive

  16. Consumer use Example of elements for consumer use • Textual summary or abstract, purpose • Observation parameters (attribute names, units, valid values, etc.) • Data source information (instrument type, manufacturer, calibration, etc.) • Originator • Quality assurance / quality control methods • Sample collection methods in field (when applicable) • Sample analysis methods (when applicable) • Data processing methods • Spatial extent (horizontal and vertical geographic location) • Accuracy of location • Temporal extent • Use constraints • Liability

  17. Recommendation – Content and Format Initial list of recommended metadata standards for use in IOOS • FGDC Content Standard for Digital Geospatial Metadata • Global Change Master Directory’s Directory Interchange Format (DIF) • Ocean Biogeographical Information System (OBIS) metadata standard • ISO 19115

  18. Recommendation - Discovery Recommendation: Submit metadata to the appropriate national or international catalog • National Spatial Data Infrastructure (NSDI) Clearinghouse • Geospatial One-Stop (GOS) • Global Change Master Directory (GCMD) • Ocean Biogeographic Information System (OBIS)

  19. Rationale These catalog services: • are national and/or international in extent • are established resources with support and ample guidance • provide for data discovery • enable initial cataloging of IOOS data • possibly applied as step in observing system “certification”

  20. Next Steps Build IOOS interfaces catalog services • NSDI Clearinghouses • GCMD and OBIS • Geospatial OneStop

  21. Recommendation - Vocabularies Adopt a standard controlled IOOS vocabulary • A discrete set of terms that can be referenced in metadata (keywords) and other IOOS documentation • Drawn from existing IOOS documentation: • The First U.S. Integrated Ocean Observing System (IOOS) Development Plan, Ocean.US Publication No. 9, January 2006. • Contains terms identified as • IOOS Identifiers • IOOS Core Variables • IOOS National Backbone Programs

  22. IOOS Identifiers • Integrated Ocean Observing System • IOOS

  23. IOOS Core Variables • Bathymetry • Bottom Character • Contaminants • Dissolved Nutrients • Dissolved O2 • Fish Abundance • Fish Species • Heat flux • Ice Distribution • Ocean color • Optical properties • Pathogens • Phytoplankton species • Salinity • Sea Level • Surface Currents • Surface Waves • Temperature • Zooplankton abundance • Zooplankton species

  24. IOOS National Backbone Programs • Altimeter Data Fusion Center • ADFC • Benthic Habitat Mapping and Monitoring • Coastal Change Assessment Monitoring • Coastal Field Data Collection Program • CFDCP • Coastal Mapping • Coastal-Marine Automated Network • C-MAN • Coastwatch • Commercial Statistics • Coral reef mapping • Coral reef monitoring • Geostationary Operational Environmental Satellites • GOES • Global Seismic Network • GSN • Habitat Assessment • Hydrographic Surveying • Integrated Buoy Program • Living Marine Resources-Ecosystem Survey • LMER-ES • National Current Observing Program • National Data Buoy Center • NDBC • National Estuarine Research Reserve System • NERRS • National Observer Program • National Stream Quality Accounting Network • NSQAN • National Streamflow Information Program • NSIP • National Water Level Observation Network • NWLON • Physical Oceanographic Real-Time System • PORTS • Polar Operational Environmental Satellite • POES • Protected Resource Surveys • Recreational Fisheries • Shoreline Change • Stream gauging • Topographic change mapping

  25. Next Steps • Expand vocabulary as recommended to include: • Additional core variables as defined by IOOS • OOS characteristics as determined by Regional Associations and individual Observing Systems • System characteristics • Sensors • Observed parameters • Etc. • Submit Version 2 for Steering Team approval (Nov 06)

  26. Required Vocabularies IOOS Vocabulary Version 1 • Identifiers (required) • Core variable (recommended) • National Backbone Programs (recommended) ISO Topic Categories • farming • biota • boundaries • climatologyMeteorologyAtmosphere • economy • elevation • environment • geoscientificInformation • health • imageryBaseMapsEarthCover • intelligenceMilitary • inlandWaters • location • oceans* • planningCadastre • sociology • structure • transportation • utilitiesCommunication

  27. Recommended Vocabularies GCMD's Science Keywords and Associated Directory Keywords • Apply to DIF and SERF metadata for use in GCMD • Can be applied to FGDC metadata for discovery through NSDI and GOS IHO Codes for Oceans and Seas • Apply to OBIS discovery metadata • Can be applied to FGDC metadata for discovery through NSDI and GOS OBIS Marine Species Groups • Apply to OBIS discovery metadata • Can be applied to FGDC metadata for discovery through NSDI and GOS CF Standard Names • Recommended by Modeling Caucus for model output datasets • Can be applied to FGDC metadata for discovery through NSDI and GOS

  28. Next Steps OOS community outreach • identification of individual OOS vocabularies • conversion of vocabularies into OWL • ontology development – harmonization among vocabularies Work with the RA Caucus, Marine Metadata Interoperability (MMI) project and other groups to advance the above items

  29. The Mega-Metadata Matrix • Excel spreadsheet created to compare metadata standards • Crosswalk of fields • Checkboxes for functions

  30. Metadata Standards Review and Supporting Tools Review of existing metadata standards to: • Identify those that meet IOOS requirements • Expedite the process to review future standards • Identify where further refinements are needed for existing metadata standards

  31. 14 metadata standards + 850 metadata fields + 6 functionalities + 4 data types Matrix

  32. Comparing Metadata Standards

  33. Status Selection of metadata standards • DMAC plan, input from team • Need input from community Crosswalk of fields • Completed for FGDC, DIF, OBIS metadata, ISO 19115 • Started for others but needs lots of input and review • Not a comprehensive list of fields to date Functionalities • Some work mapping fields to IOOS minimal metadata requirements Data/observation types • No work to date

  34. Matrix – Metadata Standards 14 metadata standards: • Federal Geographic Data Committee (FGDC) • DIF • OBIS metadata • OBIS schema • ISO 19115 • JGOFS • netCDF+ • WMO core ocean gridded data • Open Archival Information System (OAIS) • Ecological Metadata Language • Geographic Markup Language • Sensor ML • Marine XML • Earth Science Markup Language

  35. Matrix - Functionalities 6 functionalities: • Consumer use • Data management • Discovery • Access • Transport • Archive Results in IOOS minimal or “must have” metadata

  36. Matrix – FunctionalitiesExample

  37. Next Steps Critical component of DMAC Requires lots of time and attention to detail • Complete crosswalks for other metadata standards • Analyze field content for machine-to-machine interoperability • Research fields needed to adequately describe observation types, forms, and parameters • Track and incorporate efforts such as MMI, QARTOD, ORION to document observational parameters of interest to IOOS • MMI Sensor Workshop!

  38. Challenges • How many metadata standards will be acceptable/workable? • How many clearinghouse standards will be acceptable/workable? • Will DMAC need to provide tools to garner participation? • Who will write the code to make data discovery work? Who will write tools? How much funding is there for these things? • Focus on how to build data discovery from various clearinghouses (mixture of all of above?) • How will data discovery interoperate/coordinate with data access and transport? • How will data discovery figure into system architecture plan now being developed? • Build a portal to which clearinghouse?

  39. Challenges *Slide by John Graybeal Metadata Data Protocol HTTP REST Z39.50 SOAP OPeNDAP THREDDS LAS DCMI ESML Content ADL TIF DFDL WxS MarineXML EML NetCDF FGDC Coards/CF GML JPEG ISO HDF ASCII

  40. More Information More info on IOOS and DMAC http://www.ocean.us More info on Metadata expert team Anne.Ball@noaa.gov Julie.Bosch@noaa.gov

More Related