NOAA Metadata Update

1. NOAA Metadata Update Ted Habermann

2. NOAA EDMC Documentation Directive This Procedural Directive establishes 1) a metadata content standard (International Standards Organization [ISO] 19115 Parts 1 and 2) and 2) a recommended representation standard (ISO 19139) for documenting NOAA’s environmental data and information. Roles and Responsibilities Background Information

3. Building the Foundation NOAA Line/Staff Offices and Programs must build on Userubrics to establish a baseline and monitor progress Develop and implement common metadata management tools • Promote and highlight good examples a strong foundation of support across all of NOAA Support training specifically targeted at improving NOAA’s data documentation • Initiate teams to work on “special documentation problems” that cross Line and Staff Offices Encourage and support participation in the ISO and Open Geospatial Consortium (OGC)

4. Line / Staff Office & Program Responsibilities Data Collectors/Providers Data Users Data Stewards Standards Experts • 1. Identify Documentation Expertise • 2. Assess current state of documentation Metadata for Discovery Metadata for Use Metadata and Documentation for Understanding Documentation of Collections Documentation of Datasets Documentation of Services • 3. Create and Improve Metadata • 4. Publish/Preserve Metadata NOAA Data Centers / Archives

5. Building the Foundation The GOES-R Program recently expanded the scope of their metadata model from just ISO to ISO plus netCDF. This creates an integrated metadata framework that includes metadata in granules (files) as well as the ISO standards. This framework is important across NOAA and for many of our partners (NASA). • Initiate teams to work on “special documentation problems” that cross Line and Staff Offices

6. Building the Foundation The ISO 19115 metadata standard adopted by NOAA is currently undergoing a significant revision that addresses many requirements from NOAA and the netCDF-CF community that includes many NOAA data providers and partners. These changes will be adopted as part of the Draft International Standard which is about to be approved by ISO (Nov. 2011) Encourage and support participation in the ISO and Open Geospatial Consortium (OGC)

7. Building the Foundation ISO metadata standard trainings were held during June in Silver Spring as part of the annual EDMC Data Management Workshop and during October as part of the FGDC ISO Metadata Summit The GEO-IDE Wiki includes over 200 pages of information about the ISO Standards. Support training specifically targeted at improving NOAA’s data documentation

8. Building the Foundation A set of tools for translating metadata collections from FGDC to ISO have been developed and tested by NODC and NGDC. Tools for evaluating the completeness of ISO metadata that include links to the GEO-IDE wiki have been developed and tested at NGDC. Userubrics to establish a baseline and monitor progress Develop and implement common metadata management tools

9. What Goes Where?

10. Spiral Development / Training Metadata Content Independent of standard Spiral 1: Initial Content Check Back With Data Collectors/ Providers Check Back With Users Standard Guidance / Implementation Spiral 2-N: Scientific Questions New Requirements New Use Cases

11. Discovery Potential Spirals Understanding Identification Id Title Abstract Resource Date Topic Category Theme Keyword Metadata Contact Science Contact Extent Geospatial Bounding Box Temporal Start/End Vertical Min/Max Place Keywords Acquisition Information Instrument Platform Instrument Keywords Platform Keywords Content Information Attribute Type Attribute Names Attribute Definitions Attribute Units Connection OnlineResource: Name Description Function Application Schema Distribution Distributor Contact Online Resource Distribution Format Data Center Keywords Browse Graphic Text Searches Purpose Extent Description Lineage Statement Project Keywords Quality/Lineage Sources Process Steps Quality Reports / Coverages

12. Spirals and Rubrics G O A L S Progress Metadata Evaluation for Dataset X Date: 2010/10/24 Each record has a score that can be calculated using a consistent, standard tool

13. Spiral Development / Training: Rubrics

14. Questions?

15. Identify documentation expertise • Establish Data Stewardship Teams to facilitate documentation creation and improvement for appropriate organizational units or around programmatic needs. • Data Stewardship Teams should include the following expertise/skills: • Data Collectors/Providers • Data Users • Data Stewards • Standards Experts Line/Staff Office Responsibilities

16. Assess current state of documentation • Identify existing sources of documentation (Data Collectors/Providers) • Classify existing documentation into following categories (Data Stewards and Standards Experts): • Metadata for Discovery • Metadata for Use • Metadata and Documentation for Understanding • Documentation of Collections • Documentation of Datasets • Documentation of Services • Identify high-priority targets for improvement (All members of Data Stewardship Team) • Highlight best practices, successful teams and individuals (Line and Staff Office Management) Line/Staff Office Responsibilities

17. Create and Improve Metadata • Translate/transform existing metadata into the recommended representation (ISO 19139) • Create metadata for undocumented data and information • Use spiral approach for improving metadata Line/Staff Office Responsibilities

18. Publish/Preserve Metadata (Data Stewards) • Publish metadata record in new or existing Web-Accessible Folders or using a standard catalog service. This will make it possible to connect metadata records to various discovery portals using standard services • Work with NOAA Archives to ensure that documentation and metadata will be preserved for the long-term Line/Staff Office Responsibilities

19. Cross-NOAA Responsibilities • Encourageand support participation in the ISO and Open Geospatial Consortium (OGC) standards development and evolution processes • Develop and implement common metadata management tools including mechanisms for evaluating the completeness and quality of data documentation • Utilize rubrics to establish the baseline and monitor progress. • Engage users in providing feedback on data documentation efforts and opportunities. • Promote and highlight good examples of documentation and the individuals involved in their creation. • Support training specifically targeted at improving NOAA’s data documentation. • Initiate teams to work on “special documentation problems” that cross Line and Staff Offices

20. EDMC Responsibilities • Review this Procedural Directive twice a year to evaluate effectiveness and monitor progress. • Work with the CIO Council and the NOAA Observing Systems Council to implement and monitor progress on the Cross-NOAA responsibilities listed above. • Encourage and support partnerships with external organizations in the process of migration of metadata from FGDC to ISO Standards.

21. Measure Progress • An increase in the amount of NOAA environmental data and information that is well documented and discoverable via national and international discovery portals. • Improvement in the quality of Line and Staff Office and Program data documentation processes.