1 / 22

Archival Information Packages for NASA HDF-EOS Data

Archival Information Packages for NASA HDF-EOS Data. R. Duerr, Kent Yang, Azhar Sikander. Outline. What is an Archival Information Package? HDF-AIP Standards? What Standards? METS DIF/FGDC/ISO 19115-2 PREMIS Results Next Steps. OAIS Reference Model 1. Archive Information Package.

maize
Download Presentation

Archival Information Packages for NASA HDF-EOS Data

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Archival Information Packages for NASA HDF-EOS Data R. Duerr, Kent Yang, Azhar Sikander

  2. Outline • What is an Archival Information Package? • HDF-AIP • Standards? What Standards? • METS • DIF/FGDC/ISO 19115-2 • PREMIS • Results • Next Steps Archival Information Packages for NASA HDF-EOS Data, presented 11/4/09 by R. Duerr HDF and HDF-EOS Workshop XIII

  3. OAIS Reference Model1 Archive Information Package 1 Reference Model for an Open Archival Information System (OAIS), CCSDS 650.0-B-1, Blue Book, January 2002. Archival Information Packages for NASA HDF-EOS Data, presented 11/4/09 by R. Duerr HDF and HDF-EOS Workshop XIII

  4. Archival Information Package Contents • Content Information • The data object to be preserved • Information that describes the data object • Typically interpreted as the syntax and semantics of the file structure • Preservation Description Information • Provenance –Origin or source of the data, any changes that have taken place since, and who has had custody of it • Fixity – the authentication mechanisms (with keys) needed to ensure that the data object has not been altered in an undocumented manner • Reference – identification mechanisms and values • Context – relation of the object to its environment Archival Information Packages for NASA HDF-EOS Data, presented 11/4/09 by R. Duerr HDF and HDF-EOS Workshop XIII

  5. HDF-Archive Information Packages http://www.hdfgroup.org/projects/hdf5_aip/hdf5_aip_wp.html The HDF group was funded to investigate and propose a design for a complete archival information package for HDF data files The result was a METS metadata file to accompany the HDF data file Archival Information Packages for NASA HDF-EOS Data, presented 11/4/09 by R. Duerr HDF and HDF-EOS Workshop XIII

  6. Metadata Standards - METS • Metadata Encoding and Transmission Standard • An initiative of the Digital Library Federation • Provides the means to convey the metadata necessary for • management of digital objects within a repository • exchange of objects between repositories (or between repositories and their users) • Designed to facilitate • shared development of information management tools/services • interoperable exchange of digital materials Archival Information Packages for NASA HDF-EOS Data, presented 11/4/09 by R. Duerr HDF and HDF-EOS Workshop XIII

  7. METS - A very brief overview Describes the METS document itself e.g., creator or editor Describes the objectusing some external standard e.g., MARC, FGDC, Dublin Core Describes object creation, storage, intellectual property rights, source info, provenance, etc. e.g., PREMIS Provides an inventory of all of the files that are part of the object described A physical or logical map of theorganization of the materials described Allows specification of hyperlinksbetween parts of the map (mostlyuseful when preserving websites) Used to associate executable codewith parts of the content Archival Information Packages for NASA HDF-EOS Data, presented 11/4/09 by R. Duerr HDF and HDF-EOS Workshop XIII

  8. Metadata Standards - Descriptive Metadata • Discovery, Assess and Access Metadata • GCMD DIF • FGDC CSDGM • ISO 19115 Derived from Archival Information Packages for NASA HDF-EOS Data, presented 11/4/09 by R. Duerr HDF and HDF-EOS Workshop XIII

  9. Metadata Standards - ISO 19115:2003 • The international equivalent of the FGDC standard • Most fields can be mapped or generated from FGDC metadata • The exception is the Dataset Topic Keywords • Allows for national profiles Archival Information Packages for NASA HDF-EOS Data, presented 11/4/09 by R. Duerr HDF and HDF-EOS Workshop XIII

  10. Metadata Standards - ISO 19115:2003 Archival Information Packages for NASA HDF-EOS Data, presented 11/4/09 by R. Duerr HDF and HDF-EOS Workshop XIII

  11. Is there a metadata standard for AIP information? Archive Information Package 1 Reference Model for an Open Archival Information System (OAIS), CCSDS 650.0-B-1, Blue Book, January 2002. Archival Information Packages for NASA HDF-EOS Data, presented 11/4/09 by R. Duerr HDF and HDF-EOS Workshop XIII

  12. Preservation Metadata Implementation Strategies (PREMIS) • Provide a core preservation metadata set with broad applicability across the digital preservation community • Developed by an OCLC and RLG sponsored international working group • Representatives from libraries, museums, archives, government, and the private sector. • Maintained by the Library of Congress • Based on the OAIS reference model Archival Information Packages for NASA HDF-EOS Data, presented 11/4/09 by R. Duerr HDF and HDF-EOS Workshop XIII

  13. PREMIS - Entity-Relationship Diagram Intellectual Entities “an action that involves atleast one object or agentknown to the preservationrepository” e.g., created, archived,migrated Rights “a person, organization, orsoftware program associatedwith preservation events inthe life of an object”e.g., Dr. Spock donated it “a discrete unit of information in digital form” For example, a data file “a coherent set of contentthat is reasonablydescribed as a unit” For example, a web site, data set or collection of data sets Objects Agents “assertions of one or more rights or permissionspertaining to an objector an agent” e.g., copywrite notice, legalstatute, deposit agreement Events Archival Information Packages for NASA HDF-EOS Data, presented 11/4/09 by R. Duerr HDF and HDF-EOS Workshop XIII

  14. Is there a metadata standard for AIP information? PREMIS ISO 19115 1 Reference Model for an Open Archival Information System (OAIS), CCSDS 650.0-B-1, Blue Book, January 2002. Archival Information Packages for NASA HDF-EOS Data, presented 11/4/09 by R. Duerr HDF and HDF-EOS Workshop XIII

  15. NOAA Data Stewardship Prototype Technologies change regularly, organizations come and go, but data must survive But preserving data takes more than just preserving the bits, all the components of an AIP are critical NSIDC and THG demonstrated the feasibility of migrating NASA data to a standard HDF-AIP format Motivation: Archival Information Packages for NASA HDF-EOS Data, presented 11/4/09 by R. Duerr HDF and HDF-EOS Workshop XIII

  16. Project Goals • Prototype development of Archive Information Packages for HDF data: • For entire data sets • For individual “granules” • Test usability of digital library standards with geospatial data Archival Information Packages for NASA HDF-EOS Data, presented 11/4/09 by R. Duerr HDF and HDF-EOS Workshop XIII

  17. ISO-19115 CDM/NetCDF4 ECS to METS (Data Set) HDF5-AIP NSIDC/ECSMetadata ECS to METS (Granule) NetCDF4 / HDF5 Data METS NetCDF4 / HDF5 Data NSIDC/ ECS HDF4-data NSIDC/ ECS HDF4-data H4toH5 H4toH5 NetCDF4/HDF5-data NetCDF4/HDF5-data Program Plan (Modified) Archival Information Packages for NASA HDF-EOS Data, presented 11/4/09 by R. Duerr HDF and HDF-EOS Workshop XIII

  18. HDF5 Granule Level Archive Information Packages Data file HDF5 METS Metadata file Primary SchemaExtension Schema |<mets> |---<dmdSec>----------------<ISO 19115> |---<amdSec>--------------|--<techMD> | |--<rightsMD> PREMIS | |--<sourceMD> |----<fileGrp> |----<structMap> HDF5 AIP Components http://www.hdfgroup.uiuc.edu/papers/papers/AIP/HDF5_AIP_White_Paper.pdf Archival Information Packages for NASA HDF-EOS Data, presented 11/4/09 by R. Duerr HDF and HDF-EOS Workshop XIII

  19. File Level AIP Activity Status • Developed a map from NSIDC/ECS metadata to METS/PREMIS/ISO 19115 components • Prototype software completed • Issues • What goes in PREMIS vs ISO 19115? • Auxillary file handling - own AIP or not? • E.g., browse files, processing history, PGE’s • Granules vs files Archival Information Packages for NASA HDF-EOS Data, presented 11/4/09 by R. Duerr HDF and HDF-EOS Workshop XIII

  20. Issues and Questions • Inconsistent use of terminology between standards – for example, what is a data set? • Many of the standards care about distribution formats • Are these even relevant concepts any more? • Do you really want to have to update the metadata record just because a new distribution format was added? • What about new access services? Archival Information Packages for NASA HDF-EOS Data, presented 11/4/09 by R. Duerr HDF and HDF-EOS Workshop XIII

  21. Next Steps NSIDC is updating our non-ECS data systems handling of metadata including support for PREMIS, etc. metadata on all holdings Work underway to upgrade granule level metadata for NSIDC flagship sea ice products (PREMIS/METS/ISO AIP packages) Work to improve archivability of data stored in HDF formats on-going – NASA implementing a standard XML description of contents across its archives Archival Information Packages for NASA HDF-EOS Data, presented 11/4/09 by R. Duerr HDF and HDF-EOS Workshop XIII

  22. Acknowledgement This work was supported under NOAA Scientific Stewardship Program grant number NA07OAR4310286. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of NOAA.  Archival Information Packages for NASA HDF-EOS Data, presented 11/4/09 by R. Duerr HDF and HDF-EOS Workshop XIII

More Related