1 / 11

Improving long-term preservation EOS data by independently mapping HDF data objects

Improving long-term preservation EOS data by independently mapping HDF data objects. Mike Folk, Peter Cao, Kent Yang Ruth Duerr Christopher Lynnes. Two projects. Productizing HDF4 Mapping schema and tools for deployment Implement HDF5 mapping prototype.

carwile
Download Presentation

Improving long-term preservation EOS data by independently mapping HDF data objects

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Improving long-term preservation EOS data by independently mapping HDF data objects Mike Folk, Peter Cao, Kent YangRuth DuerrChristopher Lynnes

  2. Two projects • Productizing HDF4 Mapping schema and tools for deployment • Implement HDF5 mapping prototype Annual HDF Briefing to ESDIS

  3. I. Productizing HDF4 Mapping schema and tools for deployment Annual HDF Briefing to ESDIS

  4. Productizing HDF4 mapping • Investigate integration of mapping schema with existing standards • Determine HDF-EOS 2 requirements • Redesign the XML schema • Implement production quality HDF4 map writer • Develop demo HDF4 map reader and optional general reader • Develop test suite and validation utility for layouts and datatypes • Deploy tools at other NASA data centers. Annual HDF Briefing to ESDIS

  5. HDF4/HDF-EOS2 mapping workflow Annual HDF Briefing to ESDIS

  6. Estimated cost in fte The HDF Group: 29.8 fte months GES: 2.9 fte months + time for technology infusion NSIDC: 3.3 fte months + 3 months summer student + time for technology infusion Project duration: approximately 32 months Annual HDF Briefing to ESDIS

  7. HDF-EOS 2 analysis • Some crucial HDF-EOS2 information is not directly available in a prototype-mapping file • Example: • Latitude and longitude in an HDF-EOS2 grid are provided implicitly by projection parameters in an EOS2-specific HDF4 attribute “StructMetadata.” • When apps use the HDF-EOS2 library to access a grid, the library computes the lat/lon at each data point based on the StructMetadata value. • It would be burdensome for apps to implement on their own these computations for all projections. • Currently, the StructMetadata attribute is in the mapping file, but no interpretation of lat and lon is provided. Annual HDF Briefing to ESDIS

  8. HDF-EOS 2 analysis Hence the project includes a study of the mapping requirements for HDF-EOS2 grid, swath, and point objects. Will do a comprehensive analysis of issues related to the handling of NASA HDF-EOS2 files, aimed at making crucial EOS-specific data easily accessible to apps through the same tools used to access HDF4 files. Annual HDF Briefing to ESDIS

  9. II. Implement HDF5 mapping prototype Annual HDF Briefing to ESDIS

  10. Implement HDF5 mapping prototype • Assess and categorize EOS data products in HDF5. • Investigate methods to map objects in HDF5 files, and evaluate effort needed to implement methods. • Identify tools to facilitate creating and using maps; investigate requirements to develop such tools. • Carry out a proof of concept with selected files • Create maps, read back data using the maps • Create prototype tools to create map and read back map • Present results and options to the community • Assess usefulness of approach, desired modifications. • Evaluate the effort required for a full solution. • Submit a proposal for a full solution. Annual HDF Briefing to ESDIS

  11. Estimated costs The HDF Group: 7 months fte GES: 1.3 months fte NSIDC: 4.4 months fte Annual HDF Briefing to ESDIS

More Related