110 likes | 202 Views
HDF5, HDF-EOS and Geospatial Data Archives. HDF and HDF-EOS Workshop VII September 24, 2003. Don Keefer Illinois State Geological Survey. Mike Folk U. of Illinois & NCSA. NARA’s data challenge.
E N D
HDF5, HDF-EOS andGeospatial Data Archives HDF and HDF-EOS Workshop VII September 24, 2003
Don Keefer Illinois State Geological Survey Mike Folk U. of Illinois & NCSA
NARA’s data challenge The National Archives and Records Administration (NARA) has the duty to preserve the nation’s history through archival storage and management of federal records. • Includes electronic records • Unprecedented volume, diversity, complexity • Must be able to organize, mine, query and access to satisfy public and government needs • Includes geospatial data
Project goals • Evaluate HDF5 and HDF-EOS as formats for archival storage of geospatial data • Need to represent many geosptatial data models • Need to accommodate large volumes of data • Need high performance data access • Need visualization and other applications
Established open standard Earth science focus Manages huge data files Scalablility Efficient storage High performance access Geospatial adaptability HDF5 storage structures HDF-EOS data models Format features Self-describing Alternate storage structures Extendable Software features Flexible API Portable library High quality Many applications/tools Why HDF5 and HDF-EOS?
1D, 2D, 3D array Raster image Vector features 2D Curtain Table Attribute HDF-EOS Grid HDF-EOS Swath HDF-EOS Point Digital orthophoto quad Digital elevation model Geologic well log Map Contoured surface Volume model Cross-section Color map Geospatial metadata Geospatial index Mapping Data Structures to Information Types Data structures Information types
What we are doing • Exploring use of HDF5 and HDF-EOS datatypes for non remote-sensed, geospatial data types • HDF-EOS grids for geospatial surfaces and volumes • HDF5 for raster (DOQ, DEM), vector (GIS), tabular data • Addressing format conversion • Map other formats to HDF5 and HDF-EOS • Convert from other formats to HDF5 and HDF-EOS • Examine performance and storage efficiency • Exploring visualization issues • Identifying technical requirements for visualization tools • Evaluating I/O efficiency
Acknowledgements This report is based upon work supported in part by the National Archives and Records Administration (NARA) through the grant NARA NSF 0202 GPG, by a Cooperative Agreement with the National Aeronautics and Space Agency (NASA) under NASA grant NAG 5-2040 and NAG NCCS-599, and by the Illinois State Geological Survey (ISGS). Other support provided by NCSA and other sponsors and agencies (http://hdf.ncsa.uiuc.edu/acknowledge.html). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NARA, NASA or the ISGS.