1 / 36

HDF-EOS Status, Related Tools and Issues

HDF-EOS Status, Related Tools and Issues. Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS. TOOLKIT / HDF-EOS Support. Maintenance support is funded on EMD Evolution through May 22, 2009 - Bridge contract until EODDIS Evolution & Development (EED) award in 2010 Current support

neila
Download Presentation

HDF-EOS Status, Related Tools and Issues

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS

  2. TOOLKIT / HDF-EOS Support • Maintenance support is funded on EMD Evolution through May 22, 2009 - Bridge contract until EODDIS Evolution & Development (EED) award in 2010 • Current support • HDF-EOS 2 & 5 • SDP/MTD Toolkit • HDF-EOS plug-in for HDFView • HDF-EOS2 to GeoTIFF Converter (HEG) • Other Toolkit/HDF-EOS related software • Toolkit/HDF-EOS/HEG helpdesk

  3. ECS Support of HDF-EOS • Both HDF4 and HDF5 based flavors of HDF-EOS are part of ECS baseline. • OS’s: Solaris (9, 10), Irix6.5 (64, n32), Linux (including 64-bit Opteron and Itanium), Mac OS X (Power PC and Mac Intel), Windows XP(MTD Toolkit, HDF-EOS2 & 5, HEG, HDFView) • added full Cygwin support for next release • Compilers: Fortran 77/90 & g77/pgf90 , C, C++, gcc, g++, Intel Fortran (SGI Altix), gfortran (Mac Intel)

  4. Downloadable Software SDP/MTD Toolkit (Science Data Processing Tools) HDF-EOS2 (HDF4 based) HDF-EOS5 (HDF5 based) heconvert (converts HDF4 - based Grid/Point/Swath structures to HDF-EOS 5 equivalents)

  5. Downloadable Applications HEG - GeoTIFF conversion - Spatial subsetting, Field Subsetting - reprojection - stitching, sub-sampling and more …. DumpMetadata (Dump HDF-EOS metadata) HDFView Plug-in, Java-based browser (HDF4, HDF5, HDF-EOS 2 and 5 access) HE5View*(HDF5 - based browser) EOSView*(HDF4 - based browser) *Dropped support for OS updates on July 2006

  6. Software Releases Toolkit/HDF-EOS: Last release – March 2008 TOOLKIT 5.2.15 HDF-EOS 2.15, HDF- EOS5.1.11 with HDF4.2r3, HDF5-1.6.7, Szip2.1 Next release – May 2009 TOOLKIT 5.2.16 HDF-EOS 2.16, HDF-EOS5.1.12 with new releases of HDF4 (2r4), 5 (1.8.3)

  7. Software Releases (Cont.) HEG (HDF-EOS2 to GeoTIFF Converter): Last release – August 2008 Version 2.9 Next release – May 2009 Version 2.10 Already have released Datapool version to the DAACs (Feb. 2009)

  8. Software Releases (Cont.) HDF-EOS Plug-in for HDFView: Last release – September 2008 Version 2.4 Next release – June 2009 For HDFView Version 2.5

  9. Software Releases (Cont.) HECONVERT (HDF-EOS2 to HDF-EOS5 Converter): Last releases – 2001, 2004 Version ? – Oct. 2008 Version 2.0 DumpMetadata (ASCII Dump of HDF-EOS metadata) Last release Dec. 2008 Version 1.0

  10. SDP Toolkit • Science Data Production (SDP) Toolkit is a collection of tools used by - Science software developers who produce code to process instrument data. - HDF-EOS userswho need to • Access metadata in HDF-EOS files • Perform Time/Date conversion • Access Digital Elevation Model Data • Access Ancillary Data, such as Digital Chart of the World database (DCW), Olson World Ecosystem files, DEM data files, etc.

  11. SDP Toolkit (cont.) • Tools provided in (SDP, MTD) Toolkit: - AA (Ancillary Data Access) - CBP (CELESTIAL Body Position) - CSC (Coordinate System Conversion) - CUC (Constant and Unit Conversions) - DEM (Digital Elevation Model Access) - EPH (Ephemeris Data Access) - GCT (Geo Coordinate Transformation) - IO (Input Output (File I/O)) - MEM (Memory Management) - MET (Metadata Access) - PC (Process Control) - SMF (Status Message File (Error/Status)) - TD (Time and Date Conversion)

  12. What is HDF-EOS ? • HDF-EOSis an extension to HDF which standardizes storage of and access to common Earth Sciences data structures • By providing a single interface to data structures common to earth science • By providing a container for EOS inventory, archive and product specific metadata • By establishing a consistent relationship between geolocation and science data.

  13. HDF-EOS 2 • HDF4 - based, storage format for EOS standard products. • Used operationally by MODIS, MISR, ASTER, Landsat, AIRS and other EOS instruments • Support for Grid/Point/Swath structures • HDF4 and HDF-EOS 2 will be supported by THG/NASA for the fore-seeable future.

  14. HDF-EOS 5 • Based on HDF5, a complete rewrite of HDF-EOS2 with a different user interface. • First released in 2000. • Format used byEOS Aurainstruments • Designed to ‘resemble’ HDF-EOS 2 to the maximum extent possible. • Supports same data structures • Added prefix ‘HE5_’ to HDF-EOS 2 functions.

  15. HDFView • HDFView: A Java-based browser from THG for HDF4 and HDF5 – based data. • HDF-EOS plug-in extends HDFView to browse HDF-EOS data in both versions. • The extension is a plug-in module and does not change THG HDFView functionality. All copy, save, editing, image display, etc. for HDF4 and 5 objects remains.

  16. HDFView • Identical look and feel when displaying HDF and HDF-EOS objects. • Users can access a complete breakdown of HDF-EOS objects. e.g. grid information, projection info., dimensions, datafields, and attributes are displayed.  • Current Support: SUN, SGI, LINUX, Mac, WINDOWS platforms

  17. Swath, Grid, and Point Objects

  18. AMSR-E Swath Tableview Features in Version 2.4

  19. HDF-EOS to GeoTIFF Converter (HEG) HEG: Utility that converts EOSDIS data from HDF-EOS format to common Geographical Information System (GIS)-compatible formats. • Two versions available: • A downloadable desktop version • Access through NASA archive online storage (Data Pools) • Implemented also in some prototypes: • Stand-alone Data Pool • OGC (Open Geospatial Consortium) for Web Services Chaining • DOWS (Deploy OGC Web Services) on the DAACs

  20. HEG Functionality • Conversion: • HDF-EOS (Swath or Grid ) to a single-band or multi-band GeoTIFF’s • HDF-EOS Swath toHDF-EOS Grid • HDF-EOS Swath or Grid to generic Binary • Metadatapreservation/creation. • Allows file selection from users local storage

  21. HEG Functionality • Subsetting (spatial, field, band) • Stitching (mosaicing) + subsetting + Reprojecting • Reprojection ( to UTM, PS, TM, STP, LCC, LAMAZ, GEO, SIN, Albers) • Subsampling (Subsample stacks) • Running from Command Line (batch jobs) or with Java-based GUI

  22. HEG Functionality • Currently supports MODIS, MISR, ASTER, AIRS and AMSR-E products on TERRA and AQUA (>110 products), • Operable on Sun, SGI, Win, Linux, MAC • Integrated into ECS Data Pool • Reduces the transfer time of HDF-EOS data sets (if subset is requested) • Provides the end-user with the exact file required by their application • User Interface • Portable, written in Java. • Not dependent on COTS (eg. IDL).

  23. Mt. Etna Eruption: MISR L1B Stitched Blocks

  24. 2004 Hurricane Francis, MODIS MOD021KM, Multi-Band GeoTIFF, 3 bands: EV_1KM_RefSB, Bands 1,2,5

  25. AST_L1B, 2003 California FiresGeo. projection, Multi-Band GeoTIFF, (3 bands: Visible Near Infrared (VNIR)) ….. and Subsetted Two Swaths Stitched …..

  26. TOOLKIT/HDF-EOS: SZIP Insatallation: Default decoder, Choice for encoder Choice for configuring HDF4 with/without SZIP Choice for disabling HDF4’s internal netCDF Choice for installing Toolkit/HDF-EOS in 32 or 64-bit modes in 64-bit Linux platforms (except for Itanium; no –m32 flag) Recent Major Development/Fixes

  27. Support for new releases of HDF4/HDF5/SZIP (HDF4.2r4, HDF5-1.8.X, SZIP-2.1) Auto Configuration capability (like HDF) for HDF-EOS2 and HDF-EOS5 Support for Mac Intel platform. Support for CYGWIN Recent Major Development/Fixes

  28. Added APIs GDij2ll and GDll2ij to HDF-EOS2 and HDF-EOS5 Added HDF_EOS 2 & 5 utilities for dumping grid lat/lon into ASCII or binary files Bug fixes such as: - HDFEOS5: Attribute data type inquiry (code returned not useful for reuse) - HDFEOS5: Tiling/comp. consistency between dataset and structmetadata - HDF-EOS2: Correct diminfo for a field (value in structmetadata is wrong?) - HDF-EOS5: wrong datatype info for INT8 data (C code) - Toolkit: Problem in pccheck utility( hangs if a blank space at a line start ) - Fix for ODL core dump (too many file names for Input Pointer) Recent Major Development/Fixes

  29. HDF-EOS Plug-in for HDFView: Current Support: SUN, LINUX, WINDOWS, and MAC (Power and Intel) Auto installation of plug-in into HDFView running self-extracting archives. Adding to plug-in the fixes and enhancement that went into HDFView2.4 and 2.5 Display image for selected pixels in the viewed tables Recent Major Development/Fixes

  30. Display Lat/Lon for a selected pixel in the data field tables Full Mac OSX support Improved Help Window New icons for activating Lat/Lon display for Swath & Grid Format conversion: HDF-EOS2 -> HDF-EOS5 Correction for compression and tiling values for grid fields Recent Major Development/Fixes

  31. HEG: Product Support: AMSR-E: Land products (CEA Projection added) AMSR-E 5 day Snow, and Monthly Snow Weekly Ocean, and Monthly Ocean Daily 12 Km, and Daily 6 Km Sea Ice L2 Rain, and Rain Grid ASTR_07XT, ASTR_09XT, ASTR_L1BE, MOD29E1D More new versions and products tested. Will update Users’ Guide and HEG web site Recent Major Development/Fixes

  32. Stitching fixes: MOD14A1 3-D fields (band selection) TM grids Multi-band geotif/HDF-EOS in Datapool Other Fixes: Interpolation correction with AIRS products containing fillvalue in the data fields Pixel size unit consistency in conversion and stitch GUIs. Recent Major Development/Fixes

  33. Correction for droping a column when stitching MODIS SIN tiles. Fixes for processing with default pixel size instead of ignoring it for Geographic projection Spatial subsetting fix for global AMSR-E garanules Fix for ignoring re-projection of type NN interpolation after stitching support pixel resampling for gridded data Recent Major Development/Fixes

  34. Future Plans TOOLKIT/HDF-EOS: Auto Configuration for TOOLKIT XML output for Metadata in TOOLKIT Update of DEM land-sea mask Support for HDF5-1.8.X Validation tool for HDF-EOS5 files

  35. Future Plans (cont.) HDF-EOS Plug-in for HDFView: More functionalities : - Ability to modify input HDF-EOS2 granules. - Ability to cut/paste objects, modify/delete attributes. - Ability to create new grid/swath HEG: - Support for AMSR-E Level 3 Land (CEA projection) - New MRT (MODIS Reprojection Tool) release updates - Support HEG for MAC Intel

  36. Availability Access to libraries, applications, Documentation: TOOLKIT: http://newsroom.gsfc.nasa.gov/sdptoolkit/toolkit.html HDFView: http://newsroom.gsfc.nasa.gov/sdptoolkit/HDFView/ HDFView_hdfeos_plugin.html HEG: http://newsroom.gsfc.nasa.gov/sdptoolkit/HEG/HEGHome.html Email Abe_Taaheri@raytheon.com Landover_PGSTLKIT@raytheon.com

More Related