1 / 29

Current Status of HDF-EOS, Maintenance, Developments, and Tools

Current Status of HDF-EOS, Maintenance, Developments, and Tools. Abe Taaheri, Raytheon IIS HDF & HDF-EOS Workshop XI November 2007. TOOLKIT / HDF-EOS Support. Maintenance support is funded on EMD Evolution through May 2008 One year extension until May 2009 Current staff

kemal
Download Presentation

Current Status of HDF-EOS, Maintenance, Developments, and Tools

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Current Status of HDF-EOS,Maintenance, Developments, and Tools Abe Taaheri, Raytheon IIS HDF & HDF-EOS Workshop XI November 2007

  2. TOOLKIT / HDF-EOS Support • Maintenance support is funded on EMD Evolution through May 2008 • One year extension until May 2009 • Current staff - One full-time developer for maintaining: • SDP/MTD Toolkit • HDF-EOS2 & 5 • HDF-EOS plug-in for HDFView • HDF-EOS2 to GeoTIFF Converter (HEG) • Other Toolkit/HDF-EOS related software • Toolkit/HDF-EOS/HEG helpdesk - One half-time developer for HEG maintenance

  3. ECS Support of HDF-EOS • Both HDF4 and HDF5 based flavors of HDF-EOS are part of ECS baseline. • OS’s: Solaris (9, 10), Irix6.5 (64, n32), Windows XP, Linux (including 64-bit Opteron and Itanium), Mac OS X (MTD Toolkit, HDF-EOS2 & 5) (Dropped support for Solaris 8, HP, HP11, Dec, IBM for next release) • adding Mac Intel support for 2007 release and full Cygwin support for 2008 • Compilers: Fortran 77/90 & g77/pgf90 , C, C++, gcc, g++

  4. Downloadable Software and Applications • SDP/MTD Toolkit (Science Data Processing Tools) • HDF-EOS2 (HDF4 based) • HDF-EOS5 (HDF5 based) • HE5View*(HDF5 - based browser) • EOSView*(HDF4 - based browser) • HDFView, Java-based browser (HDF4, HDF5, HDF-EOS 2 and 5 access) • heconvert (converts HDF4 - based Grid/Point/Swath structures to HDF-EOS 5 equivalents) • HEG (GeoTIFF conversion, Spatial subsetting, Field Subsetting, reprojection, stitching, sub-sampling, etc.) *Dropped support for OS updates on July 2006

  5. Software Releases Toolkit/HDF-EOS: • Last release – March 2006 - TOOLKIT 5.2.14, HDF-EOS 2.14, HDF-EOS5.1.10 - with HDF4.2r1, HDF5-1.6.5-post5, Szip2.0 • Next release – November 2007 - TOOLKIT 5.2.15, HDF-EOS 2.15, HDF-EOS5.1.11 - with new releases of HDF4, HDF5, and SZIP HDF-EOS Plug-in for HDFView: • Last release – May 2006 • Version 2.3 • Next release – Dec. 2007 • HDFView2.4 was released October 31 by The HDF Group

  6. Software Releases (Cont.) HEG (HDF-EOS2 to GeoTIFF Converter): • Last release – July 2007 • Version 2.8 • Next release – May 2008 • Version 2.9

  7. SDP Toolkit • Science Data Production (SDP) Toolkit is a collection of tools used by - Science software developers who produce code to process instrument data. - HDF-EOS userswho need to • Access metadata in HDF-EOS files • Perform Time/Date conversion • Access Digital Elevation Model Data • Access Ancillary Data, such as Digital Chart of the World database (DCW), Olson World Ecosystem files, DEM data files, etc.

  8. SDP Toolkit (cont.) • Tools provided in (SDP, MTD) Toolkit: - AA (Ancillary Data Access) - CBP (CELESTIAL Body Position) - CSC (Coordinate System Conversion) - CUC (Constant and Unit Conversions) - DEM (Digital Elevation Model Access) - EPH (Ephemeris Data Access) - GCT (Geo Coordinate Transformation) - IO (Input Output (File I/O)) - MEM (Memory Management) - MET (Metadata Access) - PC (Process Control) - SMF (Status Message File (Error/Status)) - TD (Time and Date Conversion)

  9. What is HDF-EOS ? • HDF-EOSis an extension to HDF which standardizes storage of and access to common Earth Sciences data structures • By providing a single interface to data structures common to earth science • By providing a container for EOS inventory, archive and product specific metadata • By establishing a consistent relationship between geolocation and science data.

  10. HDF-EOS 2 • HDF4 - based, storage format for EOS standard products. • Used operationally by MODIS, MISR, ASTER, Landsat, AIRS and other EOS instruments • Support for Grid/Point/Swath structures • HDF4 and HDF-EOS 2 will be supported by THG/NASA for the fore-seeable future.

  11. HDF-EOS 5 • Based on HDF5, a complete rewrite of HDF-EOS2 with a different user interface. • First released in 2000. • Format used byEOS Aurainstruments • Designed to ‘resemble’ HDF-EOS 2 to the maximum extent possible. • Supports same data structures • Added prefix ‘HE5_’ to HDF-EOS 2 functions.

  12. HDFView • HDFView: A Java-based browser from THG for HDF4 and HDF5 – based data. • HDF-EOS plug-in extends HDFView to browse HDF-EOS data in both versions. • The extension is a plug-in module and does not change THG HDFView functionality. All copy, save, editing, image display, etc. for HDF4 and 5 objects remains. • Identical look and feel when displaying HDF and HDF-EOS objects. • Users can access a complete breakdown of HDF-EOS objects. e.g. grid information, projection info., dimensions, datafields, and attributes are displayed.  • Current Support: SUN, SGI, LINUX, Mac, WINDOWS platforms

  13. Swath, Grid, and Point Objects

  14. AMSR-E and OMI Point objects

  15. AMSR-E Swath New Features in Next Release Version 2.4

  16. HDF-EOS to GeoTIFF Converter (HEG) HEG: Utility that converts EOSDIS data from HDF-EOS format to common Geographical Information System (GIS)-compatible formats. • Two versions available: • A downloadable desktop version • Access through NASA archive online storage (Data Pools) • Implemented also in prototypes for Stand-alone Data Pool, OGC (Open Geospatial Consortium) for Web Services Chaining, and DOWS (Deploy OGC Web Services) on the DAACs

  17. HEG Functionality • Conversion: • HDF-EOS (Swath or Grid ) to a single-band or multi-band GeoTIFF’s • HDF-EOS Swath toHDF-EOS GridandSwath • HDF-EOS Swath or Grid to generic Binary (with metadata file) • Subsetting (spatial, field, band) • Stitching (mosaicing) + subsetting + Reprojecting • Reprojection ( to UTM, PS, TM, STP, LCC, LAMAZ, GEO, SIN, Albers) • Subsampling (Subsample stacks) • Running from Command Line (batch jobs) or with Java-based GUI

  18. HEG Functionality • Metadata preservation/creation. • Currently supports MODIS, MISR, ASTER, AIRS and AMSR-E products on TERRA and AQUA (>110 products), • Allows file selection from users local storage • Operable on Sun, SGI, Win, Linux, MAC • Integrated into ECS Data Pool • Reduces the transfer time of HDF-EOS data sets (if subset is requested) • Provides the end-user with the exact file required by their application • User Interface • Portable, written in Java. • Not dependent on COTS (eg. IDL).

  19. Mt. Etna Eruption: MISR L1B Stitched Blocks

  20. 2004 Hurricane Francis, MODIS MOD021KM, Multi-Band GeoTIFF, 3 bands: EV_1KM_RefSB, Bands 1,2,5

  21. AST_L1B, 2003 California FiresGeo. projection, Multi-Band GeoTIFF, (3 bands: Visible Near Infrared (VNIR)) ….. and Subsetted Two Swaths Stitched …..

  22. Major Development/Fixes Next Release: November 2007 • SZIP Insatallation: Default decoder, Choice for encoder • Choice for configuring HDF4 with/without SZIP • Choice for disabling HDF4’s internal netCDF • Choice for installing Toolkit/HDF-EOS in 32 or 64-bit modes in 64-bit Linux platforms • Support for new releases of HDF4/HDF5/SZIP (HDF4.2r2, HDF5-1.6.6, SZIP-2.1) • Installation fix for a problem caused by limits.h in some Linux platforms • Auto Configuration capability (like HDF) added for both HDF-EOS2 and HDF-EOS5 • Support for Mac Intel platform. Drop of support for Sun5.8, Dec, HP, HP 11, IBM • Fixes for Lat/Lon to pixel conversions in DEM tools for Linux to get results similar to those in Unix platforms

  23. Major Development/Fixes (cont.) HDF-EOS Plug-in for HDFView: Current Support: SUN, SGI, LINUX, WINDOWS, and MAC platforms Last Release: Version 2.3 • Display Point Objects • Display pixel's lat/lon for swath and grid images • Scientific notation in Table data Next Release: Version 2.4 • Auto installation of plug-in into HDFView running self-extracting archives. • Display image for selected pixels in the viewed tables • Display Lat/Lon for a selected pixel in the data field tables • Mac OSX fixes for one-button mice • Better display of Plug-in User’s Guide • New icons for activating Lat/Lon display for Swath & Grid

  24. Major Development/Fixes (cont.) HEG: Last Release (Version 2.8) • Added Support for: MOD02SSH, MYD02SSH, MOD14A1.v005, MYD14A1.005 MOD29P1D.V005, MOD08_M3.v005, MOD11_L2.v005 AMSR-E Snow and Ice, AMSR-E Ocean • MISR Bug fixes & enhancements: • Added “No-reprojection” for ouput in SOM projection • Added “Nearest Neighbour” for sampling type • Fixed problem with missing metadata for HDF-EOS outputs • Added Albers Projection • New "Subsetter" functionality for Parameters (aka Fields). This results in: • Output to HDF-EOS SWATH from input HDF-EOS SWATH • No alteration of input data because of projection operation.

  25. HDF-EOS/Toolkit Future Plans: • Auto Configuration for TOOLKIT • Full Support for Cygwin platform • Support for HDF5-1.8 • Performance improvement, if possible, for reading HDF format ephemeris/attitude files in Toolkit • Performance improvement in HDF-EOS5 by redesigning some functions • More functionalities in HDF-EOS Plug-in for HDFView: - Format conversion: HDF-EOS2 -> HDF-EOS5 - Improved Help Window - Ability to modify input HDF-EOS2 granules. - Ability to cut/paste objects, modify/delete attributes. - Ability to create new grids/swaths

  26. Availability • Access to libraries, applications, Documentation: TOOLKIT: http://newsroom.gsfc.nasa.gov/sdptoolkit/toolkit.html HDFView: http://newsroom.gsfc.nasa.gov/sdptoolkit/HDFView/ HDFView_hdfeos_plugin.html HEG: http://newsroom.gsfc.nasa.gov/sdptoolkit/HEG/HEGHome.html • Email • Abe_Taaheri@raytheon.com • Landover_PGSTLKIT@raytheon.com

  27. Major Development/Fixes Last Release HDF-EOS5: • Added a routine to return list of aliases for a field • Used HDF5’s H5T_C_S1 datatype for string type • Added Fortran wrapper for SWwritedatameta() routine • Replaced symbol “H5E_NONE_MAJOR” with other symbols for better description of errors • Ported to Solaris 9 and Solaris 10 • Fixed POINT object for: - linkage between levels - updating levels - writing Forward and Backward Linkage data - char field inquiry

  28. Major Development/Fixes (cont.) Last Release • Fixed subsetting problem in HDF-EOS5 Grid (correction to conversion of DMS degree to Decimal degree). • Set LD_LIBRARY_PATH for SZIP/HDF5 shared libraries in installation and environment setting scripts TOOLKIT/HDF-EOS2: • Ported to Solaris 9 and Solaris 10 • Gdapi.c changes for adding more significant digits after the decimal point to projection parameters if needed. • chkeph utilityfixes for reading binary ephemeris files (files with different endiannes than the machine) • SGI 7.4.2 compiler support • orbsim fixes for Linuxto create files similar to those in UNIX platforms (Random number generation fixes)

More Related