1 / 35

HDF-OPeNDAP Project Update

Learn about the latest updates and enhancements to the OPeNDAP project for HDF. Access HDF files via OPeNDAP, visualize data using OPeNDAP clients, and explore new solutions for handling HDF4 and HDF5 files.

jjanis
Download Presentation

HDF-OPeNDAP Project Update

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. HDF-OPeNDAP Project Update Joe Lee and Kent Yang The HDF Group James Gallagher OPeNDAP, Inc. The 13th HDF and HDF-EOS Workshop November 3-5, 2009 HDF/HDF-EOS Workshop XIII

  2. Outline • Introduction to OPeNDAP • What’s New? • HDF5-OPeNDAP Handler (minor) • HDF4-OPeNDAP Handler (major) HDF/HDF-EOS Workshop XIII

  3. Access HDF via FTP Aqua/Aura Users IDL AIRS/OMI FTP FTP Clients FTP Servers HDF Files Local HDF Files HDF/HDF-EOS Workshop XIII

  4. OPeNDAP Concept Typical Remote Access View Data (User) Remote Data (HDF4/5) FTP/HTTP Visualization Tools (gradsdap) Visualization tools (e.g., grads) Handler (hdf4/5_handler) Client Library (libnc_dap) Server (Hyrax) Data Access Protocol (DAP) via Internet HDF/HDF-EOS Workshop XIII

  5. Access HDF via OPeNDAP Aqua/Aura Users Hyrax HDF HANDLER IDV libnc-dap AIRS/OMI DAP Visualization Tools OPeNDAP Clients OPeNDAP Servers HDF Files HDF/HDF-EOS Workshop XIII

  6. What’s New? HDF5-OPeNDAP Handler Work HDF/HDF-EOS Workshop XIII

  7. HDF5 Handler Update Aura Users HDF5 HANDLER Update Hyrax IDV libnc-dap OMI Swath OMI Grid DAP Visualization Tools OPeNDAP Clients OPeNDAP Servers HDF5 Files HDF/HDF-EOS Workshop XIII

  8. HDF5-OPeNDAP Handler Update No XDef file for GrADS users! RadiativeCloudFraction A3RadiativeClou Support Aura HDF-EOS5 OMI Swath Access some files that follow HDF5 Dimension Scale Specification Change in Short Name Option HDF/HDF-EOS Workshop XIII

  9. Summary: Updated HDF5 Handler Official Release: October 6, 2009 http://opendap.org/hdf5_handler HDF/HDF-EOS Workshop XIII

  10. What’s New? HDF4-OPeNDAP Handler Work HDF/HDF-EOS Workshop XIII

  11. HDF4-OPeNDAP Handler Work Problems with the old HDF4 Handler Our Work Limitations Future Work HDF/HDF-EOS Workshop XIII

  12. Old HDF4 Handler Problem OPeNDAP Visualization Tools CAN NOT display most NASA HDF-EOS2 and HDF4 data HDF/HDF-EOS Workshop XIII

  13. Old HDF4 Handler Problem Aqua Users Hyrax Old HDF4 HANDLER IDV libnc-dap AIRS Grid DAP Visualization Tools OPeNDAP Clients OPeNDAP Servers HDF4 Files HDF/HDF-EOS Workshop XIII

  14. Our Work Let Visualization Clients Access HDF-EOS2 and HDF4 files HDF/HDF-EOS Workshop XIII

  15. What’s Missing? • My house in Hawaii costs $1M. • Information – Earth Science • Geo-Location, Geo-Location, Geo-Location • Unit, Scale factor • My house costs 1. • Data – Computer Science • int my_house = 1; // Use boolean to save? HDF/HDF-EOS Workshop XIII

  16. What’s Missing? • Clients expect certain data structures • DAP Grids and Shared dimensions • Clients expect some key attributes • CF conventions HDF/HDF-EOS Workshop XIII

  17. Solution: Metadata Parser • Goal: Meet the Client’s Requirements • Parse the structMetadata to • generate the DAP Grids and Shared Dims • generate the CF-compliant attributes HDF/HDF-EOS Workshop XIII

  18. New HDF4 Handler (parser) Aqua Users New HDF4 HANDLER (parser) Hyrax Old HDF4 HANDLER IDV libnc-dap AIRS Grid DAP Visualization Tools OPeNDAP Clients OPeNDAP Servers HDF4 Files HDF/HDF-EOS Workshop XIII

  19. Demo: Parser Solution Problem: Only a few HDF-EOS2 files work! AIRS Grid AIRS.2008.10.27.L3.RetStd001.v5.2.2.0.G08303124144.hdf http://daac.gsfc.nasa.gov/daac-bin/whom/mk_page_cgi.pl?PATH=datapool/AIRS/Level3_V005/01_AIRX3STD/2008/10 AIRS Swath ftp://airspar1u.ecs.nasa.gov/data/s4pa//Aqua_AIRS_Level2/AIRX2RET.005//2008/330/AIRS.2008.11.25.143.L2.RetStd.v5.2.2.0.G08331121849.hdf MODIS Grid MOD08_D3.A2008284.005.2008289105354.hdf http://modis-atmos.gsfc.nasa.gov/MOD08_D3/acquiring.html MODIS Swath MOD06_L2.A2001222.0025.004.2003127111841.hdf http://modis-atmos.gsfc.nasa.gov/MOD06_L2/acquiring.html HDF/HDF-EOS Workshop XIII

  20. New Challenges for HDF4 Handler • Hundreds of HDF-EOS2 and HDF4 products • Grid Projections • Swath Dimension Map HDF/HDF-EOS Workshop XIII

  21. New Challenges • HDF-EOS2 Grid projections • Geographic • Sinusoidal • Polar Stereographic • etc. HDF-EOS2 Swath Dimension Map HDF/HDF-EOS Workshop XIII

  22. Solution: Use HDF-EOS2 Library • Goal: Support more HDF-EOS2 products • Use HDF-EOS2 Library to • handle different Grid projections • handle Swath dimension map HDF/HDF-EOS Workshop XIII

  23. New HDF4 Handler (Parser) New HDF4 Handler (EOS2 library) New HDF4 HANDLER (parser) New HDF4 HANDLER (EOS2 library) Aura Users MODIS Swath w/ Dim. Map Hyrax IDV libnc-dap MODIS Swath DAP Visualization Tools OPeNDAP Clients OPeNDAP Servers HDF4 Files HDF/HDF-EOS Workshop XIII

  24. Demo: HDF-EOS2 Library Solution Swath with Dimension Map HDF/HDF-EOS Workshop XIII

  25. Two Solutions Metadata Parser HDF-EOS2 Library Pro: More products Con: HE2 library installation • Pro: Simple installation • Con: Limited products HDF/HDF-EOS Workshop XIII

  26. Two Solutions - Implementation • Adapted the implementation of previous • software packages Metadata Parser HDF-EOS2 Library h4toh5 tool • hdf5_handler HDF/HDF-EOS Workshop XIII

  27. Solution Comparison HDF/HDF-EOS Workshop XIII

  28. Live Demo • Tutorial Example • http://hdfeos.org/software/idv.php • Demo Servers • http://hdfgroup.org/projects/opendap • http://hdfgroup.org/projects/opendap4 HDF/HDF-EOS Workshop XIII

  29. Limitations • 7 Reasons for Visualization Failures • Sample Files for Visualization Failures • http://hdfgroup.org/projects/opendap4/limits.html HDF/HDF-EOS Workshop XIII

  30. The Tip of the Iceberg The current HDF4 handler CAN NOT visualize most of NASA HDF-EOS2 / HDF4 files HDF/HDF-EOS Workshop XIII

  31. More Challenges for HDF4 Handler • Tackle limitations and other challenges. • Handle NASA HDF4 (e.g.,CERES) files. • In 2010 HDF/HDF-EOS Workshop XIII

  32. Summary: Enhanced HDF4 Handler Official Release on Oct. 6, 2009. http://opendap.org/hdf4_handler Support some HDF-EOS2 Grid / Swath files Use HDF-EOS2 library Future work: support more NASA HDF-EOS2 / HDF4 files HDF/HDF-EOS Workshop XIII

  33. Thank You! HDF/HDF-EOS Workshop XIII

  34. Acknowledgements This work was supported by cooperative agreement number NNX06AC83A, NNX08AO77A and NNX06AG75A from the National Aeronautics and Space Administration (NASA). Any opinions, findings, conclusions, or recommendations expressed in this material are those of the author[s] and do not necessarily reflect the views of the National Aeronautics and Space Administration. HDF/HDF-EOS Workshop XIII

  35. Questions/comments? HDF/HDF-EOS Workshop XIII

More Related