1 / 36

Support for NPP/NPOESS by The HDF Group

Support for NPP/NPOESS by The HDF Group. Mike Folk, Elena Pourmal, Peter Cao The HDF Group November 5, 2009. To develop and adapt HDF5 software and provide HDF5 support for NPP/NPOESS. Goal. Areas of immediate need. Ensure easy and intuitive data access and use by diverse communities

kristen-wong
Download Presentation

Support for NPP/NPOESS by The HDF Group

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Support for NPP/NPOESS by The HDF Group Mike Folk, Elena Pourmal, Peter Cao The HDF Group November 5, 2009 HDF/HDF-EOS Workshop XIII

  2. To develop and adapt HDF5 software and provide HDF5 support for NPP/NPOESS. Goal HDF/HDF-EOS Workshop XIII

  3. Areas of immediate need • Ensure easy and intuitive data access and use by diverse communities • Enable producers and consumers to view content, manage metadata, and convert data to other formats. • Provide high-quality, rapid-response support of HDF5 for NPOESS users at all levels. HDF/HDF-EOS Workshop XIII

  4. 2009-2009 Priorities • Data accessibility and usability • Developed library of high level APIs to support NPP/NPOESS data management • Tool development • Modify h5dump to enable viewing of region references and quality flags • Modify HDFView to enhance viewing of references and quality flags • System maintenance • User support HDF/HDF-EOS Workshop XIII

  5. NPP/NPOESS Project Information • Project Web site • http://www.hdfgroup.org/projects/npoess/ • Project description • Released software • Documentation HDF/HDF-EOS Workshop XIII

  6. NPOESS Web site HDF/HDF-EOS Workshop XIII

  7. NPOESS Software FTP Site HDF/HDF-EOS Workshop XIII

  8. NPOESS Project Information • Project Wiki site • http://wiki.hdfgroup.org/NPOESS-Project • Monthly progress reports, links to software under development, presentations, etc. • Requires a password • Contact epourmal@hdfgroup.org HDF/HDF-EOS Workshop XIII

  9. HDF NPOESS Wiki HDF/HDF-EOS Workshop XIII

  10. Library of High-Level APIs to support NPOESS data HDF/HDF-EOS Workshop XIII

  11. NPOESS High-Level Library • Library to support HDF5 region references and packed bits in support of NPOESS quality flags • C APIs • Documentation • Tests • Examples • Source available from • ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/NPOESS/source/ • Documentation is available from • http://www.hdfgroup.org/projects/npoess/ • Still a prototype! HDF/HDF-EOS Workshop XIII

  12. Documentation HDF/HDF-EOS Workshop XIII

  13. h5dumpNew Features to Support Region References and Quality Flags HDF/HDF-EOS Workshop XIII

  14. h5dump and Region Reference Data • New flag –R to display data pointed to by region references • Improved display of subsets • An element of a dataset is specified by its indices • Available in HDF5 1.8.4, November 2009 HDF/HDF-EOS Workshop XIII

  15. h5dump and Quality Flags • New flag –M to display packed bit-field data • Available only in the HDF5 NPOESS branch • Will NOT be in HDF5 1.8.4 release • Source and pre-built binaries can be found at • ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/NPOESS/ HDF/HDF-EOS Workshop XIII

  16. HDFViewNew Features to Support References and Quality Flags HDF/HDF-EOS Workshop XIII

  17. HDFView and references • Aggregated granules: show the whole dataset pointed to by an object reference • Granules: show data pointed to by a dataset region reference HDF/HDF-EOS Workshop XIII

  18. Objects Pointed to by Object References HDF/HDF-EOS Workshop XIII

  19. Data Pointed to by Region References HDF/HDF-EOS Workshop XIII

  20. HDFView and quality flags • By default, integers with quality flags are displayed as 8-bit decimal values • Needed: a way to show values of specific quality flag bit fields HDF/HDF-EOS Workshop XIII

  21. Default display: 8-bit decimal integers HDF/HDF-EOS Workshop XIII

  22. Displaying quality flags in 8-bit integers • Use “Open As” on dataset with quality flags. • Apply bitmask to choose individual quality flags HDF/HDF-EOS Workshop XIII

  23. Examples of bit masks • Applied mask in red at top-right, e.g., “By bitmask {0, 1}” HDF/HDF-EOS Workshop XIII

  24. Quality Flags • Data can be displayed in decimal, binary, or hexadecimal decimal binary hexadecimal HDF/HDF-EOS Workshop XIII

  25. HDFView plug-in to Show Quality Flags • Implemented using “TreeView” plug-in • Open HDFView • Select “User Options” • Select NPOESS plug-in • Restart HDFView • Available with HDFView 2.5 patch3 HDF/HDF-EOS Workshop XIII

  26. Possible Future Work HDF/HDF-EOS Workshop XIII

  27. Possible future work • Add new features • Enhance current features • System support • User support HDF/HDF-EOS Workshop XIII

  28. Maintenance • Maintenance NPOESS/NPP features • High-Level Library • H5dump, other tools? • HDFView • Testing, documentation, releases HDF/HDF-EOS Workshop XIII

  29. New features • Support Variable-Length datasets • Ease access to information about quality flags • Use definition of quality flags, which are in: • The Common Data Format Control Book • XML files as part of the CDFCB-X delivery • Apply the structure of the XML to determine whether a field is a packed quality flag byte • Tell us what you need! HDF/HDF-EOS Workshop XIII

  30. Enhance current features • Extend reference display options • E.g., show properties of the referenced data • Revise default TreeView and TableView to improve NPOESS plug-in support • Improve quality of testing, documentation, and binary and source distribution • Tell us what you need! HDF/HDF-EOS Workshop XIII

  31. NPOESS/NPP System Support • Daily testing on systems critical to NPOESS/NPP • Linux 32 and 64-bit • Windows • AIX 5.3 • Solaris 32 and 64-bit • Regular testing of NPOESS high-Level library • Linux 32 and 64-bit • Windows • Solaris 32 and 64-bit • Porting to requested compilers • Performance tuning for applications • Improved support for C++ • Support for netCDF-4 access to NPOESS SDR and EDR products HDF/HDF-EOS Workshop XIII

  32. NPOESS/NPP User Support • Priority support to NPOESS/NPP users • Send email to help@hdfgroup.org • Add NPOESS to the subject field • Example: Subject: [NPOESS] Where is the plug-in? • Response in less than 4 hours • Continue gather requirements and enhance HDF5 software as needed • We want to hear from you! HDF/HDF-EOS Workshop XIII

  33. Special Research Projects • Improvements to HDF5 or HDF5 NPOESS products with the data management portion of the Raytheon Algorithm Development Library (ADL) • Implement OpeNDAP server for HDF5 NPOESS SDR and EDR products • Develop NPOESS specialized version of the HDF5 tool h5check • Performance tuning of HDF-NPOESS product files HDF/HDF-EOS Workshop XIII

  34. Questions? HDF/HDF-EOS Workshop XIII

  35. Questions/comments? HDF/HDF-EOS Workshop XIII

  36. Acknowledgements This work was supported by cooperative agreement number NA08AANEG0128from the National Aeronautics and Space Administration (NASA). Any opinions, findings, conclusions, or recommendations expressed in this material are those of the author[s] and do not necessarily reflect the views of the National Aeronautics and Space Administration. HDF/HDF-EOS Workshop XIII

More Related