1 / 8

Integrating HDF5 with SRB

Integrating HDF5 with SRB. Project Report at CIP AG Meeting. March 17, 2006. Sponsored by NLADR, NFS PACI Project in Support of NCSA-SDSC Collaboration. Outline. What is current status of the project? What are the plans moving forward?

eaton-henry
Download Presentation

Integrating HDF5 with SRB

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Integrating HDF5 with SRB Project Report at CIP AG Meeting March 17, 2006 Sponsored by NLADR, NFS PACI Project in Support of NCSA-SDSC Collaboration CIP Status Meeting

  2. Outline • What is current status of the project? • What are the plans moving forward? • What can be achieved for Users by the end of FY06 and by December 06? • Future work CIP Status Meeting

  3. Current Status • Released and tested a working prototype • Released source code and documentation with SRB version 3.4.0 • Released HDFView 2.3 with SRB support (Windows and Linux only) • Tested with terabyte-sized Southern California Earthquake Center data sets • Tested performance for HDF5 image files in SRB (technical report near completion) • Demonstrated remote sub-setting access with HDFView Introduction to HDF5 CIP Status Meeting

  4. Current Status (continue) • Reported and published the work • Four workshop presentations • NCSA technical seminar . We presented the project design in a technical seminar at NCSA on February 24, 2005. People from different divisions of NCSA attended the seminar and we have received a lot of feedback. • Cyberinfrastructure seminar series. This work was reported in the Cyberinfrastructure Seminar Series on the Access Grid on May 17, 2005. • HDF and HDF-EOS Workshop IX. We presented the work at the HDF and HDF-EOS workshop IX on December 1, 2005 at San Francisco. • SRB Worksop. We presented the work at the SRB Workshop on February 2, 2006 at San Diego • Two publications • Published the project report and HDF-SRB user’s guide at http://hdf.ncsa.uiuc.edu/hdf-srb-html/ • Published a paper at SRB Workshop proceedings http://www.sdsc.edu/srb/Workshop/ CIP Status Meeting

  5. Current Status (continue) • Revised the task list for half-FTE (proposed for one-FTE but funded for half-FTE) • Worked on un-planned tasks (important but not proposed activities) • Began investigation of HDF5 Access Information Package (AIP) API • Began working with NARA to install HDF-SRB in NARA test-bed and demo lab. Email quoted from Mark Conrad • Worked with Federico Bassetti on setting up NCSA SRB server for grid test on HDF-SRB performance CIP Status Meeting

  6. Plans Moving Forward • 04/30/06 -- Complete prototype implementation of HDF5 AIP (NCSA and SDSC) • 05/31/06  -- Write test suite and fix bugs (NCSA and SDSC) • 06/30/06  -- Improve performance for transferring raw data (NCSA and SDSC) • 07/31/06-- Add more features to HDFView for SRB support (NCSA)                -- integrate the software into the SRB configuration and distribution (SDSC) • 09/30/06  -- Support primitive writing capabilities for HDF5 files in SRB (NCSA and SDSC) • 11/30/06   -- Write documentation and extensive testing (NCSA and SDSC) • 12/30/06  -- Release the production system and publish documentation CIP Status Meeting

  7. Deliverables • By the end of FY06 • A prototype implementation of HDF5 AIP can be finished • Primitive writing capability can be added to the HDF-SRB system • The system can be tested extensively and improved • By December 06 • A production system can be released CIP Status Meeting

  8. Future Work Depend on Funding • Support extensive writing capabilities on the HDF-SRB system • Support Grid-enabled OPeNDAP • Deploy the HDF-SRB system to Tera-Grid system • Support for complex datatypes such as variable-length datatypes • Support for efficient file access across multiple servers • Implement a SRB driver for the HDF5 library that uses SRB logical file location • Study further whether the current HDF-SRB system can performance effectively with SCEC • Write HDF5 ingest for HDF5 AIP in HDF-SRB system CIP Status Meeting

More Related