1 / 17

Status of the GHRSST GDAC (Global Data Assembly Center) at the JPL PO.DAAC

Status of the GHRSST GDAC (Global Data Assembly Center) at the JPL PO.DAAC. Ed Armstrong, Jorge Vazquez and Andrew Bingham http://ghrsst.jpl.nasa.gov. Overview of GDAC Tasks. . . . .data management and distribution center for all GHRSST products. Ingest project data files

jam
Download Presentation

Status of the GHRSST GDAC (Global Data Assembly Center) at the JPL PO.DAAC

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Status of the GHRSST GDAC (Global Data Assembly Center) at the JPL PO.DAAC Ed Armstrong, Jorge Vazquez and Andrew Bingham http://ghrsst.jpl.nasa.gov

  2. Overview of GDAC Tasks . . . .data management and distribution center for all GHRSST products • Ingest project data files • L2P, L4, HR-DDS, MMR and MDB files from RDACs • Stage on public FTP site for 40 days • Deliver data and metadata to LTSRF (NODC) for 30 day old data • Maintain metadata in a relational database (MMR) • Provide web-base interface for data discovery and metadata query • Fill ancillary fields for all L2P products • Support GHRSST User Services and Applications • Maintain a match-up database of in situ/satellite obs • RDAC: MODIS L2P production US GHRSST meeting, RSMAS

  3. Recent Activities • Public GHRSST data release in August 2005 • Interacted with OBPG and RSMAS to design the MODIS L2P processing architecture • New Linux-based web and database severs, and development workstation integrated into PO.DAAC • GDAC web pages enhanced • Access to MMR improved • Input for GDS rev 1.7 including metadata revision • IDL and C readers for L2P • Implemented FGDC metadata and transfer to LTSRF • Ingested NWP and model data streams for L2P ancillary fields • Developing code and methodology for realtime ancillary fillingImplemented user support through a customer management tool (RightNow) • First draft of a user requirements document US GHRSST meeting, RSMAS

  4. GDAC System Architecture RDACs Medspiration (Europe) NGSST (Japan) BlueLink (Australia) NAVOCEANO (U.S.) RSS (U.S.) PO.DAAC (U.S.) GDAC (JPL U.S.) OCEANIDS (Data Management) MMR/MDB NRT Users Data Tools L4PC (MIST – U.S.) Archive (NODC – U.S.) Users US GHRSST meeting, RSMAS

  5. GDAC Web Page US GHRSST meeting, RSMAS

  6. Master Metadata Repository US GHRSST meeting, RSMAS

  7. Data Access • FTP: podaac.jpl.nasa.gov/pub/GHRSST • Data public since late August 2005 • Organised by: datatype / sensor / RDAC / year / DOY • Search interface through the MMR • Future delivery mechanisms • POET (L2 subsetting) • OpenDAP • Live Access Server • Subscription • Data casting US GHRSST meeting, RSMAS

  8. Data Ingest US GHRSST meeting, RSMAS

  9. Data Latencies US GHRSST meeting, RSMAS

  10. Oceanids ingest statistics Granules Volume (GB) US GHRSST meeting, RSMAS

  11. Ancillary fields • JPL GDAC will fill the L2P ancillary fields of wind, solar insolation, sea ice and aerosols • Some RDACs cannot fill these fields in an operational environment • GDAC has accepted this additional responsibility at last GHRSST ST meeting • Ingest numerical weather prediction and model data, i.e., NCEP, NAAPS, FNMOC sea ice and SST • A backup data stream will be implemented for each ancillary field (e.g., ECMWF model output) • Ancillary filling methodology and management has been conceptualized. • Readers for input fields being built US GHRSST meeting, RSMAS

  12. NOAA LTSRF Interface Status • After 30 days data are placed on a rolling stage for NOAA to pull • FGDC metadata created “on the fly” through access to the MMR • Formatted for the NOAA accession record (daily collection of same product files) • Currently NOAA has implemented an automated retrieval of L2P data and FGDC metadata • FGDC records for all L2P Medspiration data created through Nov 2005 • Other products to follow US GHRSST meeting, RSMAS

  13. MMR Status • MMR (Master Metadata Repository) • Fully transactional MySQL relational database • Schema finalized and implemented for FR and DSD metadata in Jan 2005 • New database server implemented in Jun 2005 • Over 80,000 DSD and FR records ingested since Feb 2005 • No operational failures • Web search interface improved, access is very fast. • Automated translation to FGDC metadata through XML style sheet conversion • Other standards can also be supported US GHRSST meeting, RSMAS

  14. User services and applications • Updated US GDAC User Requirements document • GHRSST requirements • GDAC system requirements • RDAC requirements • User requirements • E.g., OOSes US GHRSST meeting, RSMAS

  15. MODIS L2P • Joint effort between Univ. of Miami RSMAS, NASA Ocean Biology Processing Group (OBPG) and JPL • This is a new RDAC activity • Initially concentrate on Aqua SST • RDAC Division of Labor • RSMAS: Algorithm and error statistics • OBPG: 1 km L2 SST production with some ancillary fields • JPL GDAC: L2P integration with error statistics and ancillary fields • Current Status • OBPG has produced a realtime L2 SST stream for MODIS Aqua (and Terra) • RSMAS has developed a MODIS error “hypercube” • JPL has tested throughput and acquisition from OPBG server US GHRSST meeting, RSMAS

  16. Future activities . . . • Ancillary fields filling • Accept delivery of Linux 8 node cluster • Installed in Jan 2006 • Implement code for realtime filling • Investigating parallelization of processing • MODIS L2P • Finalize transfer between three parties • Implement error statistics and L2P • Schedule for Aqua and Terra SST L2P • Interface to new RDACs • NAVO • NOAA NESDIS • OBPG • Update ICD US GHRSST meeting, RSMAS

  17. . . . Future activities • FGDC metadata • Operational transfers for all GHRSST products (L2P, L4) to LTSRFs • User Services • GHRSST product support • Applications development • Data delivery • L2P subsetting in POET • OpenDAP • Improved search and delivery via MMR • New technology infusion • GDAC will be a test bed for the Earth Science Data Casting Project • a recently funded NASA ROSES activity for XML-based data management and delivery • Implement the Matchup Database (MDB) with similar functionality as MMR US GHRSST meeting, RSMAS

More Related