1 / 29

Introduction - GHRSST data management

Distribution of Medspiration products from JPL Global Data Assembly Center (GDAC) GHRSST Medspiration and GlobCOLOUR Users Meeting 4-6 Dec 2006 Villefranche sur mer, France. Ed Armstrong, Jorge Vasquez, Andrew Bingham, Sue Heinz, Alex Cervantes, Qui Chau, Tim McKnight, Wendy Zhang

thais
Download Presentation

Introduction - GHRSST data management

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Distribution of Medspiration products from JPL Global Data Assembly Center (GDAC)GHRSST Medspiration and GlobCOLOUR Users Meeting 4-6 Dec 2006Villefranche sur mer, France Ed Armstrong, Jorge Vasquez, Andrew Bingham, Sue Heinz, Alex Cervantes, Qui Chau, Tim McKnight, Wendy Zhang edward.m.armstrong@jpl.nasa.gov

  2. Introduction - GHRSST data management • GHRSST international and regional task (and cost) sharing model: Built on 3 core components • 1) Regional centers of SST expertise produce near realtime data (L2P/L4) for their sensors • 2) Distribution and coordination from regional centers requires “Clearinghouse” for data and metadata management • Fulfilled by the JPL Global Data Assembly Center (GDAC) • Built on the 13 year expertise of the NASA Physical Oceanography DAAC in satellite data management • 3) Archiving and reanalysis occurs the Longterm Stewardship and Reanalysis Center (NOAA NODC) • The GDAC serves as the link between data users and producers, and between producers and longterm data stewardship (between 1 and 3)

  3. GDAC System Architecture

  4. Overview of GDAC tasks • Ingest realtime project data files • L2P, L4, L2P_GRIDDED data and metadata files from RDACs • Stage on public FTP site for 45 days • Deliver to long-term archive (LTSRF; NOAA NODC) after 30 days • Provide a variety data access methods (FTP, OPenDAP, POET) • Ingest and quality assure metadata into the Master Metadata Repository. Produce FGDC metadata products for LTSRF • Provide web-base interface for querying database and data discovery • Ancillary data production for unfilled L2P files • Provide USO and applications development • MODIS L2P (collaboration with the NASA OBPG)

  5. Support and value added services • FGDC metadata • Creating and staging FGDC metadata for LTSRF interface • OPenDAP • L2/L4 accessible via JPL server http://dods.jpl.nasa.gov • IDL and C L2P software readers • Read entire L2P structure, can be used to test integrity of L2P • RDAC assistance with L2P/L4/HRDDS file format and metadata anomalies • L2P subsetting design in progress • Value add L2P with ancillary filling • Wind , sea ice, solar insolation, aerosol ( from ECMWF, FNMOC etc.)

  6. Filling for solar insolation ECMWF Solar insolation Same day AVHRR L2P filled with solar insolation data

  7. Services continued • Web portal to GHRSST activities and data: http://ghrsst.jpl.nasa.gov • Webex site for intraproject collaboration on applications and user information • Application User Services (AUS) • Supporting applications development (S. Heinz) • Actively addressing user inquiries

  8. Data products and supported sensors 27 unique L2 and L4 products AVHRR AATSR MODIS AMSRE TMI SEVIRI GOES L4 blends

  9. MODIS L2P Flags SST Bias STD

  10. MODIS L2P granule with OC SST K490 Chlorophyll A

  11. GOES L2P SST AOD SSI Wind

  12. Data Ingest Statistics -2006

  13. Data Distribution Statistics -2006 Users Vol(GB) #Files EUR JPL NAVO REMSS

  14. Data uses and impact studies • Data Use and Impact Studies from the Multisensor Improved Sea Surface Temperature Project (MISST) • NAVOCEANO: impact of assimilation of high resolution L2P including AATSR and SEVIRI on NCODA (NRL Coupled Ocean Data Assimilation system) and K10 merged SST product

  15. NOGAPS TC track • Although there was no significant difference in the track forecast errors overall, there were areas where the use of the MISST SST analyses resulted in significantly improved NOGAPS TC track forecasts

  16. Data Use Cases and impact studies • Additional Data Use and Impact Studies • NOAA-RSS: Hurricane intensity impact used blended microwave L4 • UK Met: L2P for OSTIA L4 merged products • RSS: L4 blend using AMSRE, TMI, MODIS • NOAA: improved Reynolds OI using TMI and AATSR L2P • FNMOC: evaluate AATSR and SEVIRI in operational SST blend • Canadian Meteorological Forecast Office: AVHRR L2P

  17. MISST research • MISST research • Aerosols • AATSR used to study impact of aerosols on infrared measurements as compared to AVHRR and MODIS • Crucial for assigning flag or weight on IR measurements in SST blends • Diurnal warming models for deriving foundation temperature • SEVIRI measurements have excellent calibration and resolution necessary to resolve the daily diurnal cycle of SST • Skin layer temperature • Used to derive CO2 flux maps (B. Ward) • AATSR has most accurate skin layer measurement available

  18. Data Use Cases continued • US regional users for climate, weather and ecosystem studies at federal or state agencies, non profit organizations, universities coordinated by the GDAC. • Shorterm prediction and research center (SPORT) • SPORT supports infusion of NASA observations, data assimilation and modeling research into NWS forecast operations and decision making at the regional and local level. • Using the OSTIA product for modeling and forecasting activities for the US Gulf and North Atlantic regions. • SPORT is generating daily images available at: http://weather.msfc.nasa.gov/sport/sstAnimation/

  19. Data Use Cases • Pacific Disaster Center • The PDC has pledged their support to prepare and disseminate gridded GHRSST data via the APNHVA. • Atlas is published both as Asia Pacific basin-wide and Hawaiian Island region. • Support disaster community applications for preparedness, mitigation, response and recovery activities. • User include Hawaii State Civil Defense, FEMA, NOAA, UN agencies, Thailand’s National Disaster Warning Center and Vietnam’s Disaster Management Committee. • http://www.pdc.org

  20. Data Use Cases continued • US regional users for climate, weather and ecosystem studies at federal or state agencies, non profit organizations, universities. • Shorterm prediction and research center (SPORT) • Pacific Disaster Center • Asia Pacific Natural Hazards Information Network • NASA Aquarius Salinity Mission • Gulf of Maine Ocean Observing System (GOMOOS) • Southern California Coastal Ocean Observing System (SCCOOS) • Ecosystem Management • California State University Humbolt Bay • NOAA Protected Species Management • Southeast Phytoplankton Monitoring Network (SEPMN) • ECCO assimilation

  21. GHRSST metadata model • Follows the NASA Directory Interchange Format (DIF) • Granule records ( File Record [FR] ) • Dataset record ( Dataset Description [DSD] ) • XML based • L2P/L4 products only accepted with valid metadata • Metadata ingested into a “clearinghouse”: the Master Metadata Repository (MMR) • Support for FGDC metadata standard implemented for distribution and archiving at the LTSRF • Future support • ISO19115

  22. Master Metadata Repository (MMR) • MMR • mySQL database (ver 4x) • Fully relational, transactions implemented • Separate multi-table relational databases for DSD and FR records linked with one common field • DSD metadata records jointly maintained by Met Office and GDAC • Over 150K FR records ingested without error • QA checkers/XML parsers for FR and DSD records implemented • Web search interface • Supports output to FGDC format through XML stylesheet conversions

  23. MMR Search Interface

  24. Interface to the LTSRF • FGDC records created for 30 day (and older data). • 30 day old data and metadata extracted from an operational FTP site by NOAA • GHRSST data products expired from GDAC (PO.DAAC) FTP site after 45-60 days (“rolling store”) • Size of the rolling store can easily be adjusted to accommodate changing disk space resources • To date 100% file integrity between GDAC and LTSRF

  25. DIF to FGDC conversion Extract FR from MMR Format to XML Extract and format DSD to XML Apply XSL FGDC formatted record

  26. GDAC future activities • Integrated into the PO.DAAC in 2008 • ISO19115 support • Routine Ancillary L2P filling • Delivery of L4 and L2P products via the POET visualizer and subsetter • Google Earth browse capability • Additional L2P, L2P_GRIDDED, and L4 products • NetCDF 4 support • Ingestion of metadata into ECHO

  27. GDAC team • Jorge Vazquez - Project manager • Ed Armstrong - GDAC Cognizant Engineer • Andy Bingham - Data management and Oceanids developer • Sue Heinz - Applications and User Services lead (sheinz@geologics.com) • Alex Cervantes - Software engineer for Oceanids development and L2P ancillary fill • Qui Chau - Software engineer for L2P ancillary fill, MODIS L2P, L2 subsetting development • Wendy Zhang - Software engineer on MMR • Tim McKnight - Operations engineer (txm@seastar.jpl.nasa.gov) • Pat Liggett - PO.DAAC team and management

  28. Backup - Data Access • ftp://podaac.jpl.nasa.gov/GHRSST • Organized by: datatype / sensor / RDAC / year / DOY • OPenDAP: • dods.jpl.nasa.gov • User support: • Email ghrsst@podaac.jpl.nasa.gov

  29. OCEANIDS OVERVIEW

More Related