1 / 14

TIGGE Data Archive at NCAR

TIGGE Data Archive at NCAR. 8th GIFS-TIGGE Working Group World Meteorological Organization Geneva 22-24 February, 2010 Doug Schuster Steven Worley Dave Stepaniak Others. Archive Summary. Online Data Period, most recent two weeks ~ 5 TB , public products

bran
Download Presentation

TIGGE Data Archive at NCAR

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. TIGGE Data Archive at NCAR 8th GIFS-TIGGE Working Group World Meteorological Organization Geneva 22-24 February, 2010 Doug Schuster Steven Worley Dave Stepaniak Others

  2. Archive Summary • Online Data • Period, most recent two weeks • ~ 5 TB , public products • ~ 1 TB, data preparation, subsetting • ~ 65 GB Tropical Cyclone Track Data (CXML) • Offline Data • Full period of record • ~ 300 TB, NCAR MSS system

  3. Provider Summary • Archive Status • Service Improvements • Special Projects • Metrics • Future Service Additions

  4. Data Delivery (02/2010) • Data exchanges using UNIDATA LDM, HTTP and FTP • 440 GB/day (ECMWF = 332 GB/day) • 1.7+ million 2-D gridded fields/day

  5. Archive Inhomogeneities

  6. Parameter vs Provider Matrix PL = Pressure Level, PT = 320K θ Level, PV = ± 2 Potential Vorticity Level, SL = Single/Surface Level

  7. Parameter vs Provider Matrix PL = Pressure Level, PT = 320K θ Level, PV = ± 2 Potential Vorticity Level, SL = Single/Surface Level

  8. 2009 Service Improvements • Improved Web Portal Performance • User data selection interface runs smoother. • Automated capability to stage offline data for download. • Added “Client for URLs” (cURL) based parameter subsetting capability. • Directly extract selected 2D grids from files across multiple data providers. • Downloads managed from user’s workstation. • Script templates are provided to drive “subscription style” parameter download capability. • Strengthened data ingest processing. • System monitoring (disk capacity, LDM status, etc) • Failover mechanisms for LDM downtime. • Data integrity checking.

  9. 2009/2010 Supported Special Projects • NCAR Research Applications Laboratory (RAL) Google African Meningitis Forecast Initiative • Dr. Thomas Hopson • Integrate weather and health data to manage African meningitis outbreaks (in Ghana as starting point). • Looking for 2-week lead time to position vaccines. • Processing ECMWF data from present backwards through the archive for model calibration. • Total Precipitation, 2m Temperature, 2m Dew Point. • Global Subarea 40 deg N to 40 Deg S. • 0.5 x 0.5 degree resolution. • Assist RAL and Unidata in providing “Real-time” access to TIGGE based Multi-Model Ensemble Forecast data. • http://www.ral.ucar.edu/projects/ghana/ • http://www.google.org/predict.html

  10. NCAR TIGGE Full Model Data Archive Usage cURL impact

  11. NCAR TIGGE CXML Archive Usage

  12. Planned Service Improvements • Additional Online Storage • 45 TB data online (3 months of most recent data). • 15 TB for data processing request output. • Interface driven data processing and immediate download capability extended from 2 weeks to 3 months. • Improved “Offline” Storage. • Data being transitioned from current NCAR MSS system to new HPSS system. • Improved access speeds to offline data. • Add data processing capability to offline data? • Expanded/Updated Computing. • Improved LDM ingest reliability. • Faster “turn-around” time on user sub-set requests. • Potential for subscription services?

  13. Potential Service Improvements • Model Validation Data Portal (Pending Funding)? • Wide variety of observational and analyzed data sources. • In Situ Observation data (NCEP operationally collected data). • Ocean based observations (ICOADS). • Global tropical cyclone position and intensity data. • Atmospheric Reanalysis (JRA, ERA, NNRR, CFSR). • Global PCP datasets. • Suggested TWG sets to support FDP’s. • Will need discussions to set priorities for development. • Data sub-set and output format selection capability. • Integrated With TIGGE model data portal • Data analysis tools and codes. • Model Evaluation Tools offered by the Developmental Testbed Center at NCAR • http://www.dtcenter.org/mrt/users

  14. http://tigge.ucar.edu/

More Related