1 / 71

Kelvin K. Droegemeier University of Oklahoma Internet2 Spring Meeting 20 April 2004

The Collaborative Radar Acquisition Field Test (CRAFT): A Unique Public-Private Partnership in Mission-Critical Data Distribution. Kelvin K. Droegemeier University of Oklahoma Internet2 Spring Meeting 20 April 2004. NCDC. NEXRAD Doppler Radar Network. The Moore, Oklahoma Tornado

meena
Download Presentation

Kelvin K. Droegemeier University of Oklahoma Internet2 Spring Meeting 20 April 2004

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Collaborative Radar Acquisition Field Test (CRAFT): A Unique Public-Private Partnership in Mission-Critical Data Distribution Kelvin K. DroegemeierUniversity of OklahomaInternet2 Spring Meeting20 April 2004 NCDC

  2. NEXRAD Doppler Radar Network

  3. The Moore, Oklahoma Tornado of 3 May 1999

  4. Damage Paths Ground Truth - 53 Tornadoes Storm H Storm E Storm G Storm A Storm C Storm D Storm B 100 km range ring Courtesy D. Zittel

  5. Trimmed Detections and Ground Truth Damage Paths Hits (142) Misses (25) FAs (21) Courtesy D. Zittel

  6. Doppler Weather Radar: Measuring the Wind Speed USA Today

  7. A Unique Tool for Sampling the 3D Atmosphere in Detail • The radar scans a 3-D conical volume of the atmosphere every 5-6 minutes through a series of 360 degree sweeps…

  8. A Unique Tool for Sampling the 3D Atmosphere in Detail • … at several elevation angles

  9. NEXRAD Facts and Figures • 159 radars (142 in the Continental US) • 121 National Weather Service radars • 26 Department of Defense radars • 12 Federal Aviation Administration radars

  10. NEXRAD Data Types • Archive Level I • analog, time domain output • Level II data • digital base data at full precision and all elevation angles • Archive Level III • digital products + lowest four elevation angles • reduced precision: data are binned • Archive Level IV

  11. NEXRAD Data Types • Archive Level I • analog, time domain output • Level II data • digital base data at full precision and all elevation angles • Archive Level III • digital products + lowest four elevation angles • reduced precision: data are binned • Archive Level IV

  12. The Value – Hazard Detection

  13. The Value – Real Time 3D Analyses http://caps.ou.edu

  14. The Value – Fundamental Predictability Carbone et al. (2002)

  15. The Value – Storm Climatologies

  16. The Value – Computer Prediction Models

  17. 7 pm 8 pm 6 pm Radar Hourly Radar Observations (Fort Worth Shown by the Pink Star)

  18. 7 pm 8 pm 6 pm Radar 4 hr 3 hr 2 hr Computer Forecast

  19. 7 pm 8 pm 6 pm Radar 4 hr 3 hr 2 hr Fcst w/o Radar

  20. Space Shuttle Columbia Debris Trail

  21. Bird Migration Studies

  22. NEXRAD data are *THE* mostvaluable data for mesoscale meteorology… but they were not available in real time to the academic and corporate communities

  23. The Collaborative Radar Acquisition Field Test (CRAFT) • Begun in fall, 1998 as a collaborative R&D project to demonstrate the real time compression and Internet-based transmission of WSR-88D base (Level II) data • Philosophy: Leverage existing infrastructures (Internet), public-domain software (Unidata LDM), and secure, low-cost services (56K phone lines) • Short-term goals: Storm-scale research and improvement of NCDC archive capability • Long-term goal: National real-time capability for operations, research, and private industry

  24. Coordinating Organizations • Initiated and led by the OU Center for Analysis and Prediction of Storms (CAPS) • National Severe Storms Laboratory (NSSL) • NOAA WSR-88D Radar Operations Center (ROC) • UCAR Unidata Program • National Climatic Data Center (NCDC) • University of WashingtonTREMENDOUS support from the US WeatherResearch Program

  25. Foundation #1: NSSL’s RIDDS Courtesy NSSL

  26. Foundation #2: Unidata Local Data Manager (LDM) Courtesy D. Fulker, Unidata

  27. Foundation #3: Data Compression – BZIP2 The Internet Adaptation to LDM by Harry Edmon, U. of Washington

  28. Foundation #4: The Internet NGI Internet2 Abilene TheInternet Commodity Invention courtesy A. Gore

  29. Foundation #5 • Incredible cooperation, collegiality, and agency support NCDC

  30. Original Technical Strategy At the radar site WSR-88D RIDDS

  31. Original Technical Strategy Linux PC + LDM + Compression ($1000) At the radar site WSR-88D RIDDS

  32. Original Technical Strategy Router ($2000) Linux PC + LDM + Compression ($1000) At the radar site WSR-88D RIDDS

  33. Original Technical Strategy Router ($2000) Linux PC + LDM + Compression ($1000) At the radar site Dedicated 56K line, T1, ($2000 - $6000/year) or NWS LAN LDM Server (OU) WSR-88D RIDDS

  34. Original Technical Strategy Router ($2000) Linux PC + LDM + Compression ($1000) At the radar site Dedicated 56K line, T1, ($2000 - $6000/year) or NWS LAN LDM Server (OU) WSR-88D Internet or Abilene RIDDS Users (LDM)

  35. Slow But Sure Growth February, 2000

  36. Technical Highlights • BZIP2 off-the-shelf compression algorithm • maximum 16:1 in clear air; 8:1 in the worst case; average of about 12:1 • Maximum aggregate data rate is 15 Mbytes/radar/5 min uncompressed = 48 Mbits/sec compressed for entire national network • bandwidth isn’t the main issue, but rather topology, quality of service, and data storage • 56K lines found to be inadequate in some cases (squall lines) – maybe 10% of time • Exceed available bandwidth by 10-20% • Can create latencies of 10s of minutes (more later) • Up to 26% improvement in compression possible in BZIP2, but requires changing code at user end • A pre-processing algorithm (NSSL) provides 10% improvement – no changes in BZIP required

  37. Humble Beginnings… January, 1999

  38. Eight More in 1 Year January, 2001 14 Radars

  39. Latency Data

  40. The Idea Catches Hold January, 2002

  41. The NWS Makes a Bold Decision Fall 2004

  42. Regional Collection Concept Proposed 5 Yrs Ago NCEP NCDC

  43. Reiterated Fall 2002

  44. Current Status • NWS is implementing a data delivery system based upon the CRAFT concept (regional hubs, Abilene as the backbone) • This infrastructure could become THE NWS data and model grid delivery system of the future

  45. We’ve Come a Long Way! Today – 123 Radars!!

  46. Private Sector Joins the Effort • Initiated after 2001 Stakeholders Workshop • NWS agreed to let OU provide real time Level II data to private companies and others (academia), as R&D partners, in exchange for latency and other information to help evaluate system performance • Any group could participate • Provided an opportunity for private sector partners to gain experience using Level II data and develop new capabilities with them • Had 6 private sector partners • Companies agreed to the collaborative nature of the project, i.e., everyone wins when everyone contributes

  47. The Benefits – Industry Development Baron Services, Inc.

  48. A True Public-Private Partnership Recent NOAA + Pvt Sector Money  $3.0M

More Related