1 / 50

Development and Calibration of Ensemble Based Hazardous Weather Products at the Storm Prediction Center

Development and Calibration of Ensemble Based Hazardous Weather Products at the Storm Prediction Center. David Bright Gregg Grosshans, Jack Kain, Jason Levit, Russ Schneider, Dave Stensrud, Matt Wandishin, Steve Weiss October 11, 2005 NCEP Predictability Discussion Group.

madeleine
Download Presentation

Development and Calibration of Ensemble Based Hazardous Weather Products at the Storm Prediction Center

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Development and Calibration of Ensemble Based Hazardous Weather Products at the Storm Prediction Center David Bright Gregg Grosshans, Jack Kain, Jason Levit, Russ Schneider, Dave Stensrud, Matt Wandishin, Steve Weiss October 11, 2005 NCEP Predictability Discussion Group Where Americas Climate and Weather Services Begin

  2. STORM PREDICTION CENTER MISSION STATEMENT The Storm Prediction Center (SPC) exists solely to protect life and property of the American people through the issuance of timely, accurate watch and forecast products dealing with hazardous mesoscale weather phenomena. MISSION STATEMENT The Storm Prediction Center (SPC) exists solely to protect life and property of the American people through the issuance of timely, accurate watch and forecast products dealing with tornadoes, wildfires and other hazardous mesoscale weather phenomena.

  3. STORM PREDICTION CENTER HAZARDOUS PHENOMENA • Hail, Wind, Tornadoes • Excessive rainfall • Fire weather • Winter weather

  4. SPC Forecast Products • TORNADO & SEVERE THUNDERSTORM WATCHES • WATCH STATUS MESSAGE • CONVECTIVE OUTLOOK • Day 1; Day 2; Day 3; Days 4-8 • MESOSCALE DISCUSSION • Severe Thunderstorm Potential/Outlook Upgrade • Thunderstorms not expected to become severe • Hazardous Winter Weather • Heavy Rainfall • FIRE WEATHER OUTLOOK • Day 1; Day 2; Days 3-8 • OPERATIONAL FORECASTS ARE BOTH DETERMINISTIC AND PROBABILISTIC 75% of all SPC products are valid for < 24h period

  5. EXPERIMENTAL WATCH PROBABILITIES Severe Thunderstorm Watch 688 Probability Table

  6. CONVECTIVE OUTLOOKSOperational through Day 3

  7. 12h Periods (> 10%; 40%; 70%) Thunderstorm Outlooks: 12h Enhanced Thunderstorm (Today) 24h General Thunderstorm 24h Period (> 10%) 12h Enhanced Thunderstorm (Tonight)

  8. Product Guidance at the SPC • Operational emphasis on… • Observational data • Short-term, high-resolution NWP guidance • Specific information predicting hazardous mesoscale phenomena • NWP needs range from the very-short range to medium range • Very short-range: Hourly RUC; 4.5 km WRF-NMM • Short-range: NAM, GFS, SREF • Medium-range: GFS, ECMWF, MREF • Today’s focus: SREF • Overview of the ensemble product suite • Specific ensemble calibrated guidance

  9. Objective: Provide a wide range of ensemble guidance covering all of the SPC program areas Overview of Ensemble Guidance

  10. Sample of Ensemble Products Available… MEAN & SD: 500 mb HGHT SPAGHETTI: SFC LOW MEAN: MUCAPE, 0-6 SHR, 0-3 HLCY MEAN: PMSL, DZ, 10M WIND http://www.spc.noaa.gov/exper/sref/

  11. Sample of Ensemble Products Available… STP = F (mlCAPE, mlLCL, SRH, Shear) Thompson et al. (2003) Omega < -3 -11 < T < -17 RH > 80% PROB: DENDRITIC GROWTH PROB: SIG TOR PARAM > 3 STP = F (mlCAPE, mlLCL, SRH, Shear) Thompson et al. (2003) MEDIAN, UNION, INTERSECTION: SIG TOR PARAM MAX OR MIN: MAX FOSBERG INDEX http://www.spc.noaa.gov/exper/sref/

  12. F63 SREF POSTAGE STAMP VIEW: PMSL, HURRICANE FRANCES Red = EtaBMJ Yellow= EtaKF Blue = RSM White = OpEta SREF Member

  13. Combined Probability CAPE (J/kg) Green solid= Percent Members >= 1000 J/kg; Shading >= 50% Gold dashed = Ensemble mean (1000 J/kg) F036: Valid 21 UTC 28 May 2003 • Probability surface CAPE >= 1000 J/kg • Relatively low • Ensemble mean is < 1000 J/kg (no gold dashed line)

  14. Combined Probability 10 m – 6 km Shear (kts) Green solid= Percent Members >= 30 kts; Shading >= 50% Gold dashed = Ensemble mean (30 kts) F036: Valid 21 UTC 28 May 2003 • Probability deep layer shear >= 30 kts • Strong mid level jet through Iowa

  15. Combined Probability 3 Hour Convective Precipitation >= 0.01 (in) Green solid= Percent Members >= 0.01 in;Shading >= 50% Gold dashed = Ensemble mean (0.01 in) F036: Valid 21 UTC 28 May 2003 • Convection is likely WI/IL/IN • Will the convection become severe?

  16. A quick way to determine juxtaposition of key parameters Fosters an ingredients-based approach Not a “true” probability Dependence Different members contribute Combined Probability Prob Cape >= 1000 X Prob Shear >= 30 kts X Prob Conv Pcpn >= .01” F036: Valid 21 UTC 28 May 2003

  17. Combined Probability Prob Cape >= 1000 X Prob Shear >= 30 kts X Prob Conv Pcpn >= .01” F036: Valid 21 UTC 28 May 2003 • A quick way to determine juxtaposition of key parameters • Fosters an ingredients-based approach • Not a “true” probability • Dependence • Different members contribute Severe Reports Red=Tor; Blue=Wind; Green=Hail

  18. Combined Probability Ingredients for extreme fire weather conditions over the Great Basin F15 SREF PROBABILITY TPCP x RH x WIND x TMPF (< .01” x < 10% x > 30 mph x > 60 F)

  19. Objective: Develop calibrated probabilistic guidance for CG lightning Calibrated Thunderstorm Guidance

  20. Combine Lightning Ingredients into a Single Parameter • Three first-order ingredients (readily available from NWP models): • Lifting condensation level > -10o C • Sufficient CAPE in the 0o to -20oC layer • Equilibrium level temperature < -20o C • Cloud Physics Thunder Parameter (CPTP) CPTP = (-19oC – Tel)(CAPE-20 – K) K where K = 100 Jkg-1 and CAPE-20 is MUCAPE in the 0o C to -20o C layer

  21. Example CPTP: One Member 18h Eta Forecast Valid 03 UTC 4 June 2003 Plan view chart showing where grid point soundings support lightning (given a convective updraft)

  22. SREF Probability CPTP > 1 3 hr valid period: 21 UTC 31 Aug to 00 UTC 01 Sept 2004 15h Forecast Ending: 00 UTC 01 Sept 2004 Uncalibrated probability: Solid/Filled; Mean CPTP = 1 (Thick dashed)

  23. SREF Probability Precip > .01” 3 hr valid period: 21 UTC 31 Aug to 00 UTC 01 Sept 2004 15h Forecast Ending: 00 UTC 01 Sept 2004 Uncalibrated probability: Solid/Filled; Mean precip = 0.01” (Thick dashed)

  24. Joint Probability (Assume Independent) P(CPTP > 1) x P(Precip > .01”) 3 hr valid period: 21 UTC 31 Aug to 00 UTC 01 Sept 2004 15h Forecast Ending: 00 UTC 01 Sept 2004 Uncalibrated probability: Solid/Filled

  25. Uncalibrated Reliability (5 Aug to 5 Nov 2004) Frequency [0%, 5%, …, 100%] Perfect Forecast No Skill Climatology P(CPTP > 1) x P(P03I > .01”)

  26. Adjusting Probabilities • Calibrate ensemble thunderstorm guidance based on the observed frequency of occurrence

  27. Ensemble Thunder Calibration • Bin separately P(CPTP > 1) and P(P03M > 0.01”) into 11 bins (0-5%; 5-15%; …; 85-95%; 95-100%) • Combine the two binned probabilistic forecasts into one of 121 possible combinations (0%,0%); (0%,10%); … (100%,100%) • Use NLDN CG data over the previous 366 days to calculate the frequency of occurrence of CG strikes for each of the 121 binned combinations • Construct for each grid point using 1/r weighting • Bin ensemble forecasts as described in steps 1 and 2 and assign the observed CG frequency (step 3) as the calibrated probability of a CG strike • Calibration is performed for each forecast cycle (09 and 21 UTC) and each forecast hour; domain is entire U.S. on 40 km grid (CG strike within ~12 miles)

  28. Before Calibration

  29. Joint Probability (Assumed Independence) P(CPTP > 1) x P(Precip > .01”) 3 hr valid period: 21 UTC 31 Aug to 00 UTC 01 Sept 2004 15h Forecast Ending: 00 UTC 01 Sept 2004 Uncorrected probability: Solid/Filled

  30. After Calibration

  31. Calibrated Ensemble Thunder Probability 3 hr valid period: 21 UTC 31 Aug to 00 UTC 01 Sept 2004 15h Forecast Ending: 00 UTC 01 Sept 2004 Calibrated probability: Solid/Filled

  32. Calibrated Ensemble Thunder Probability 3 hr valid period: 21 UTC 31 Aug to 00 UTC 01 Sept 2004 15h Forecast Ending: 00 UTC 01 Sept 2004 Calibrated probability: Solid/Filled; NLDN CG Strikes (Yellow +)

  33. Calibrated Reliability (5 Aug to 5 Nov 2004) Frequency [0%, 5%, …, 100%] Perfect Forecast Perfect Forecast No Skill Climatology No Skill Calibrated Thunder Probability

  34. 3h probability of > 1 CG lightning strike within ~12 mi 09Z and 21Z SREF valid at F003 through F063 May 15 – Sept 15 2005 Reliability Economic Potential Value

  35. 12h probability of > 1 CG lightning strike within ~12 mi 09Z SREF valid at F012 through F063 May 15 – Sept 15 2005 Reliability Economic Potential Value

  36. Objective: Develop calibrated probabilistic guidance of the occurrence of severe convective weather(Available for 3h, 12h, and 24h periods; calibration not described today) Calibrated Severe Thunderstorm Guidance

  37. 24h probability of > 1 severe thunderstorm within ~25 mi SREF: 2005051109 Valid 12 UTC May 11, 2005 to 12 UTC May 12, 2005 SVR WX ACTIVITY 12Z 11 May to 12Z 12 May, 2005 a= Hail w=Wind t=Tornado

  38. 24h probability of > 1 severe thunderstorm within ~25 mi 21Z SREF valid at F039 through F039 (i.e., Day 1 Outlook) May 15 – Sept 15 2005 Reliability Economic Potential Value Hail > .75” Wind > 50 kts Tornado

  39. Objective: Develop calibrated probabilistic guidance of snow accumulation on road surfaces Experimental Calibrated Snow Accumulation Guidance

  40. Ensemble Snow Calibration • Use frequency of occurrence technique -- similar to the calibrated probability of CG lightning • Produce 8 calibrated joint probability tables • Take power mean (RMS average) of all 8 tables for the 3h probability of snow accumulating on roads in the grid cell • Calibration period is Oct. 1, 2004 through Apr. 30, 2005 • MADIS “road-state” sensor information is truth (SREF is interpolated to MADIS road sensor)

  41. Goal: Examine the parameter space around the lower PBL T, ground T, and precip type and calibrate using road sensor data. • SREF probability predictors • Two precipitation-type algorithms • Baldwin algorithm in NCEP post. (Pr[Sn, ZR, IP]) • Czys algorithm applied in SPC SREF post-processing. (Pr[Sn, ZR, IP]) (2) Two parameters sensitive to lower tropospheric and ground temperature • Snowmelt parameterization (RSAE)– Evaluates fluxes to determine if 3” of snow melts over a 3h period. If yes, then parameter is assigned: 273.15 – TG. (Pr[>1; >2; >4]) • Simple algorithm (RSAP) F (Tpbl, TG, Qsfcnet rad. flux,) where values > 1 indicate surface cold enough for snow to accumulate. (Pr[>1])

  42. Frequency Calibration Tables LAYERSREF INGREDIENT 1SREF INGREDIENT 2 1 Prob(RSAE > 1) Prob(Baldwin Snow, ZR, or IP) 2 Prob(RSAE > 2) Prob(Baldwin Snow, ZR, or IP) 3 Prob(RSAE > 4) Prob(Baldwin Snow, ZR, or IP) 4 Prob(RSAE > 1) Prob(Czys Snow, ZR, or IP) 5 Prob(RSAE > 2) Prob(Czys Snow, ZR, or IP) 6 Prob(RSAE > 4) Prob(Czys Snow, ZR, or IP) 7 Prob(RSAP > 1) Prob(Baldwin Snow, ZR, or IP) 8 Prob(RSAP > 1) Prob(Czys Snow, ZR, or IP)

  43. Example: New England Blizzard (F42: 23 January 2005 03Z) SREF 32F Isotherm (2 meter air temp) Mean (dash) Union (At least one SREF member at or below 32 F - dots) Intersection (All members at or below 32F- solid) SREF 32F Isotherm (Ground Temp) Mean (dash) Union (At least one SREF member at or below 32 F - dots) Intersection (All members at or below 32F- solid) 3h probability of freezing or frozen pcpn (Baldwin algorithm; uncalibrated) 3h calibrated probability of snow accumulating on roads

  44. Example: Washington, DC Area (F21: 28 February 2005 18Z) SREF 32F Isotherm (2 meter air temp) Mean (dash) Union (dots) Intersection (solid) SREF 32F Isotherm (Ground Temp) Mean (dash) Union (dots) Intersection (solid) 3h probability of freezing or frozen pcpn (Baldwin algorithm; uncalibrated) 3h calibrated probability of snow accumulating on roads

  45. 3h Prob Snow Accum on Roads Oct 15, 2005 (F006 v15 UTC) 6h Prob Snow Accum on Roads Oct 15, 2005 (F006 v15 UTC)

  46. Blind Test • Calibration period: Oct 1, 2004 through April 30, 2005 • 5 days randomly selected for each month in the sample => 35 days in test • Test days withheld from the monthly calibration tables (i.e., cross validation used) • The SREF forecasts were reprocessed for the 35 days and verified against the MADIS surface state observations (F03 – F63)

  47. Verification Reliability Economic Potential Value Reliability Diagram: All 3 h forecasts (F00 – F63); 35 days (Oct 1 – Apr 30)

  48. Test Results • 3 h forecast results (F00 – F63) • Forecast are reliable • Brier score is a 21% improvement over sample climatology • ROC area = .919 • Ave probability where new snow detected: 23% • Ave probability where new snow not detected: 4% • Economic value for a wide range of users peaking over 0.7

  49. Road-Snow: Summary • Method appears reliable – although 3h probabilities rarely exceed 50% • Highlights importance of ground temp predictions from SREF and deterministic models • Possible improvements: • Bias correction to 2m and ground temps from SREF* • Statistical post-processing of 2m and ground temps* prior to road-state calibration • Addition of asphalt tile to LSM of SREF members * See the next slide for temp correction information

  50. Under dispersive SREF 2m temp forecast (F15) and cold bias Raw 2m Temp F15 SREF 2m Temp Verf Period: ~August, 2005 F15 cold bias in 2m temp removed but remains under dispersive Bias adjusted 2m Temp Bias adjustment and recalibration with the addition of asphalt-type ground temp tile in LSM might be very useful for snow accumulation from SREF Recalibrated 2m Temp Uniform VOR after statistical adjustment to SREF

More Related