Verification of the Cooperative Institute for Precipitation Systems‘ Analog Guidance Probabilistic Products

1. Verification of the Cooperative Institute for Precipitation Systems‘ Analog Guidance Probabilistic Products John P. Gagan NOAA/NWS Springfield, MO Fred H. Glass NOAA/NWS St. Louis, MO Michael S. Evans NOAA/NWS Binghamton, NY Chad M. Gravelle and Dr. Charles E. Graves Saint Louis University - Department of Earth and Atmospheric Sciences 2009 National Weather Association Annual Meeting - Norfolk, Virginia 20 October 2009

2. The What and Why of Analogs What: • If the current state of the atmosphere resembles a previous state then the two are termed analogs, and for a period of time, the current state may evolve in a similar fashion as the past state (Lorenz 1969). • Recent research has had success using the “perfect prog” approach to find analogs. • Hanson (2007) • Root et al. (2007) • Diomede et al. (2008) • Evans and Murphy (2008) Why: • Provides a conditional climatology of the forecast • Confidence in NWP model output • Historical Impacts - NCDC Storm Data, COOP snow event maps, snowfall potential, etc. • Historical framework

3. CIPS Analog Guidance - The Big Picture • Search the 29-yr North American Regional Reanalysis (NARR) dataset against the model forecast (GFS212-40km) for potential analogs. • 6 months over the winter season (OCT - MAR) • 6 h temporal resolution • ~21,112 potential analogs (29 winters, 6 months, 4 per day) • Remove “duplicate” times by choosing the “best” analog over a 24-h period. 1984011512, 1984011518, 1984011600, 1984011606 • Refine and rank the resulting analogs. • Create products that are useful for winter weather guidance. 300 HGHT 500 HGHT 700 FRNT 850 HGHT 850 TMPC 850 TMPC 850 FRNT 850 THTEADV 2m TMPC PMSL PWTR

4. CIPS Analog Guidance Products - Examples

5. CIPS Analog Guidance Products - Examples

6. CIPS Analog Guidance Products - Examples

7. CIPS Analog Guidance Products - Examples

8. CIPS Analog Guidance Products - Examples

9. CIPS Analog Guidance Products - Examples

10. Goals of the Verification Study • Establish a basic set of metrics to assess future improvements in the analog system. • Do the top analogs consistently identify snowfall potential (e.g., does the >6” 50% probability guidance capture the area of >6” of snowfall?)? • Assess the significance of the probabilistic snowfall guidance (e.g., what does a >4” 30% probability of snow indicate?). • What thresholds and probabilities are the most reliable in providing snowfall guidance?

11. Winter 08-09 Organized Snow Event Verification Data Verification statistics were determined using the following: • 35 organized (>2”) snow events occurred during the winter of 2008-2009 east of the Rocky Mountains. • 26 events occurred in one of the Midwest domains. • 14 events occurred in one of the East Coast domains. • 5 events occurred in both Midwest and East Coast domains. • Four GFS forecasts (F036, F048, F060, F072) and the run’s associated CIPS Analog Guidance Probabilistic Products were used for each event and domain where applicable; 180 total forecast runs. • Seasonal verification statistics represent the average over the 180 forecast runs.

12. East Coast Domain 1 Case - 27-29 January 2009 • Winter storm affected the CONUS from the mid-Mississippi River Valley through New England between 27 and 29 January 2009. • Continental United States mid-latitude cyclone track. • Area of >8” snow fell from western New York into interior New England with amounts >12” in higher elevations. • A mix of snow, sleet, and freezing rain fell to the south of the heavy snow. • Analogous to the winter storm that affected the Northeast from 15-17 December 2007 (inset). Similar track, snowfall amounts, and precipitation type. 3rd Best Analog COOP Snow Event for the 72-h period ending 1200 UTC 20090129

13. East Coast Domain 1 Case - >4” Probability Guidance

14. East Coast Domain 1 Case - >4” Probability Guidance

15. East Coast Domain 1 Case - >4” Probability Guidance

16. East Coast Domain 1 Case - >4” Probability Guidance >40% probability: Observed Yes No 4360 1319 667 14844 Yes No Forecast • POD: 0.867 (1.0) • FAR: 0.232 (0.0) • THREAT SCORE: 0.687 (1.0) • BIAS: 1.13 (1.0)

17. Winter 08-09 Probabilistic Snow Guidance Results

18. Attribute Diagrams • The attribute diagram plots the observed relative frequency against the forecast probability (in our case the analog guidance). • How well do the predicted probabilities of an event correspond to their observed frequencies? WWRP/WGNE Forecast Verification Research • Reliability is indicated by the proximity of the plotted curve to the diagonal. How close are you to perfect reliability? • Resolution indicates the ability to assess the change in frequency. How close is your curve to a 1:1 slope ?

19. Attribute Diagrams • The attribute diagram plots the observed relative frequency against the forecast probability (in our case the analog guidance). • How well do the predicted probabilities of an event correspond to their observed frequencies? WWRP/WGNE Forecast Verification Research • Reliability is indicated by the proximity of the plotted curve to the diagonal. How close are you to perfect reliability? • Resolution indicates the ability to assess the change in frequency. How close is your curve to a 1:1 slope ?

20. East Coast Domain 1 Case - >4” Probability Guidance • Contour area shows the region where analog probability guidance is 40%. • Observed Relative Frequency: 0.395 • Example of nearly perfect reliability.

21. East Coast Domain 1 Case - >4” Probability Guidance • Contour area shows the region where analog probability guidance is 50%. • Observed Relative Frequency: 0.678 • Example of under forecasting.

22. East Coast Domain 1 Case - >4” Probability Guidance • Contour area shows the region where analog probability guidance is 30%. • Observed Relative Frequency: 0.214 • Example of over forecasting.

23. Winter 08-09 Probabilistic Snow Guidance Results

24. Conclusions • POD results indicate at a given snowfall threshold that the area enclosed by the lower percentiles (10-40%) contain the majority of the threshold snowfall. • The threat scores indicate that for all thresholds there is a maximum in the 30-50% percentile range. • The attribute diagrams reveal an expected “overforecasting” of guidance probabilities at all thresholds. However, results between 30-70% have the most reliability. • In all thresholds, higher guidance probabilities indicate higher relative observed frequency of snowfall (i.e., good resolution). • Though not examined, we speculate that higher guidance probabilities at low thresholds (i.e., 80% at 4”) may be a better indicator of higher end snowfall potential than low probabilities at high thresholds (i.e., 30% at 8”).

25. Questions NWA 2009 Analog Posters: P3.19 Using Regional Historical Analogs as Guidance for a Midwestern Winter Weather Event P3.22 Use of a Historical Analog-Based Winter Storm Guidance Package for Forecasting a Central New York Snow Event Questions or comments? gravelle@eas.slu.edu or gravesce@slu.edu The analog guidance can be found at: http://www.eas.slu.edu/CIPS/ANALOG/analog.php