1 / 32

A Statistical Analysis with Operational Forecasting Applications

Reliability Trends of the Global Forecast System Model Output Statistical Guidance in the Northeastern U.S. A Statistical Analysis with Operational Forecasting Applications. John M. Goff National Weather Service Burlington, VT.

alexander-morris
Download Presentation

A Statistical Analysis with Operational Forecasting Applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Reliability Trends of the Global Forecast System Model Output Statistical Guidance in the Northeastern U.S. A Statistical Analysis with Operational Forecasting Applications John M. Goff National Weather Service Burlington, VT

  2. How is Skill Measured in Probabilistic Precipitation Forecasting (PoP)? • A commonly used method of gauging PoP forecasting skill is through the Brier Score (Brier 1950). 1 BS = 1/n  (yk – ok)2 k =1

  3. Measuring PoP Forecast Skill Continued • The Brier Score is analogous to the Mean Squared Error of the PoP forecast and thus is a measure of accuracy.

  4. Measuring PoP Forecast Skill Continued • An equally valuable measure of PoP forecast skill is reliability. • Reliability is a measure of bias, and is a gage of how accurate probability values are assigned (AWS 1978). • When statistical forecasts have little to no bias they are said to be reliable.

  5. Impetus for Research • Noticeable positive (wet) bias noted in GFS reliability scores at Burlington, VT across lower PoP categories (i.e. 0 < PoP  40). • Occurrence appeared to occur at other northeast U.S. sites, especially during the winter months.

  6. Data Set Information • Three separate data subsets were examined 1) Northeastern U.S. (20 sites) 2) New England (6 sites) 3) Burlington, VT (1 site)

  7. Data Set Domain

  8. Data Treatment and Processing • Data examined on three time scales and gathered from NWS verification website. -Two year period from October 2000 to September 2002 -Two year combined cool season from October 2000(01) to March 2001(02) -Two year combined warm season from April 2001(02) to September 2001(02)

  9. Data Treatment and Processing • GFS MOS PoP reliability scores calculated for the first three 12 hour forecast periods. • Three period average scores calculated and plotted for the time periods discussed. - Low Pop (0 < PoP  40) and high PoP (60  PoP < 100) trends are analyzed.

  10. GFS Alphanumeric Guidance 12 hour pop guidance

  11. Two Year GFS Reliability Plots

  12. Two Year Data Set Results

  13. Two Year GFS Warm Season Reliability Plots

  14. Two Year Warm Season Data Set Results

  15. Two Year GFS Cool Season Reliability Plots

  16. Two Year Cool Season Data Set Results

  17. Possible Causes of Observed Trends • GFS model coarseness/resolution • GFS MOS PoP regional regression equations • Other error sources - ASOS site location - Accuracy in precipitation measurement of the ASOS heated tipping bucket (esp. in winter).

  18. GFS MOS PoP Warm Season Regression Equations

  19. GFS MOS PoP Cool Season Regression Equations

  20. Applicability of Results to Operational PoP Forecasting • By slightly lowering GFS MOS PoP forecasts across lower PoP categories, bias may be reduced (esp. across interior northeast/New England during winter). • Discreet adjustment of GFS MOS PoP forecasts across higher PoP categories is not recommended (limited number of events).

  21. Applicability to PoP Forecasting Continued • This seems to contradict traditional Brier Score theory that hedging PoP forecasts towards the middle probabilities offers a higher probability of success over the long run (Hughes 1980).

  22. Overview • GFS MOS PoP shows consistent positive bias at lower PoP categories across the northeastern U.S. • Lower associated bias was observed across higher PoP categories.

  23. Overview Contd. • Possible causes of the observed trends include… - Coarseness in model resolution - Design of the GFS MOS Pop regional regression equations - ASOS site location and accuracy of measurement techniques (esp. in winter)

  24. Overview Contd. • It is suggested that by lowering GFS PoP forecast values by 5 to 10 percent across lower PoP categories, overall bias may be reduced at many interior northeast and New England sites…especially in winter.

  25. Conclusions • Despite noted trends the study was of limited temporal and physical constraints. • Further research on these and/or other sites across the U.S. is needed to ascertain whether trends are inherent within the GFS MOS PoP scheme.

  26. Acknowledgements • The author would like to thank Paul Sisson (SOO WFO BTV) for guidance and oversight. • Thanks is also given to Mark Antolik of MDL for guidance and expertise in the GFS MOS PoP scheme.

More Related