Skill Variability Diagnosis for Discriminating Use of CPC Long-Lead Seasonal Forecasts

1. Skill Variability Diagnosis for Discriminating Use of CPC Long-Lead Seasonal Forecasts Bob Livezey and Marina Timofeyeva NOAA/NWS/OCWWS/Climate Services Division Climate Prediction Applications Science Workshop March 9-11, 2004 Tallahassee, FL

2. Outline • Introduction • CPC Skill Graphs and Some Other Stratifications • Results • Conclusions and Lessons

3. Seasonal Temperature Forecast Skill 1960s to 80s All Seasons 8.3 Winter 12.6 Spring 8.6 Summer 9.3 Fall 2.8 Introduction • Users should only care about the performance of forecasts that can potentially benefit their decision process • Livezey (1990): There are non-random subsets of seasonal forecasts that were skillful enough to be useful • After 1997-98 a common presumption was that forecasts are generally skillful enough to be useful

4. Introduction (Cont.) • This talk will make the point : • That was made by Livezey (1990) • That there are many non-random subsets of forecasts that do not have useful skill • That it is critical for this information to be shared with potential users • That skill analyses with different stratifications are highly informative while CPC’s web displays are not

5. Displays and Stratifications • CPC Seasonal Forecasts • For 3-equally probable temperature and precipitation classes at 102 Climate Divisions • Made every month from 1995 to present for 0.5-, 1.5-, …, 12.5 month leads • Skill Measure: Modified Heidke Skill Score of Categorized Forecasts • Displays and Stratifications • CPC: Summed over all forecasts for each lead and displayed with times series for this lead • Here: • Summed over all forecasts for each lead and all leads displayed together • Stratified further by cold seasons (DJF to FMA) and warm seasons (MAM to NDJ) • Stratified further by strong ENSO years vs. other years • Stratified by region

6. Stratification by Lead and Seasons: Temperature Displays and Stratifications • CPC Seasonal Forecasts • For 3-equally probable temperature and precipitation classes at 102 Climate Divisions • Made every month from 1995 to present for 0.5-, 1.5-, …, 12.5 month leads • Skill Measure: Modified Heidke Skill Score of Categorized Forecasts • Displays and Stratifications • CPC: Summed over all forecasts for each lead and displayed with times series for this lead • Here: • Summed over all forecasts for each lead and all leads displayed together • Stratified further by cold seasons (DJF to FMA) and warm seasons (MAM to NDJ) • Stratified further by strong ENSO years vs. other years • Stratified by region

7. Further Stratification by Strong-ENSO vs Other Years: Temp.

8. Further Stratification by Strong-ENSO vs Other Years: Temp.

9. Displays and Stratifications • CPC Seasonal Forecasts • For 3-equally probable temperature and precipitation classes at 102 Climate Divisions • Made every month from 1995 to present for 0.5-, 1.5-, …, 12.5 month leads • Skill Measure: Modified Heidke Skill Score of Categorized Forecasts • Displays and Stratifications • CPC: Summed over all forecasts for each lead and displayed with times series for this lead • Here: • Summed over all forecasts for each lead and all leads displayed together • Stratified further by cold seasons (DJF to FMA) and warm seasons (MAM to NDJ) • Stratified further by strong ENSO years vs. other years • Stratified by region

11. Stratification by Lead and Regions: Temp.

12. Results • Seasonal Temperature: • Useable skill confined to strong ENSO years and mainly at short to medium leads • Otherwise skill is dominantly level with lead (derived from biased climatologies, ie long-term trend) • Best forecasts are for strong-ENSO cold seasons at very short leads • Worst forecasts are for cold seasons at longer leads for strong ENSOs and at very-short leads for other years • Skill is substantially higher than average in the West, and substantially lower in the East • Short-lead forecasts are better now than for the 1960s-80s, ~13 vs ~8 overall, ~20 vs ~13 for the winter

13. Stratification by Lead and Seasons: Precipitation

14. Further Stratification by Strong-ENSO vs Other Years: Precip.

15. Further Stratification by Strong-ENSO vs Other Years: Precip.

17. Stratification by Lead and Regions: Precip.

18. Results • Seasonal Precipitation: • Barely useable skill entirely confined to strong ENSO years in short to medium leads • Otherwise skill is either negative or statistically indistinguishable from zero • Best forecasts are for strong-ENSO cold seasons at short leads • Skill is a little higher than average in the South, and a little lower in the North • Short-lead forecasts overall seem to be no better now than for the 1960s-80s

19. Conclusions and Lessons • There are non-random subsets of seasonal forecasts that are skillful enough to be useful • There are many non-random subsets of forecasts that do not have useful skill (should we be issuing them?) • It is critical for this information to be shared with potential users • Skill analyses with different stratifications are highly informative while CPC’s web displays are not • Skills mainly reflect ENSO and trend signals • Temperature forecasts are getting better