Evaluation of the Climate Forecast System (CFS) and consolidation forecasts

1. Evaluation of the Climate Forecast System (CFS) and consolidation forecasts Satish Regonda1,2, Dong-Jun Seo1,3 1NOAA, National Weather Service, Office of Hydrologic Development, Silver Spring, Maryland 20910, USA 2Riverside Technology, Inc., Fort Collins, Colorado 80525, USA 3University Corporation for Atmospheric Research, Boulder, Colorado 80307, USA

2. Goal • Develop technique(s) to assimilate the climate forecast information to produce reliable and skillful ensemble input forcing for mid-range to long-range forecast periods • Evaluate the climate forecast (includes calculation and analysis of various verification measures, and identification of key issues) • Develop techniques that addresses the issues in the raw climate forecast (e.g., bias removal, ensemble spread correction), and then produces reliable and skillful ensemble climate forecast Potential forecast sources: Climate Forecast System (CFS), Consolidation forecast, CPC outlook

3. Forecasts considered in this study • The CFS hindcasts • The CFS is a fully coupled model representing the interaction between the Earth's oceans, land and atmosphere (Saha et al., 2006) • Forecasts issued once in a day, but only on few specific days; i.e., successive 5-days that end on 3rd, 13th, and 23rd of each month • Forecasts produced @ every 12-hour interval for 9 months into future • Archive: 1981 – 2004 • The CFS gridded data mapped onto a climate region, and then monthly data developed using 12-hourly gridded data (data sets are produced by John Schaake)

4. Forecasts considered in this study • Consolidation forecasts • Weighted combination of four different climate models, i.e., the CFS, Canonical Correlation Analysis, Screening Multiple Linear Regression and Optimal Climate Normals (O’lenic et al., 2008; Unger et al., 2009) • Forecast issued once in a month • Forecasts produced for a season (i.e., three consecutive months) and for 13 seasons into future • Archive: 1982 – 2008

5. Literature • Very few published studies systematically evaluated the forecasts • CFS hindcasts (Luo and Wood, 2006 ) • CPC outlook forecasts (Livezy and Timfofeyeva, 2008) • Consolidation forecasts ( O’lenic et al., 2008; Unger et al., 2009) • Key points • Skill scores suggested much of the skill in the winter months, irrespective of lead time; skill is lead independent • Skill in the CPC outlook forecasts attributed to decadal climate variability, climate change and ENSO • Consolidation forecasts exhibited significant improvement compared to climatology and the official foreacsts • Performance is fn[season,lead,location,situation]

6. In this study • Seasonal forecasts corresponds to 1983 January – 2004 December with different lead times are used • Common forecast horizon is 5 seasons • Following verification measures estimated • Mean error • Correlation • Continuous Rank Probability Score (CRPS)

7. River Forecast Centers

8. Climate Regions Climate regions in the CBRFC region: 48,49,83,84,85,95,96,97,98,99

9. Climatology of seasonal precipitation

10. Climatology of seasonal precipitation

11. Climatology of seasonal precipitation

12. Climatology of seasonal precipitation

13. Climatology of seasonal precipitation

14. Climatology of seasonal precipitation

15. Climatology of seasonal precipitation

16. Boxplots of seasonal precipitation: Climatology, ensemble mean of CFS forecast, ensemble mean of consolidation forecast Click here for other climate regions

17. Mean Error, #049

18. Correlation, #049 Click here for other climate regions

19. Scatter Plot, #049 Mean Error Correlation

20. Consolidation forecast ensemble mean, verifying observed value, Boxplot of CFS ensemble, CFS ensemble mean

21. Verifying observed value (bars); Error, cumulative mean error, cumulative absolute mean error

22. Consolidation forecast ensemble mean, verifying observed value, Boxplot of CFS ensemble, CFS ensemble mean

23. Verifying observed value (bars); Error, cumulative mean error, cumulative absolute mean error

24. Continuous Ranked Probability Score (CRPS) CRPS = Reliability – Resolution + Uncertainty CRPS potential = Uncertainty – Resolution CRPSS = 1 – (CRPSforecast / CRPSreference forecast)

25. CRPS

26. Reliability

27. Resolution

28. Uncertainty

29. CRPS Potential

30. CRPSS

31. CRPS Potential

32. CRPS, Climate Region 48

33. CRPS, Climate Region 83

34. CRPS, Climate Region 84

35. CRPS, Climate Region 97

36. CRPS, Climate Region 99

37. Conclusions • Performance of the forecast varies with season, but is lead independent • For few seasons, slightly better correlation values are observed for CFS forecasts compared to consolidation forecasts • Large and small (approximately zero) biases are observed in CFS and consolidation forecasts, respectively. • Large variability is seen in CFS forecasts • Negative CRPSS values of the CFS forecasts suggest better performance of Climatology over the CFS forecasts, but positive CRPSS potential values suggest that applying bias correction techniques may improve the quality of the CFS forecasts • Sampling issues, i.e., 22 years of data is small

38. Consolidation forecast ensemble mean, verifying observed value, Boxplot of CFS ensemble, CFS ensemble mean

39. Verifying observed value (bars); Error, cumulative mean error, cumulative absolute mean error

40. Consolidation forecast ensemble mean, verifying observed value, Boxplot of CFS ensemble, CFS ensemble mean

41. Verifying observed value (bars); Error, cumulative mean error, cumulative absolute mean error

42. Mean Error, #048

43. Correlation, #048

44. Mean Error, #083

45. Correlation, #083

46. Mean Error, #084

47. Correlation,#084

48. Mean Error, #097

49. Correlation, #097

50. Mean Error, #099