S2D Prediction Activities at CCCma(p)

1. S2D Prediction Activities at CCCma(p) • HFP2 (2-tier seasonal) • basis of CMC operational forecasts • Analyses • Trend in SIP forecasts • CHFP1 and 2 (1-tier seasonal/interannual) • initial approach and results • analyses and initialization • development path • DHFP (decadal) • potential predictability results • WGCM/WGSIP/CMIP5 and decadal prediction

2. Why retrospective forecasts? • Retrospective forecasts are crucial for - establishing forecast skill - providing forecast climatology for bias correction - guiding forecast calibration and post-processing • Current EC 2-tier operational system: - 4 AGCMs x 10-ensemble - validated by 2nd Historical Forecast Project (HFP2) - 4-month retrospective forecasts initialized each month 1969-2003 • Validate also 1-tier coupled forecasts in CHFP

3. HFP2 • CMC deterministic and probabilistic seasonal forecasts are based on HFP2 • HFP2 (2nd Historical Forecasting Project) • retrospective 1-season forecast experiment • operational context – no information from the future • 2-tier - SST forecast provides boundary conditions for AGCM forecasts • multi-model, multi-realization – each of 4 models produces 10-member ensemble forecasts • we use results from 33-year period 1970-2002

4. Seasonal forecastsJan-Feb-Mar 2009 Deterministic Probabilistic

5. Skill assessment of HFP2 forecasts • analysis by S. Kharinet al. (2008) • deterministic forecasts • continuous valued • categorical • combining multi-model ensemble mean forecast information • unweighted • variance weighted (2 methods) • regression improved • scale unweighted ensemble mean (single parameter) • scale each model mean (four parameter) • skill measures: correlation and MSSS • probabilistic categorical • combining multi-model forecast information • count value • gaussian • adjusted gaussian • skill measures: Brier skill score • benefits of Multi-model approach

6. Correlation and MSSS for Sfc air T methods of combining multi-model forecasts 1 parameter regression weighting 4 parameter regression weighting

8. Percent correct, 3-category forecasts Seasonal variation of skill No dependence on combination method

9. 3-category probabilistic forecasts Brier skill score Gaussian methods Count methods Adjusted gaussian Below Normal Reliability

10. The virtues of the MME approach For same overall ensemble size more models give better scores

11. More models are better Diminishing returns Four models is a “practical” choice

12. Summary • Deterministic forecasts • not critical how MME formed • scaled MME result (1 parameter) increases MSSS but degrades correlation • scaled MME result (4 parameters) degrades scores • Probabilistic forecasts • gaussian method better than count • adjusted gaussian only better over ocean (on account of persisted SSTAs) • Multi-model methods • for same ensemble size more models the better • because of diminishing returns, 4 models (as HFP2) is reasonable “practical” choice

13. Climate trends and seasonal forecasts • trend in HFP2 • no GHG or other forcing in HFP2 • implicit in persisted SSTAs and initial conditions • characterization of trend • statistical correction • trend in CHFP1 • GHG forcing effect in coupled forecast model Boer (2008)

14. Background • Observational studies show long timescale trends thought to be due to GHG and other forcing - together with long timescale variability • Externally forced trends should provide an additional seasonal forecasting signal - if present and properly forecast • Do we capture these trends in the 33-years of retrospective forecast data of HFP2 (and does it matter)?

15. -17 t 17 Linear trends in <T850> for DJF • For global means <T> = <a>t + <T’> - fit linear trends by least squares • forecast trend weaker than trend in NCEP reanalysis data • we also fit trends to each point NCEP Forecast Units: oC years

16. Trends in SAT from NCEP and HFP2 DJF NCEP JJA NCEP DJF HFP2 JJA HFP2 Units oC/decade

17. Trends in Z500 from NCEP and HFP2 DJF NCEP JJA NCEP DJF HFP2 JJA HFP2 Units m/decade

18. Trends • trends are larger at higher latitudes, over land and in winter – the signature of global warming • trends in forecasts are weaker than in reanalysis data • suggests that lack of GHG forcing may degrade the forecasts

19. Time evolution of error • spatial average of MSE over globe • 3 components • error in trend • error in non-trend component • cross-product

20. Dealing with anomalies minimizes MSE in DJF T850 Mean error Mean square error total Anomalies non-trend trend Mean square error Mean error NCEP total Trends set to zero at t=0 non-trend trend HFP2 cross-product global means MSE over globe

21. Can we improve skill by adjusting trend? • try simple statistical adjustments based on scaling or trend adjustment • only cases where single parameter is estimated to avoid over-fitting • evaluate in cross-validation mode • use MSSS and Correlation scores

22. Statistical adjustments

23. Raw forecast Scaled Trend replaced Trend added

24. Raw forecast Scaled Trend replaced Trend added

25. Raw forecast Scaled Trend replaced Trend added

26. Temperature at 850hPa (T850) Anomaly Correlation 1 December Initialization Preliminary evidence of impact of radiative forcing trend on coupled model forecast skill No radiative forcing trend in model Weak radiative forcing trend Stronger radiative forcing trend Forcing can affect both land and ocean in this case

27. Summary • Observed/reanalysis trends have GHG signature • Trends in seasonal forecasts are weak • Statistically “correcting trends” gives some improvement in skill over Asia but not, unfortunately, over North America • Suggests seasonal forecasting models should include radiative forcing due to GHG and aerosols - physically realistic (if properly inserted) - some hope for improved forecasts • For coupled CHFP forecasts SSTs may also drift and we have some evidence GHG forcing is needed

28. The Coupled Historical Forecast Project, version 1: Formulation, results, and progress towards CHFP2 Bill Merryfield, George Boer, Greg Flato , Slava Kharin , Woo-Sung Lee , Badal Pal , John Scinocca Canadian Centre for Climate Modelling and Analysis Environment Canada

29. Anomalous wind stress warm cool South America Asia Pilot Project: CHFP1 • Based on CGCM3.1/T63 (IPCC AR4) • Simple SST nudging initialization after Keenlyside et al. (Tellus 2005): - Strongly relax SST to observed 1970-2001 time series - Anomalous wind stress tends to set up correct equatorial thermocline configuration:

30. CHFP1 Ensemble Generation • Construct 10 initial conditions for 1 Sep (e.g.) by combining atm and ocn states from preceding week: • Launch forecasts 1 Feb, 1June, 1 Sep, 1 Dec 1971-2000 • (10 ensemble members) x (4 initializations yr-1) x 30 yrs 1200 years of coupled model integration

31. CHFP1 Persistence Damped persistence CHFP1 Results (Ensemble size=10) NINO3.4 skills Anomaly correlation Mean square skill score • Not too bad for off-the shelf model, crude initialization • Much room for improvement, improvement being realized…

32. Coupled Forecast System Development Path CGCM development AGCM, OGCM Ocean data assimilation “Off the shelf” CGCM S assim after Troccoli et al. (MWR 2002) 2D Var after Tang et al. (JGR 2004) Improved error covariances CHFP2 CHFP1 Atmospheric initialization Simple SST nudging initialization Insertion of reanalyses Atm data assimilation Land initialization Off-line forcing by bias-corrected reanalysis GHG forcing, trend correction, etc Analysis and Verification

33. Much improved model ENSO with new AGCM, OGCM • Exemplified by “hit” for 11-month lead prediction of 1982/83 El Nino: Deterministic forecast SSTA Nov 1982 New AGCM, OGCM Lead=11 mo Obs SSTA Nov 1982

34. NINO3 Power Spectrum CGCM4 OBS 1970-99 CGCM3.8 OBS 1940-69 1 10 10-1 100 PERIOD (Y) CGCM3.1

35. Ocean Data Assimilation • Initially use approach of Tang et al. (JGR 2004): - input ocean reanalysis in lieu of observations - simple variational assimilation level-by-level (2D Var) - background error covariances of Derber & Rosati (JPO, 1989) • Assimilate multiple ocean analyses • Explore methods to improve error covariances

36. Ocean Initialization by multi-analysis assimilation • Experiment: compare NINO3.4 skill and ensemble spread for three ensemble initialization strategies: - Multi-analysis: off-lineassimilation of 6 ocean analysis products (same atm) - Exp_atmos: 6 AGCM states from consecutive days prior to forecast start (same ocn) - Exp_ocean: 6OGCM states from consecutive days prior to forecast start (same ocn) • 1980-2001: 22 years of Sep 1–initialized forecasts

37. NINO3.4 skill and ensemble spread Ensemble Spread  RMS Error Lead month multi-reanalysis exp_atmos exp_ocean SST Forecast Skill Correlation Lines : Ensemble mean Symbols : Ensemble members Lead month Multi-analysis ocean initialization leads to • Improved skill at longer leads • Larger ensemble spread in first two months

38. Atmospheric Initialization • SST nudging informs AGCM of boundary forcing, but not correct synoptic configuration, i.e. weather major loss of skill in first month of forecast • Two approaches are being pursued: - Direct insertion of atmospheric analysis (cf. HFP2) - Simple assimilation of analysis into AGCM

39. .95 .85 .75 .65 .55 >.5 Land Initialization • CFCAS/GOAPP funded collaboration with A. Berg (Guelph) • Force land surface model with bias-corrected reanalyses • after Berg et al. (Int J Clim 2005) Correlation of NCEP monthly precip with gauge-based measurements in USA: before bias correction after bias correction Berg et al., 2003: 2005

40. Summary • Coupled forecasts offer means for seasonal forecasting at longer leads, where future evolution of SSTA is critical • Prototype CHFP1 competitive with 4-model HFP2 at 1-month lead, but has only simplest initialization • CHFP1 provides a benchmark against which model and initialization improvements leading to CHFP2 can be assessed • CCCma participation in international CHFP through CLIVAR/WGSIP

41. Many scientific “opportunities” • improved models • analysis of variability and of modes of variability • improved analysis methods especially in the ocean • for model initialization • for verification • for model development • ensembles • ensemble generation • multi-model ensembles • prediction studies and the “DHFP” • WGSIP/WGCM/CMIP5 coordinated project

42. Coupled Forecast System Development Path CGCM development AGCM, OGCM Ocean data assimilation “Off the shelf” CGCM S assim after Troccoli et al. (MWR 2002) 2D Var after Tang et al. (JGR 2004) Improved error covariances CHFP2 DHFP1 CHFP1 Atmospheric initialization Simple SST nudging initialization Insertion of reanalyses Atm data assimilation Land initialization Off-line forcing by bias-corrected reanalysis GHG forcing, trend correction, etc Analysis and Verification

43. End of presentation