1 / 32

Model uncertainty in climate forecasting from seasons to decades: the ENSEMBLES project

Model uncertainty in climate forecasting from seasons to decades: the ENSEMBLES project. Francisco J Doblas-Reyes ECMWF, Reading, UK. Outline. Description of the ENSEMBLES project. Hindcast production in ENSEMBLES: Streams 1 and 2.

vidar
Download Presentation

Model uncertainty in climate forecasting from seasons to decades: the ENSEMBLES project

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Model uncertainty in climate forecasting from seasons to decades: the ENSEMBLES project Francisco J Doblas-Reyes ECMWF, Reading, UK

  2. Outline • Description of the ENSEMBLES project. • Hindcast production in ENSEMBLES: Streams 1 and 2. • Diagnosis of the ENSEMBLES simulations and forecast quality assessment. • Applications: the actual value of forecast systems. • Data quality control and dissemination. • Inclusion of NCEP CFS in a European multi-model.

  3. The ENSEMBLES project • Integrated Project funded by the EC within the VIth FP, Sep 2004-Aug 2009, 69 partners. • Main goal: development of an integrated probabilistic prediction system for time scales from seasons to decades, and beyond. Therefore, research on both seasonal-to-decadal (s2d) and anthropogenicclimate change (ACC). • S2d to be used to assess the reliability of forecast systems used for scenario runs, both from a climate modelling and user perspective, in a seamless framework. • Forecast uncertainty: comparison of the merits of multi-model, perturbed parameters and stochastic physics approaches to deal with model error. • Diversity of applications: health, crop yield, energy…

  4. Dealing with forecast uncertainty • Model error is a major source of forecast uncertainty. Three approaches to deal with model error are being investigated in ENSEMBLES: multi-model, stochastic physics and perturbed parameters. • The multi-model is built from ECMWF, Met Office, Météo-France operational activities and the DEMETER experience at IfM-Kiel, CERFACS and INGV. • Perturbed parameter system stems from the decadal prediction system (DePreSys) created at the Met Office. • Stochastic physics system uses the ECMWF stochastic backscatter system developed for medium-range forecasts. • A major s2d experiment to determine the benefits of each approach is ongoing (~20,000 years of integrations).

  5. S2d ENSEMBLES global experiment • Three systems: multi-model (ECMWF, GloSea, DePreSys, Météo-France, IfM-Kiel, CERFACS, INGV), stochastic physics (ECMWF) and perturbed parameters (DePreSys). • Hindcasts in two streams: • Stream 1: hindcast period 1991-2001, seasonal (7 months, May and November start date), annual (14 months, November start date) and 2 decadal (1965 and 1994), 9 member ensembles, ERA40 initialization in most cases. • Stream 2: As in Stream 1 but over 1960-2005, with 4 start dates for seasonal hindcasts, at least 1 for annual and at least one 3-member decadal hindcast every 5 years. • Additional simulations: DePreSys_PP carries out 14-month hindcasts, a 10-year hindcast every year and a 30-year hindcast every 5 years + lots of sensitivity experiments from the other contributors.

  6. Stream 1 annual hindcasts Sea surface temperature RMSE (solid) and spread (dashed) averaged over the Niño3.4 region for the 1st November (bottom row) start dates over the period 1991-2001. RMSE of simple persistence in dashed black. All forecast systems beat persistence. Multi-model is the most skilful system, with highest reliability (RMSE~spread), in the first 6 months, while the perturbed parameter system is as good for longer lead times. Perturbed Parameters (9 members) ECMWF Stochastic Physics (9 members) Multi-model (5 models, 45 members)

  7. Stream 1 seasonal hindcasts Reliability diagrams for 1-month lead seasonal (JJA) precipitation above the upper tercile over the tropical band for the ENSEMBLES Stream 1 multi-model (left, 45 members), stochastic physics (centre, 9 members) and perturbed parameters (right, 9 members) hindcasts started in May over the period 1991-2001 verified against GPCP. The Brier and ROC skill scores, along with 95% confidence intervals (in brackets) computed using a bootstrap method, are shown on top of each panel. Stochastic physics Perturbed parameters Multi-model 0.110 (0.060,0.160) 0.415 (0.348,0.477) 0.018 (-0.043,0.070) 0.347 (0.272,0.411) 0.016 (-0.054,0.083) 0.339 (0.271,0.408)

  8. ROCSS for anomalies above the lower tercile Anomaly correlation coefficient Ratio between spread and RMSE Stream 1 seasonal hindcasts Scores for southern South America precipitation from Stochastic Physics, Perturbed Parameters (both with 9-member ensembles) and Multi-model (5 models, 45 members). Sample values are shown with black dots along with 95% confidence intervals obtained using a bootstrap method (verified against GPCP over 1991-2001).

  9. Stream 1 seasonal hindcasts Ratio of spread to RMSE of the three different forecast systems for several regions, lead times (1-1, 2-4, 5-7 months), events (anomalies above/below the upper/lower tercile), start dates (May and November) and variables (T2m, precipitation, Z500 and MSLP) computed over the period 1991-2001. The inset numbers indicate the number of cases where a system is superior with 95% confidence. 200 157 156 17 78 58

  10. Stream 1 seasonal hindcasts ROC skill score of the three different forecast systems for several regions, lead times (1-1, 2-4, 5-7 months), events (anomalies above/below the upper/lower tercile), start dates (May and November) and variables (T2m, precipitation, Z500 and MSLP) computed over the period 1991-2001. The inset numbers indicate the number of cases where a system is superior with 95% confidence. 115 203 33 25 9 106

  11. Stream 2 ROC area for Stream 2 JJA (start date 1st of May) 2-metre temperature above the upper tercile over the period 1960-2005 (left) and 1981-2005 (right, hindcast period for ECMWF System 3). 1981-2005 1960-2005

  12. Decadal predictions: IfM 10-year mean prediction of meridional overturning circulation and tropical Pacific SST from 3-member ensembles started every five years over 1955-2005. From Keenlyside et al. (2008)

  13. Decadal predictions: IfM 10-year mean prediction of global mean surface temperature from 3-member ensembles started every five years. From Keenlyside et al. (2008)

  14. Decadal predictions: ECMWF T2m difference with ERA40/OPS for 10-year ensemble integrations started on 1st Nov 1994 of different IFS cycles (HOPE ocean model). Global ocean Global land NH land Tropical oceans

  15. Decadal forecasts: DePreSys First two years of predictions (anomalies wrt 1979-2001) from June 2005 (Jun 2005-May 2007) for uninitialized (NoAssim), initialized (DePreSys) and AR4 simulations. Comparison to NCEP analyses. Note the improved forecast for initialized predictions, although not everywhere. D. Smith & J. Murphy (Met Office)

  16. DePreSys: Stream 1 decadal predictions Anomaly correlation coefficient for DePreSys_PP as a function of lead time (from 1-1 to 27-62). Each dot shows the result for a version of the model, the final dot of each set being for the ensemble mean. The coloured bars indicate the 95% confidence intervals obtained using a bootstrap method with 1,000 samples. Tropical precipitation Northern Hemisphere T850

  17. DePreSys: Stream 1 decadal predictions Scores for the DePreSys versus DePreSys_PP for several regions (tropics, Northern Hemisphere, Europe, North America), lead times (3-14, 15-26, 27-62 months), events (anomalies above/below the upper/lower tercile and above the median), start dates (May and November) and variables (T2m, Z500, T850, precipitationand MSLP) over the period 1991-2001. The inset numbers give the cases when the value for a system is significantly above the corresponding value of the other. BSSI RelSS ROCSS 6 3 9 43 43 18

  18. Surface temperature THC intensity 30-year climate forecasts: DePreSys Thirty-year DePreSys forecasts started on March 2007 for uninitialized (NoAssim, 4 members) and initialized (DePreSys, 10 members) ensembles. D. Smith & J. Murphy (Met Office)

  19. A user strategy: the end-to-end approach • A broad range of forecast products might be offered, but a specific analysis of the user requirements is necessary. • End-to-end is based on collaboration and continuous feedback. • End users develop their models taking into account climate prediction limitations. • Users should use objective records of performance. • The final level of forecast quality that provides added value is defined by the application -> user-oriented verification. End users should assess the final value of the predictions. • Forecast reliability becomes a major issue.

  20. Adaptation using climate information • The possibility of adaptation to climate change via a learning process taking place at the interannual time scale, when actions can be corrected, is an obvious way to achieve a high degree of integration (integration of time scales) between climate information providers and users. • This implies: • That similar forecast information is provided in the climate forecasting and climate-change contexts. • Involvement of both climate scientists and end-users to consider the whole range of time scales. For instance, crop managers consider long-term climate change as a process that takes place on a yearly basis.

  21. ………… 4 3 2 1 N Downscaling N-1 Application model ………… 2 1 4 3 N N-1 non-linear transformation 0 0 Forecast probability of PP Forecasts probability of malaria Seasonal prediction with dynamical models ………… 4 3 2 1 Forecast N N-1

  22. Areas with epidemic malaria in Africa Dynamical predictions for malaria warning Precipitation composites for the five years with the highest (top row) and lowest (bottom row) malaria incidence in Botswana for DJF DEMETER seasonal predictions (left) and CMAP (right). Quartiles define extreme events (outbreaks) for malaria prediction

  23. Very low malaria Available in March Available in November Very high malaria Gain in lead time for malaria warning Probabilistic predictions of standardised malaria incidence quartile categories in Botswana with five months lead time. -- high malaria years -- low malaria years

  24. Gain in lead time for malaria warning Botswana malaria index from Thomson et al. (2005) and incidence simulated by the process-based Liverpool malaria model (LMM) using ERA40. There is a disagreement between both models for the year 2000: is it due to the impact of extreme temperature or precipitation? Interaction of variables (which might be triggered by climate change) affects the user From A. Jones (Univ. of Liverpool)

  25. Public services • ENSEMBLES s2d intends to provide the scientific community with full access to the hindcasts and with automatic tools to perform different analysis. A public data dissemination system with innovative features has been developed. • We are collaborating with some ENSEMBLES partners on: • Expanding the KNMI Climate Explorer for exploratory analysis. • Developing a downscaling web portal at the University of Cantabria.

  26. Hindcasts run/archived at ECMWF (access to member state users) MARS ECFS common dataatmosphere common dataocean additional data ECMWF firewall ENSEMBLES public data server (5 Tb) Diagnostics and downscaling common dataocean common dataatmosphere ClimateExplorer & Downscaling portal OPeNDAPserver MARS client Archiving and dissemination strategy

  27. EUROSIP and NCEP CFS • CFS: • Monthly 9-month long, 15-member ensembles hindcasts over the period 1981-2005 with a coupled ocean-atmosphere. • Atmospheric component initialized from NCEP analysis using lagged average method and ocean from an assimilated ocean state. • T62L64 atmospheric resolution, 1°x1° (1/3° in the tropics) 40 levels ocean. • EUROSIP: • ECMWF (11 members, 1981-2005 hindcast period), GloSea (15 members, 1987-2005) and Météo-France (5 members, 1993-2005, although now updated to 11 members and 1981-2005) integrated monthly following the EUROSIP strategy. • Common period 1993-2005.

  28. EUROSIP and NCEP CFS: biases Precipitation biases against GPCP for summer (JJA, May start date) over the period 1993-2005 ECMWF Met Office Météo-France CFS

  29. EUROSIP and NCEP CFS: tropical SSTs Top row: SST RMSE (solid) and spread (dashed) for the Niño3.4, Niño4 and Niño1.2 regions against Reynolds for the May and November start dates over the period 1981-2005. Bottom row: SST ACC Multi-model has a similar skill to System 3; CFS does not seem to add much skill, although individual forecasts are fairly different; CFS has more variability than System 3 Niño3.4 Niño1.2 Niño4 Persistence System 3CFSMulti-model

  30. EUROSIP and NCEP CFS: Forecast quality Scatter plots of EUROSIP vs EUROSIP+CFS multi-model forecast quality over 1993-2005 in terms of reliability skill score (RelSS) and ROC skill score (ROCSS). Symbols indicate results for 19 regions, 2 start dates (May and November), 3 variables (T2m, MSLP, PP), 4 lead times (1-3, 2-4, 3-5, 4-6) and 2 events (lower and upper terciles). The numbers in brackets indicate differences significant with a 95% confidence using a two-sample inference test.. RelSS ROCSS 101 77 3 22

  31. Summary • Stream 1 and 2 ensemble simulations are available from ECMWF. In the seasonal/annual time scale, the multi-model seems to provide better results overall. • Decadal forecasting has made significant progress, forecast systems have been developed and preliminary attempts show that useful climate information is available. • Progress has been made to increase the public access to the data. • The inclusion of NCEP CFS in the EUROSIP multi-model increases the reliability of the ensemble (mainly because of the increase in the ensemble size) and the accuracy.

More Related