Dave Stainforth Acknowledgements: A. Lopez. F. Niehoerster, E. Tredger, N. Ranger, L. A. Smith

1. Challenges in the Extraction of Decision Relevant Information from Multi-Decadal Ensembles of Global Circulation Models Dave Stainforth Acknowledgements: A. Lopez. F. Niehoerster, E. Tredger, N. Ranger, L. A. Smith Grantham Research Institute & Centre for the Analysis of Timeseries, London School of Economics and Political Science Climate Change Workshop Statistical and Applied Mathematical Science Institute 18th February 2010 • Introduction and context. • The difficulties in predicting climate. • Domains of possibility. • Metrics. • Implications for future experiments. • [Transfer functions]

2. Introduction • Climate models can help us: • understand the physical system. • generate plausible storylines for the future. • build better models. • Context: • responding to societal desire for predictions of the impacts of climate change • providing information to guide climate change adaptation strategies. • “minimise vulnerability/maximise resilience” .vs. “predict and optimise” • International adaptation – when is adaptation “adaptation” and when is it development? • More uncertainty, please.

6. Climate Prediction – A Difficult Problem • A problem of extrapolation: • Verification / confirmation is not possible. • Model deficiencies: • Model inadequacy: they don’t contain some processes which could have global impact. (methane clathrates, ice sheet dynamics, a stratosphere, etc.) • Model uncertainty: Some processes which are included are poorly represented – e.g. ENSO, diurnal cycle of tropical precipitation. • Model interpretation: • Lack of model independence. • Metrics of model quality • Observations are in-sample. • Ensembles are analysed in-sample. • Models which are bad in some respects may contain critical feedbacks in others. • Non-linear interactions: selecting on a subset of variables denies the highly non-linear nature of climatic interactions.

7. Types of Climate Uncertainty • External Influence (Forcing) UncertaintyWhat will future greenhouse gas emissions be? • Initial Condition Uncertainty(partly aleatory uncertainty)The impact of chaotic behaviour. • Model Imperfections(epistemic uncertainty)Different models give very different future projections. Figure: IPCC – AR4

8. Uncertainty Exploration

9. Climate Prediction – A Difficult Problem • A problem of extrapolation: • Verification / confirmation is not possible. • Model deficiencies: • Model inadequacy: they don’t contain some processes which could have global impact. (methane clathrates, ice sheet dynamics, a stratosphere, etc.) • Model uncertainty: Some processes which are included are poorly represented – e.g. ENSO, diurnal cycle of tropical precipitation. • Model interpretation: • Lack of model independence. • Metrics of model quality • Observations are in-sample. • Ensembles are analysed in-sample. • Models which are bad in some respects may contain critical feedbacks in others. • Non-linear interactions: selecting on a subset of variables denies the highly non-linear nature of climatic interactions.

10. Consequences of Lack of Independence 1 See Stainforth et al. 2007, Phil Trans R.Soc A Climateprediction.net data

11. Consequences of Lack of Independence 2 From Stainforth et al. 2005

12. Can Emulators Help Out Here? No • Even the shape of model parameter space is arbitrary so filling it in does not help in producing probabilities of real world behaviour.

13. An Aside: UK Climate Projections 2009 (UKCP09) - 1 Change in mean summer precip: 10% 90% Murphy et al, 2004 UKCIP, 2009

14. An Aside: UK Climate Projections 2009:Change in Wettest Day in Summer Medium (A1B) scenario 2080s: 90% probability level:very unlikely to be greater than 2080s : 67% probability level:unlikely to be greater than

15. An Aside: A (Very) Basic Summary of My Understanding of the Process • sample parameters, • run ensemble, • emulate to fill in parameter space, • weight by fit to observations Emulate

16. An Aside: Issues • Size of ensemble given size of parameter space. • The ability of the emulator to capture non-linear effects. • The choice of prior i.e. how to sample parameter space. • The justification for weighting models. • On what scales do we believe the models have information?

17. Choices of Model Parameters • Most model parameters are not directly representative of real world variables. e.g. the ice fall rate in clouds, the entrainment coefficient in convection schemes. • Their definition is usually an ad hoc choice of some programmer. (Possibly a long time ago, in a modelling centre far away.) • Thus a uniform prior in parameter space has no foundation and • testing the importance of such a prior is not a matter of tweaks around the edges (adding 15% to the limits, or exploring a triangular prior around central values); • rather it is a matter of sensitivity to putting the majority of the prior points in one region:

18. An Aside: Issues • Size of ensemble given size of parameter space. • The ability of the emulator to capture non-linear effects. • The choice of prior i.e. how to sample parameter space. • The justification for weighting models. • On what scales do we believe the models have information? Choice of parameter definition

19. Estimated distributions for climate sensitivity: upper bounds depend on prior distribution Uniform prior in sensitivity Uniform prior in feedbacks Frame et al, 2005

20. Climate Prediction – A Difficult Problem • A problem of extrapolation: • Verification / confirmation is not possible. • Model deficiencies: • Model inadequacy: they don’t contain some processes which could have global impact. (methane clathrates, ice sheet dynamics, a stratosphere, etc.) • Model uncertainty: Some processes which are included are poorly represented – e.g. ENSO, diurnal cycle of tropical precipitation. • Model interpretation: • Lack of model independence. • Metrics of model quality • Observations are in-sample. • Ensembles are analysed in-sample. • Models which are bad in some respects may contain critical feedbacks in others. • Non-linear interactions: selecting on a subset of variables denies the highly non-linear nature of climatic interactions.

21. Domains of Possibility 1 From Stainforth et al. 2005

22. Domains of Possibility 2 See Stainforth et al. 2007, Phil Trans R.Soc A Climateprediction.net data

23. Climate Prediction – A Difficult Problem • A problem of extrapolation: • Verification / confirmation is not possible. • Model deficiencies: • Model inadequacy: they don’t contain some processes which could have global impact. (methane clathrates, ice sheet dynamics, a stratosphere, etc.) • Model uncertainty: Some processes which are included are poorly represented – e.g. ENSO, diurnal cycle of tropical precipitation. • Model interpretation: • Lack of model independence. • Metrics of model quality • Observations are in-sample. • Ensembles are analysed in-sample. • Models which are bad in some respects may contain critical feedbacks in others. • Non-linear interactions: selecting on a subset of variables denies the highly non-linear nature of climatic interactions.

24. Best Information Today / Best Ensemble Design For Tomorrow • For tomorrow: Design ensembles to push out the bounds of possibility. • For today: Use the best exploration of model uncertainty combined with the best global constraints.

25. Issues/Questions in Ensemble Design to Explore Uncertainty • Emulators to guide where to focus parameter space exploration.(Potentially very powerful in distributed computing experiments.)How? • Simulation management to minimise the consequence of in-sample analysis.How? • Questions of how we describe “model space” to enable its exploration. • How do we evaluate the spatial and temporal scales on which a model is informative? • How do we integrate process understanding with model output in such a multi-disciplinary field. • How do we integrate scientific information with other decision drivers. • Better understanding and description of the behaviour non-linear systems with time dependent parameters. • How do we evaluate information content?

26. Resolution .vs. complexity .vs. uncertainty exploration • What processes do we need to include in our models? • What do we need our models to do to answer adaptation questions? • What would be the perfect ensemble? • What should be the next generation ensemble?

27. Let’s Be Careful Out There