Deterministic vs Probabilistic Forecast

1. La Prévision Saisonnière Deterministic vsProbabilistic Forecast J.P. Céron – Météo-France

2. The Predictability « a Thunderstorm will be observed next Sunday over the Toulouse « Météopole » between 15h and 16h »  Irrealistic, the confidence that one can have in this forecast is very low « a rainy system will cross the Toulouse region Sunday afternoon  »  realistic, one can be quite confident in this forecast

3. The Predictability The predictability depends on : • The scale of the forecasted phenomenum (Thunderstorm, Easterly Wave, Blocking situation, ENSO, …) • The Range of the forecast (NowCasting, Short , Medium , Seasonnal , Climatic) Deterministic formulation error or lost of informations Probabilistic formulation more possibilities in the forecast but interpretation problems

4. The different Forecasts • How do you play to Horse races? • Informations about the form of horses, trainers, jockey, race conditions (type of soil, weather, …), predictions, … • Synthesis then making a bet on horses (in a more or less subjective choice) • Use of seasonnal forecasts. • Informations about the states of atmosphere, continental surfaces, oceanic system and its evolutions, the different forecasts, … • Synthesis and decision/action (i.e. make a bet on the real solution in a more or less objective way)

5. The different Forecasts • The horse n°5 will win the race. It will rain 650 mm at Niamey during the next rainy season. • Consequences : gain or lost depending of the success or not of the forecast.

6. The different Forecasts Deterministic forecast

7. The different Forecasts Deterministic forecast

8. The different Forecasts Deterministic forecast

9. Network density of surface observations Over the whole globe Limits of Numerical Forecasting ANALYSIS - errors (or uncertainty) from measurements, assimilation. - lack of observations over some regions The forecast errors can come from different part of the forexast system : But also ...

10. etc... Limits of Numerical Forecasting ANALYSIS - errors (or uncertainty) from measurements, assimilation. - lack of observations over some regions MODELS LIMITS - mesh, number of vertical levels (resolution of the model) - Equations, parameterisations - small scale phenomena (under the mesh or badly know) Equations are generally “simplifyed” and one calibrate “parametrisations” in the model, that is to say that one use data computed in an approximate form (even sometime as “constant”). But also ...

11. Limits of Numerical Forecasting ANALYSIS - errors (or uncertainty) from measurements, assimilation. - lack of observations over some regions MODELS LIMITS - mesh, number of vertical levels (resolution of the model) - Equations, parameterisations - small scale phenomena (under the mesh or badly know) INTERPRETATION - Models generally provide “raw data” and consequently the interpretation is often difficult) And finally ...

12. Limits of Numerical Forecasting ANALYSIS - errors (or uncertainty) from measurements, assimilation. - lack of observations over some regions MODELS LIMITS - mesh, number of vertical levels (resolution of the model) - Equations, parameterisations - small scale phenomena (under the mesh or badly know) INTERPRETATION - Models generally provide “raw data” and consequently the interpretation is often difficult) COMMUNICATION - misfitted vocabulary - misreading of users’ needs - dissemination without feedbacks from the user

13. Limits of Numerical Forecasting • Natural variability of the Ocean/Atmosphere system and the Atmospheric respons to external forcing

14. Limits of Numerical Forecasting • Uncertainties of the initial state of the Climatic system • Modelisation Error (both Oceanic and Atmosphéric) • Natural varibility of Atmosphere and its respons to external forcing • Interpretation of the forecast • To sample the initial state uncertainty  disturbances of the analysis • To sample the model uncertainty  distubances of the model • To sample in the forecast all the possible solutions of the Ocean/Atmosphere system

15. error at time t0 + range error at time t0 (initial uncertainty) Limits of Numerical Forecasting The numerical forecast using "ARPEGE«  model at short range is a « déterministic »  forecast. It uses the initial state description of the climate system and, using model’s equation, allow to perform the time evolution of the atmospheric state. Major errors of this type of forecasts mainly come from initial state errors intoduced inside the model. The uncertainty spread increase with the range of the forecast.

16. At this range: Strong probability region Deterministic forecast value of the parameter Forecast range Initial time Limits of Numerical Forecasting To take into account the probability of the deterministic evolution, one perform several forecasts starting from the same initial time but using slightly modified values of the parameters of the simulation (namely inside the probable range of errors introduced at the initial time).

17. 1 Deterministic model 0 - 6% 25 - 50% 75 - 100% 850 hPa Temperature plumes (Toulouse) base 21 september 1999 at 12h UTC verification 6 - 25% 50 - 75% Limits of Numerical Forecasting Because of computer ressources, one use a larger mesh model (to limit the computation time) and one perform, typically, around thirty forecast using different conditions for the model.

18. 1 Deterministic model 0 - 6% 25 - 50% 75 - 100% 850 hPa Temperature plumes (Toulouse) base 21 september 1999 at 12h UTC verification 6 - 25% 50 - 75% Temperature dispersion plumes as a function of the time integration This method is named “Ensemble forecast” Limits of Numerical Forecasting One can look at the parameter’s dispersion as a function of the time intergration and describe the encountered value distribution rath rather than the values themselves. One can also give a confidence indice with a more or less high value (the larger the dispersion of the forecasts, the lower the indice that is to say the lower the confidence).

19. Limits of Numerical Forecasting

20. The ensemble forecast (Resume) • To sample the initial state uncertainty  disturbances of the analysis • To sample the model uncertainty  distubances of the model • To sample in the forecast all the possible solutions of the Ocean/Atmosphere system • Several forecasts – trying to get the distribution of the possible solutions instead of a single value • Several models – How can we merge the informations coming from many different models – empirical, AGCM, COAGCM ? Multimodel approach.

21. Limites of Statistical Forecasting • Uncertainty already included inside the statistical tools

23. The different Forecasts • The horse n°5 will win the race. It will rain 650 mm at Niamey during the next rainy season. • The horse n°5 has good chances to be in the firsts in this race. The situation of the Ocean/Atmosphere system and its probable evolutions indicate that the next rainy season in Niger has a good probability to be « above the Normal ». • Consequences : gain or lost depending of the success or not of the forecast

24. The different Forecasts Probabilistic Forecast

25. The different Forecasts Probabilistic Forecast

26. The different Forecasts Probabilistic Forecast

27. The different Forecasts Probabilistic Forecast

28. The different Forecasts • Analogue technics

29. Probabilistic Forecast : formulation • 1 model et n members • p models et n members • Gaussian : mean + standard deviation • Analogues • Statistical Methods (Discriminant Analysis, Multiple Regression, Probabilistic Regression, …)

30. Catégories Forecast Probabilistic Forecast

31. Catégories Forecast • How to define the categories? • Number • Categories Limits • Needs of user? • How to evaluate the forecasted probabilities for each category? • Frequency/Probability , Climatological Probabilities, Conditionnal probabilities, Confidence Indice • Statistical Models • Numerical Models • How to transform the forecast in « readable and comprehensive » form for the user?

32. Probabilistic Forecast: verification • Quadratic Scores (Brier, RPS, …) • Relative Operating Characteristic (FA vs ND) • Cost/Lost ratio approach • Deux categories: + dry / + wet et Ratio c/L=0.5 C1= averaged cost using a climatological forecast C2 = averaged cost using a perfect forecast C3= averaged cost using a the model forecast

33. Brier Scores : BrSc = 1/N  (pi – oi)2 pi probabilité prévue pour l’événement oi variable indicatrice de l’observation de l’événement 1 BrSc= o(p) 1-p2 + 1-o(p) p2 g(p) dp 0 g(p) Fréquence relative avec laquelle l’événement est prévu avec une probabilité comprise entre p et p+dp 1 1 1 BrSc= f 1- f  +  p1-o(p) 2- f-o(p) 2 g(p) dp 0 0 0 Uncertainty Reliability Resolution Pour un système qui aurait toujours prévu la probabilité climatique d’occurrence ( pflt ) LCBrSc= f 1- f  + f 1- flt [L/S]CBrSkSc = 1 – BrSC / [L/S]CBrSc Brier Skill Score BSSREL = 1 – BSREL // [L/S]CBrSc Brier Reliability SC BSSRSL = BSRSL / Uncertainty Brier Resolution SC

34. Probabilistic Forecast: verification • Reliability Diagrams

35. Probabilistic Forecast • Cost/Lost ratio approach

36. Probabilistic Forecast • Differents users

37. Probabilistic Forecast • Comparison between Deterministic and Probabilistic formulation Probabiliste Déterministe

38. Probabilistic Forecast • Resume