Ensemble Prediction Systems and Probabilistic Forecasting

1. Ensemble Prediction Systems and Probabilistic Forecasting Richard (Rick) Jones SWFDP Training Workshop on Severe Weather Forecasting Bujumbura, Burundi, Nov 11-16 , 2013 Slides courtesy: UKMO Paul Davies and Chris Tubbs

2. ”The Blame Game” or ”The Passing of The Buck” The atmosphere is ”chaotic” Atmosphere Errorneous observations misled the NWP Scientists Computer models The NWP misled me Forecaster The forecaster misled me Customer/Public

3. Understanding chaos

4. Forecast distribution Analysis Error Data Assimilation Diagnostic Tools and Products Numerical Weather Prediction The Atmosphere Forecast Model System

5. Quantifying uncertainty with ensembles Deterministic Forecast Forecast uncertainty Initial Condition Uncertainty X CHAOS Analysis Climatology time

6. Exchanging the “accurate” forecast with a more “honest” one ψ Dangerous threshold Today’s NWP forecast Today’s EPS forecast -12 0 +12 +24 +36 +48 +60 +72 +84 h April 2013 Anders Persson

7. Correction for systematic errors ψ -12 0 +12 +24 +36 +48 +60 +72 +84 h 22/09/2014 7 Anders Persson

8. The final ensemble forecast – with verification ψ ● ● 70% 50% ● ● ● ● ● ● ● ● ● ● ● ● ● obs -12 0 +12 +24 +36 +48 +60 +72 +84 h 22/09/2014 8 Anders Persson

9. Probability NWP today We might form an opinion by looking at the last NWPs from the same model, so called “lagged” forecasts Investigations show that “jumpiness” correlates badly to the accuracy of the last forecast NWP the day before yesterday Ψ NWP yesterday Anders Persson

10. Medium range forecasting…with deterministic and EPS information Most common case Latest three NWP Most common case with good agreement between EPS spread and NWP “jumpiness” Latest EPS 22/09/2014 Anders Persson

11. Rather poor agreement between larger EPS spread and small NWP “jumpiness”. • The analysis system has obviously managed to avoid possible problems because the NWP is not very “jumpy” Rather common case Should the forecasters be more certain than the EPS indicates? 22/09/2014 Anders Persson

12. Not uncommon case Rather poor agreement between small EPS spread and large NWP “jumpiness”. The perturbations have not been quite able to cover the analysis uncertainties Should the forecasters be more uncertain than the EPS indicates? 22/09/2014 Anders Persson

13. Rare case Poor agreement between the main directions of the EPS and the NWP Best choice: create a “super ensemble” See TIGGE This puts the forecasters in a very difficult situation and there is not enough experience or investigations about this situation 22/09/2014 Anders Persson

14. The Effect of Chaos • We can usually forecast the general pattern of the weather up to about 3 days ahead. • Chaos then becomes a major factor • Tiny errors in our analysis of the current state of the atmosphere lead to large errors in the forecast – these are both equally valid 4-day forecasts. • Fine details (eg rainfall) have shorter predictability

15. Ensembles • In an ensembleforecast we run the model many times from slightly different initial conditions • This provides a range of likelyforecast solutions which allows forecasters to: • assess possible outcomes; • estimate risks • gauge confidence.

16. Reminder on scales and predictability

17. 1000km 100km 10km 1km Tropical Cyclone Space Scale MCS Front Thunderstorm Hail shaft Lifetime Predictability? 10mins 1 hr 12hrs 3 days 30mins 3 hrs 36hrs 9 days Temporal Resolution

18. Short-range Ensembles ECMWF EPS has transformed the way we do Medium-Range Forecasting • Uncertainty also in short-range: • Rapid Cyclogenesis often poorly forecast deterministically • Uncertainty of sub-synoptic systems (eg thunderstorms) • Many customers most interested in short-range • Assess ability to estimate uncertainty in local weather • QPF • Cloud Ceiling, Fog • Winds etc

19. Initial conditions perturbations • Perturbations centred around 4D-Var analysis • Transforms calculated using same set of observations as used in 4D-Var (including all satellite obs) within +/- 3 hours of data time • Ensemble uses 12 hour cycle (data assimilation uses 6 hour cycle)

20. Initial conditions perturbations Differences with ECWMF Singular Vectors: • It focuses on errors growing during the assimilation period, not growing period: - Suitable for Short-range! • Calculated using the same resolution than the forecast • ETKF includes moist processes • Running in conjunction with stochastic physics to propagate effect

21. Model error: parameterisations • QUMP (Murphy et al., 2004) • Initial stoch. Phys. Scheme for the UM (Arribas, 2004) Random parameters

22. Using probabilities • Recipients of forecasts & warnings are sensitive to different levels of risk: reflecting cost of mitigation vs expected loss • An intelligent response to forecasts & warnings depends on risk analysis, requiring knowledge of impact probability • Use of ensembles to estimate probability at longer lead times is well established in meteorology

23. 0% 25% 50% 75% 100% 10 20 30 40 50 Ψ 25%=12-13 forecasts Ψ= 10-22 25%=12-13 forecasts Ψ= 23-30 25%=12-13 forecasts Ψ= 31-37 25%=12-13 forecasts Ψ= 38-50 Median 50-50% | | ||| ||| || | | | ||| | | || || ||| |||| || || || | | | | | ||| | || | | | | 10 20 30 40 50 Ψ WMO SWFDP Macau 10 April 2013 Anders Persson

24. Probability maps

26. EC Total ppn prob > 20mm 12z Tue – 12z Wed

27. EPSgrammes

28. max 90% median (50%) Total cloud cover Deterministic 75% 6 hourly precipitation 25% 10m wind speed EPS control 10% 2m temperature min

29. Bujumbura 06 Nov 12UTC

31. The Extreme Forecast Index (EFI)

32. Extreme forecast index (EFI) • EFI measures the distance between the EPS cumulative distribution and the model climate distribution • Takes values from –1 (all members break climate minimum records) and +1 (all beyond model climate records) • The main idea is to have an index that can be conveniently mapped – removing the effect from different climatologies – to use as an “alarm bell”

33. EFI • Experience suggests that EFI magnitudes of 0.5 - 0.8 (irrespective of sign) can be generally regarded as signifying that "unusual" weather is likely whilst magnitudes above 0.8 usually signify that "very unusual" or extreme weather is likely. Although larger EFI values indicate that an extreme event is more likely, the values do not represent probabilities as such

34. Advantages with probability density functions Means and asymmetric variances are easily spotted 100% Clim. mean EPS mean Climate distribution EPS distribution 0% Temperature

35. Advantages with probability density functions Means and asymmetric variances are easily spotted 100% Climate distribution EPS distribution 0% EPS mean Temperature Clim. mean

36. EFI ~ 70% The EFI generally does not take the probability into account EFI ~ 70%

37. EFI ~ 50% For temperature the EFI can take values < 0 EFI ~ -50%

38. EFI 2m Temp

39. EFI 10m gusts

40. EFI wind speed

41. EFI precip

42. Other products

43. Tropical strike probability

44. Tropical strike

45. Ensemble mean

46. Forecast Probability

47. UKMO African LAM

48. UKMO African LAM

49. Ensemble forecasts • Ensemble forecasts help us • To judge the (un)certainty of the weather situation • To acquire probability estimates of anomalous events (extreme or high impact) • To get the most accurate and least “jumpy” deterministic forecast value

50. Questions & Answers