Long-term trends of precipitation verification results for GME, COSMO-EU and COSMO-DE

1. Long-term trends of precipitation verification results for GME, COSMO-EU and COSMO-DE Ulrich Damrath

2. Overview • Verification using „Fuzzy“ methods • Example for the FSS and he upscaling HSS for January and July • Time series for special thresholds and window sizes • Consisency of precipitation forecasts using the CRA method (update) • current state

3. Fuzzy verification January 2008: Accumulated frequency distribution of precipitation, observation and GME

4. Fuzzy verification January 2008: Accumulated frequency distribution of precipitation, observation and CEU

5. Fuzzy verification January 2008: Accumulated frequency distribution of precipitation, observation and CDE

6. Fuzzy verification January 2008: FSS Monthly average of precipitation: 68 mm

7. Fuzzy verification January 2009: FSS Monthly average of precipitation: 30 mm

8. Fuzzy verification January 2010:Frequency distribution of precipitationObservation, GME, CEU and CDE

9. Fuzzy verification January 2010: FSS Monthly average of precipitation: 44 mm

10. Accumulated frequency distribution of precipitationJuly 2007(Observation and GME)

11. Accumulated frequency distribution of precipitationJuly 2007(Observation and CEU)

12. Accumulated frequency distribution of precipitationJuly 2007(Observation and CDE)

13. Fuzzy verification July 2007: FSS Monthly average of precipitation: 120 mm

14. Fuzzy verification July 2008: FSS Monthly average of precipitation: 88 mm

15. Fuzzy verification July 2009: FSS Monthly average of precipitation: 108 mm

16. Fuzzy verification July 2010:Frequency distribution of precipitationObservation, GME, CEU and CDE

17. Fuzzy verification July 2010: FSS Monthly average of precipitation: 78 mm

18. Fuzzy verification January 2008: HSS(UPS) Monthly average of precipitation: 68 mm

19. Fuzzy verification January 2009: HSS(UPS) Monthly average of precipitation: 30 mm

20. Fuzzy verification January 2010:Frequency distribution of precipitationObservation, GME, CEU and CDE

21. Fuzzy verification January 2010: HSS(UPS) Monthly average of precipitation: 44 mm

22. Fuzzy verification July 2007: HSS(UPS) Monthly average of precipitation: 120 mm

23. Fuzzy verification July 2008: HSS(UPS) Monthly average of precipitation: 88 mm

24. Fuzzy verification July 2009: HSS(UPS) Monthly average of precipitation: 108 mm

25. Fuzzy verification July 2010:Frequency distribution of precipitationObservation, GME, CEU and CDE

26. Fuzzy verification July 2010: HSS(UPS) Monthly average of precipitation: 78 mm

27. Fuzzy verification: Time series, choice of windows and thresholds

28. Fuzzy verification: Time series, ETS UPS GME VV:06-30

29. Fuzzy verification: Time series, ETS UPS CEU VV:06-30

30. Fuzzy verification: Time series, ETS UPS GME VV:06-18

31. Fuzzy verification: Time series, ETS UPS CEU VV:06-18

32. Fuzzy verification: Time series, ETS UPS CDE VV:06-18

33. Fuzzy verification: Time series, FSS GME VV:06-30

34. Fuzzy verification: Time series, FSS CEU VV:06-30

35. Fuzzy verification: Time series, FSS GME VV:06-18

36. Fuzzy verification: Time series, FSS CEU VV:06-18

37. Fuzzy verification: Time series, FSS CDE VV:06-18

38. Observed Forecast • Entity-based QPF verification (rain “blobs”) • by E. Ebert (BOM Melbourne) • Verify the properties of the forecast rain system against the properties of the observed rain system: • location • rain area • rain intensity (mean, maximum) CRA error decomposition The total mean squared error (MSE) can be written as: MSEtotal = MSEdisplacement + MSEvolume+ MSEpattern Configuration for the current study: - “Observations”: forecasts: 06-30 hours - Forecasts : forecasts: 30-54 hours and forecasts: 54-78 hours

39. Consistency of precipitation forecasts: Parts of error decomposition Autumn 2009 Dark :forecasts 30-54 h Light:forecasts 54-78 h

40. Consistency of precipitation forecasts: Parts of error decomposition Winter 2009/10 Dark :forecasts 30-54 h Light:forecasts 54-78 h

41. Consistency of precipitation forecasts: Parts of error decomposition Spring 2010 Dark :forecasts 30-54 h Light:forecasts 54-78 h

42. Consistency of precipitation forecasts: Parts of error decomposition Summer 2010 Dark :forecasts 30-54 h Light:forecasts 54-78 h

43. Consistency of precipitation forecasts: Parts of error decomposition Summer 2010 Dark :forecasts 30-54 h Light:forecasts 54-78 h

44. Summary • Fraction skill score and upscaling ETS are considered. Both scores are relatively high correlated. • Fuzzy verification in general shows best results for low precipitation values and large window sizes • For some months best results can be seen for precipitation amounts around 2 mm (12 h)-1 • CEU and CDE models have nearly the same quality and are better than GME especially during summer times. • A positive long term trend of precipitation quality can be seen for low precipitation values and large window sizes. No clear trend is visible for high precipitation values for any window size. • Results for the check of consistency of precipitation forecasts lead to the expected (but proved) results that for high thresholds the inconsistency is most obvious. During winter time pattern errors are dominant. During summer times displacement errors are prevailing.

45. One conclusion • Forecasters sometimes really like CDE. • But the future is CDE-EPS!