On the use of radar data to verify mesoscale model precipitation forecasts

1. On the use of radar data to verify mesoscale model precipitation forecasts Martin Goeber and Sean Milton Model Diagnostics and Validation group Numerical Weather Prediction Division Met Office, Bracknell, U.K. contributions from Clive Wilson, Dawn Harrison, Dave Futyan and Glen Harris

2. Outline of talk • Importance of precipitation verification • Verifying observations and ‘wet’ model • Statistical methods and interpretation • Examples from operational mesoscale model forecasts from the wet autumn 2000

3. # observations per model grid box 0.1:1 5:1 12 km Model 5km gauge 5km 12 km Nimrod Rain gauge representative of an area of about 20 km2 on mesoscale timescales (Kitchen and Blackall 1992)

4. Characteristics of the Nimrod data • Ground clutter removal • fixed Z-R conversion • attenuation correction • removal of corrupt images • removal of anaprop • accounting for variations in the vertical reflectivity profile • gauge adjustment

5. ‘Wet’ Forecast system characteristics • 3D-Var • latent heat nudging, 3D-cloud from MOPS • cloud microphysics with ice and explicit calculation of transfer between phases • prognostic cloud liquid water and ice • penetrative convection scheme based on an ensemble of buoyant entraining plumes with a treatment of downdraughts

6. Categorical statistics Count concurrent event/no-event, e.g. precipitation > 2 mm / 3 hours

7. Categorical measures (1) Hit rate False alarm rate

8. Which measures for categorical statistics ? • complete description of 2*2 contingency table • description of different aspects of relationship between forecasts and observations, e.g. independence from marginal distributions • confidence intervals • interpretation • relationship to value

9. Categorical measures (2) Frequency bias Hansen-Kuipers score Odds ratio

10. Categorical measures (3)Confidence intervals

11. Autumn 2000 Orography Accumulation

12. Accumulation autumn (SON) 200012km resolution, 18-24 h forecasts Model Nimrod-obs Bias

13. Rain/no-rain (>0.4 mm/6hrs)12km resolution, 12-18 h forecasts Bias HKS Odds ratio

14. Heavy precipitation (>4mm/6hrs)12km resolution, 12-18 h forecasts Bias HKS Odds ratio

15. Heavy precipitation (>4mm/6hrs)36km resolution, 12-18 h forecasts Bias HKS Odds ratio

16. Estimates of confidence intervals Minimum(a,b,c,d) Error(HKS) Error(log(OR))

17. Summary (6h) Frequency bias Hansen-Kuipers score Odds ratio

18. Summary (3h) Frequency bias Hansen-Kuipers score Odds ratio

19. Regionally integrated statistics A) Obs b) 2-3hr c) 8-9hr d) 14-15 e) 20-21 f) 26-27 a) observed area, b) hourly accumulation c) wet area, d) maximum

20. Regionally integrated statistics

21. Regionally integrated statisticsProbability for rain in one hour Nimrod (obs) 6-12hrs. forecast 18-24 hrs. forecast

22. Regionally integrated statisticsBrier skill score for p(rain in one hour) 6-12 hrs. forecasts 18-24 hrs. forecasts

23. Summary • Nimrod (radar) data are relatively great for verifying mesoscale quantitative precipitation forecasts, because of their spatial and temporal resolution and near real time availability. • Last autumn’s extreme precipitation in England and Wales was relatively well forecasted. • Time series of regionally integrated statistics and categorical data analysis provide scientifically based, yet customer friendly, measures to verify quantitative precipitation forecasts.

24. Future developments • Application and development of tests on significance of difference between two samples (e.g. convective vs. frontal precipitation, orographic enhancement, spinup, dependency on resolution, new model formulations) • Extension of investigations on catchment scale • Comparison of spatio-temporal spectral characteristics of model and observations • Lagrangian (event based) statistics