Poor Man's EPS experiments and LAMEPS plans at the Met Office

1. Poor Man's EPS experiments and LAMEPS plansat the Met Office Ken Mylne and Kelvyn Robertson Met Office

2. Why PEPS (Poor Man’s EPS)? • Storms of Dec 1999 over Europe were poorly forecast by most deterministic models, even at 24h • Need for effective short-range ensemble to reduce risk of missing severe weather events • Existing operational ensembles (eg ECMWF) designed for medium-range (3-10days) • some evidence of poor performance for severe events in short-range • PEPS is an ensemble formed by combining the operational output from several NWP centres • provides a relatively cheap way of obtaining short-range ensemble forecasts

3. Why PEPS might work • Multi-model multi-analysis ensemble • experiments in USA have shown this is important (eg Hou et al, 2001; Stensrud et al, 1999) • Random sampling of initial condition errors • may be important for estimating probabilities at short-range • Previous studies (eg Ziehmann, 2000) have shown encouraging results

4. Preliminary system • 9 models • Low-res (5x5°) • H500 and pmsl only • Output every 24h • Data stored and used by VT, not DT

5. Verification - Brier Skill • Brier Skill Scores, using the ECMWF EPS as ref. • Several PEPS configurations • all available models • one model removed (all versions) • all plus 6 members of EPS • reduced combinations • Range of PMSL thresholds • 126 days from 7th Feb to 12th June 2001

6. Hi-Res PEPS Success of the preliminary system encouraged us to set up a much larger PEPS system: • Larger ensemble • around 15 members from 9 models • 2m Temp • 10m Windspeed • Precipitation • Higher resolution • tests at 1.25x1.25° • output every 12h • More fields • PMSL • H500 • T850

7. Data Exchange • Met Office UM • ECMWF • DWD • Meteo-France • BoM • JMA • KMA • CMC • NCEP • Russia • 9 centres agreed to supply forecast data • Data are pulled from FTP sites in near-real time • European data via ECMWF fast link • Other centres via the internet

8. Brier Skill - Winter DJF 2001/02 • Results similarto preliminary experiments • Reference EPS is 12 hours older due to late data cut-off • provides the gain which could be achieved operationally

9. Effect of 12h Advantage With 12h Without 12h • Re-ran verification without giving PEPS the 12h advantage • Apparent PEPS skill mostly comes from the 12h advantage • Without: • No skill at T+24 • Slight advantage at T+84

10. BSS - Different Weather Parameters Results similar for all weather parameters:- H500 T850 T 2m 10m WS PMSL T+24 T+72

11. BSS - PMSL in Regions PMSL results poor over S. Hemisphere. Europe N. Am. S. Hem. N. Hem. T+24 T+72

12. BSS - 2m Temperature in Regions T2m results poor over S. Hemisphere. Best over continents but still poorer than EPS. Europe N. Am. S. Hem. N. Hem. T+24 T+72

13. BSS - Wind Speed in Regions Benefit for more extreme events in all regions:- Europe N. Am. S. Hem. N. Hem. T+24 T+72

14. Rank Histograms • PMSL over Northern Hemisphere • over-spread at 24-48h • good spread but slight bias at longer lead-times • EPS underdispersive at all times to T+120

15. Rank Histograms • Focus on over-spreading at T+24-48 • Northern hemisphere average hides strong regional bias over Europe • still some over-spreading • And an opposite regional bias over N. America

16. Rank Histograms • Focus on over-spreading at T+24-48 • Southern hemisphere shows stronger over-spreading • probably due to analysis biases • Difficult to separate ensemble spread from differences in model biases • Some apparent over-spreading may be due to biases in the verifying ECMWF analysis • Need for bias correction

17. Rank Histograms • Weather parameters • PMSL • 500hPa Height • Strong bias (analysis?) • Some over-spreading • T850 • Over-spreading

18. Rank Histograms • Weather parameters • PMSL • 2m Temperature • Over-spreading • 10m Wind Speed • Over-spreading • Bias

19. Reliability Diagrams • PMSL<970mb over Northern Hemisphere • reliability good for PEPS and for EPS

20. Reliability Diagrams • H500<480dm over Northern Hemisphere • some general under-forecasting - possibly bias in ECMWF analysis, as seen in Rank Histograms

21. Reliability Diagrams • 2m Temperature • <260 deg C • better reliability than EPS for all thresholds • <280 deg C • <300 deg C

22. Conclusions on PEPS • PEPS advantage over EPS was due to the 12h lag applied to EPS • little scientific advantage of PEPS method at T+24 • slight advantage at T+84 (multi-model?) • PEPS over-spread at short-range • regional biases make interpretation difficult • some evidence for better reliability for extreme events • Experiments with bias-corrected PEPS should clarify results • set up to run over the coming winter

23. Plans for LAMEPS Aims: • Risk assessment for rapid cyclogenesis • Uncertainty of sub-synoptic systems • assess probability forecasts of precipitation, low cloud and visibility • LBCs for future storm-scale ensembles • The Met Office is devising plans for a short-range ensemble based on a LAM covering the Atlantic and Europe.

24. LAMEPS Perturbation Strategy To be fully effective LAMEPS will need perturbations to: • Initial conditions • Model physics parametrizations • Lateral boundaries • Surface parameters

25. LAMEPS Perturbation Strategy To be fully effective LAMEPS will need perturbations to: • Initial conditions • Model physics parametrizations • Surface parameters • Lateral boundaries

26. Initial Condition Perturbations Options: • Singular vectors (as used at ECMWF) • Error breeding (Toth and Kalnay, 1993) (as used at NCEP) • Ensemble data assimilation (CMC, Houtekamer et al, 1996) • Ensemble Kalman Filter (Bishop et al, 2001) • Multi-analysis (INM)

27. Singular vectors Error breeding Ensemble data assimilation Ensemble Kalman Filter Multi-analysis Maximise ensemble growth over early forecast range (48h at ECMWF) Possibility of combining SVs optimised at 6h, 12 and 18h (Hollingsworth, personal communication) Some evidence that SVs only provide reliable probabilities for severe weather events well after the optimisation period Initial Condition Perturbations

28. Early Warnings of Severe Weather from EPS 1 day 2 days • Verification of severe weather warnings based on the EPS • Discrimination of events is best at 4 days (ROC) • Better discrimination is independent of calibration • Reliability is best at day 4 and useless at days 1-2 3 days 4 days

29. Early Warnings -Brier Skill Scores • Brier Skill also tends to increase after day 2. Heavy Rain Severe Gales

30. Singular vectors Error breeding Ensemble data assimilation Ensemble Kalman Filter Multi-analysis Relatively simple to implement Identifies modes growing rapidly at analysis time may provide a more random sampling in the early forecast But… bred vectors are not orthogonal tend to converge not worth running more than 5-8 cycles Initial Condition Perturbations

31. Singular vectors Error breeding Ensemble data assimilation Ensemble Kalman Filter Multi-analysis Multiple data assimilation cycles with perturbed observations computationally expensive Accounts for model errors Monte-Carlo method random sampling, so should provide reliable probabilities In practice did not perform very well at CMC insufficient spread to scale with forecast errors Initial Condition Perturbations

32. Singular vectors Error breeding Ensemble data assimilation Ensemble Kalman Filter Multi-analysis Various configurations exist Theoretically optimal not tested in full NWP models difficulties with some obs types computationally expensive Ensemble Transform Kalman Filter (Bishop et al, 2001) may provide the best system in the long-term Initial Condition Perturbations

33. Singular vectors Error breeding Ensemble data assimilation Ensemble Kalman Filter Multi-analysis Relatively cheap and simple reliability may be a problem Accounts for model errors No attempt to identify rapidly growing modes Monte-Carlo method random sampling, so should provide reliable probabilities PEPS results suggest: over-spreading need for bias corrections Initial Condition Perturbations

34. Singular vectors Error breeding Ensemble data assimilation Ensemble Kalman Filter Multi-analysis Initially we will use Error Breeding Later we hope to develop EnKF Initial Condition Perturbations

35. LAMEPS Perturbation Strategy To be fully effective LAMEPS will need perturbations to: • Initial conditions • Model physics parametrizations • Surface parameters • Lateral boundaries

36. Model Physics Perturbations Again many options… main priorities: • Convection • Cloud/microphysics • impact on radiation • Surface roughness

37. Model Physics Perturbations Approaches: • Multi-model • effective • opportunity for effective collaboration • Multi-scheme • eg. Kain-Fritsch or Betts-Miller convection • Perturbed tendency • as used at ECMWF • Stochastic physics schemes • conceptually and theoretically elegant • research required - role for universities

38. LAMEPS Perturbation Strategy To be fully effective LAMEPS will need perturbations to: • Initial conditions • Model physics parametrizations • Surface parameters • Lateral boundaries

39. Surface Parameters • Surface Roughness • fixed but uncertain - perturb between members • variable over sea • impact through windspeed, heat and moisture fluxes • Soil moisture, SST, snow cover etc • analysed • could be perturbed randomly

40. LAMEPS Perturbation Strategy To be fully effective LAMEPS will need perturbations to: • Initial conditions • Model physics parametrizations • Surface parameters • Lateral boundaries

41. Lateral Boundary Conditions • Large domain designed to allow uncertainties to grow within the domain, but... • By T+72 significant uncertainty may emanate from beyond the western boundary • Error breeding will grow modes over the previous 24h, so important even for 48h forecasts

42. Lateral Boundary Conditions Options: • ECMWF Ensemble • Random perturbations • Global model breeding at low resolution

43. ECMWF Random Global breeding Readily available especially if use member-state time on ECMWF computers But… Possible balance problems using LBCs from different model Each new EPS run has new perturbations - no continuity with the LAM bred modes likely generate noise Lateral Boundary Conditions

44. ECMWF Random Global breeding Simple to apply Usual problem of random perturbations - not focussing on the growing modes Lateral Boundary Conditions

45. ECMWF Random Global breeding Avoids problems of others: identifies growing modes continuity from run to run But… Expensive, unless run at low resolution grid-length for LBCs should not be more than 4-5 times longer Lateral Boundary Conditions

46. Lateral Boundary Conditions Options: • ECMWF Ensemble • Random perturbations • Global model breeding at low resolution No decision has been taken

47. Outline of LAMEPS Plans • Ensemble based on European Mesoscale • 20km grid-length initially • Minimum 10 members • Run to T+48, possibly to T+72 later • Error breeding - possibly EnKF later • Multi-schemes for convection • research into stochastic physics • Perturbed Surface Roughness • Perturbed LBCs • ECMWF EPS or low-resolution Global breeding

48. Collaboration Opportunity Dispersed multi-model ensemble • Relatively simple approach to model errors • Share computing demands • Share system maintenance demands • Option to run multiple components at ECMWF on member-states’ time

49. New Met Office HQ, Exeter Planned Time-Scales • Start work April 2003 • Year 1 (incl. Relocation): • Error breeding system • Convection perturbations • First test run • Year 2 (2004-2005): • Version 1 of full perturbation system • System set up for real-time running • Year 3 (2005-2006): • Verification report on real-time performance

50. Questions?