1 / 33

Sarah Beare (nee John) Neill Bowler, Marie Dando, Anette Van der Wal, Rob Darvell

Performance of the MOGREPS Regional Ensemble. Sarah Beare (nee John) Neill Bowler, Marie Dando, Anette Van der Wal, Rob Darvell. Outline. Overview of MOGREPS Configuration Overview of probabilistic verification Results Summary and future work. MOGREPS-Regional. Regional Ensemble

davida
Download Presentation

Sarah Beare (nee John) Neill Bowler, Marie Dando, Anette Van der Wal, Rob Darvell

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Performance of the MOGREPS Regional Ensemble Sarah Beare (nee John) Neill Bowler, Marie Dando, Anette Van der Wal, Rob Darvell

  2. Outline • Overview of MOGREPS Configuration • Overview of probabilistic verification • Results • Summary and future work

  3. MOGREPS-Regional • Regional Ensemble • Regional ensemble over N. Atlantic and Europe (NAE) • 23 perturbed members + control • IC Perturbations from regional ETKF • LBC from global ensemble • Stochastic physics - random parameters • 24km resolution, 38 levels • Run at 6Z & 18Z to T+54

  4. MOGREPS Operational System diagram New global analysis Global ensemble forecast using stochastic physics LBCs Perturbations mixed and scaled by ETKF New NAE analysis New NAE analysis 0Z 12Z 18Z 06Z

  5. MOGREPS Verification

  6. Verification • Aim to verify products issued from MOGREPS system • Two systems available • Focus on surface parameters and higher thresholds • Further results available in verification report recently published on met office website. http://www.metoffice.gov.uk/research/nwp/publications/papers/technical_reports/index.html

  7. Site-specific Verification system - 79 sites across the UK and Europe. • Probabilistic forecast data binned into 10% categories • Ability to compare with ECMWF forecasts - Ability to compare RAW model forecasts and post-processed data • Can calculate and plot : Reliability diagrams, Empirical ROC Curves, Brier Score and decompositions Brier Skill Score • CRPS in the process of being added

  8. Area Based Verification System • Can verify against observations or analyses - Forecast probabilities not binned • Can calculate and plot : Reliability and attributes diagrams, Empirical and parametric ROC, Economic value, Brier Score and decompositions Brier Skill Score Spread and Skill based measures • Would like to include the RPS, CRPS • Can compare Global v Regional ensemble. • THORPEX group introducing ECMWF EPS data for comparison purposes (no facility to account for different ensemble sizes).

  9. Performance of ensemble and high-res (12km) Ensemble mean has lower RMSE than high-res for day 2 forecasts, indicating presence of substantial uncertainties 10m w/s 2m Temp Verification for UK stations for Dec 2006 to Feb 2007. T+48 data only available after 6 Feb

  10. Spread-skill relationship – wind speed T+30 • Spread-skill for wind-speed binned into equal population bins by spread • Skill corrected for observation error • Blue – MOGREPS • Pink – No Skill • Green – Perfect DJF 06/07 JJA 06

  11. Spread-skill relationship – temperature • Spread-skill for temperature binned into equal population bins by spread • Skill corrected for observation error • Blue – MOGREPS • Pink – No Skill • Green – Perfect DJF 06/07 JJA 06

  12. Temperature

  13. Temp DJF06/07 (79 sites UK & Europe) Temp < 0oC T+36 Temp < -2oC T+36

  14. Brier Skill Score components - Temperature > 10oC 6 Nov 2006 - 28 Feb 2007 Temp>10C • 79 sites UK & Europe • 6 Nov 2006 – 28 Feb 2007 Resolution Brier Skill score Reliability

  15. Reliability diagram for surface temperature • Reliabilitydiagram for Temp>10C • 79 sites UK & Europe • 6 Nov 2006 – 28 Feb 2007

  16. Temperature > 10oC KFMOS Kalman-filter bias-corrected forecasts. Resolution Brier Skill score Reliability

  17. Precipitation

  18. Brier Skill Score components for Precipitation • 12-hour precip>0.5mm • 79 sites UK & Europe • 1 July 2006 – 31 March 2007 • 12-hour precip>5mm

  19. Reliability diagram for precipitation • Reliabilitydiagram for 12h precip>5mm • 79 sites UK & Europe • 1 July 2006 – 31 March 2007

  20. 6hr precip > 0.3mm against gridded analysis • Reliability and sharpness diagram for T+36 forecast. • 6h precip > 0.3mm • Verification against Nimrod Analysis over the UK at 1.0 degree resolution. • 1st Jan 06 – 28th Feb 07

  21. 6hr precip 5mm against gridded analysis • Reliability and sharpness diagram for T+36 forecast. • 6h precip > 5mm • Verification against Nimrod Analysis over the UK at 1.0 degree resolution. • 1st Jan 06 – 28th Feb 07

  22. 6hr precip 25mm against gridded analysis • Reliability and sharpness diagram for T+36 forecast. • 6h precip > 25mm • Verification against Nimrod Analysis over the UK at 1.0 degree resolution. • 1st Jan 06 – 28th Feb 07

  23. Wind speed

  24. Reliability diagram for wind speed • Reliabilitydiagram for wind speed >F5 at T+30 • 79 sites UK & Europe • 6 Nov 2006 – 31 March 2007

  25. Wind speed at least gale force 8 • Reliability and sharpness diagram for T+36 forecast. • 10m Wind > F8 • Verification against surface obs over UK and Europe. • 1 Jan 06 – 28 Feb 07

  26. Wind speed at least severe gale force 9 • Reliability and sharpness diagram for T+36 forecast. • 10m Wind > F9 • Verification against surface obs over UK and Europe. • 1 Jan 06 – 28 Feb 07

  27. Wind speed at least storm force 10 • Reliability and sharpness diagram for T+36 forecast. • 10m Wind > F10 • Verification against surface obs over UK and Europe. • 1 Jan 06 – 28 Feb 07

  28. Summary of verification • Ensemble mean beats control and hi-res from ~18h • Very good reliability for non-extreme events • Useful skill for some more severe events • T<-2C • Windspeed > F9 • 6h Precip > 5mm/12h • KFMOS bias correction effective in reducing many biases • Better forecasts from NAE than ECMWF, even after KFMOS for some parameters • Combination of resolution and perturbation strategy • Useful spread-skill in many cases • Exception is temperature in summer

  29. Future Plans • Fully operational early 2008 • SKEB2 scheme including convection for improved stochastic physics • Vertical localization of ETKF to improve near-surface perturbation scaling • Perturbations to lower boundary (eg soil moisture, SST) • Enhanced resolution: • Global 60km from 2009 • Regional 16km from 2009 and 12 from 2011 • Small convection-resolving ensemble from 2011 • ~1.5km over UK • ~6 members • Nested on members selected by clustering

  30. Any Questions?

  31. More Information…. • The MOGREPS short-range ensemble prediction system: Verification • report - Trial Performance of MOGREPS January 2006 - March 2007 • N Bowler, M Dando, S Beare and K Mylne (2007) Technical Reports 497 The MOGREPS short-range ensemble prediction system Part 1: System Description N Bowler, A Arribas, K Mylne and K Robertson, (2007) • Comparison of error breeding, singular vectors,random perturbations and • ensemble Kalman filter perturbation strategies on a simple model. • N Bowler (2004) 452 Results of an initial stochastic physics scheme for the Met Office Unified Model. A. Arribas (2004)

  32. NEW! Regional ETKF • Until end of May MOGREPS Perturbations for the NAE were interpolated from the global at T+6 • Large scale perturbations • Spread too large (especially at upper levels) • Since beginning of June perturbations to regional ensemble generated by a regional ETKF. • Smaller, more realistic perturbations • Better representation of important short length scales • But surface perturbations may be too small

  33. Subjective Evaluation of Severe Weather • Forecaster comments/feedback Severe Gale Event – 02/09/06 …. the NAE MOGREPS output clearly captured the strength and gave a good forecast for potential severe gale gusts…. (Paul Davies, Chief Forecaster) Intense rainfall - 20/07/07 … The risk of very high totals is confirmed by NAE MOGREPS members and also other models… (Steve Willington, Chief Forecaster)

More Related