590 likes | 747 Views
The COSMO-LEPS system: present status and outlook Andrea Montani, Chiara Marsigli and Tiziana Paccagnella ARPA-SIMC Hydrometeorological Service of Emilia-Romagna, Bologna, Italy. XI General COSMO meeting Offenbach,7-10 September 2009. Present status, recent updates, verification
E N D
The COSMO-LEPS system: present status and outlookAndrea Montani, Chiara Marsigli and Tiziana PaccagnellaARPA-SIMCHydrometeorological Service of Emilia-Romagna, Bologna, Italy XI General COSMO meeting Offenbach,7-10 September 2009 A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
Present status, recent updates, verification • Future changes and plans Outline A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
COSMO-LEPS suite @ ECMWF: present status 16 Representative Members driving the 16 COSMO-model integrations (weighted according to the cluster populations) employing either Tiedtke or Kain-Fristch convection scheme (randomly choosen) + perturbations in turbulence scheme 3 levels 500 700 850 hPa 4 variables Z U V Q d+3 d+4 d d+5 d+1 d+2 d-1 Cluster Analysis and RM identification Cluster Analysis and RM identification older EPS 00 2 time steps younger EPS 12 European area clustering period Complete Linkage • suite runs as a “time-critical application” managed by ARPA-SIMC; • Δx ~ 10 km; 40 ML; fc+132h; • COSM0 v4.7 since Feb09 (with Runge-Kutta + multi-layer soil scheme); • computer time (6.0 million BU for 2009) provided by the COSMO partners which are ECMWF member states. COSMO-LEPS Integration Domain COSMO-LEPS clustering area A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
25 February 2009: • upgrade of COSMO to model version 4.7 with a few Meteoswiss code adaptation; use of “old” wind-gust diagnostic (itype_diag_gust=2); • 24 May 2009: • beginning of experimental suite at 7 km with model version 4.8; • 15 June 2009: • upgrade of INT2LM to model version 1.8 (no use of the “official executable” available at ECMWF, as ECMWF local definitions are not handled); use of ECMWF orography retrieved in spherical harmonics and no more on gaussian grid. Recent upgrades A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
Model explosion for COSMO-LEPS (last year) every now and then, “Floating-point exceptions” when the model writes output files over small subdomains … Temporarily patch (which works, anyway!): the failed task is resubmitted with half time-step. A more sophisticated investigation of model explosions will be carried out in the near future. CURED by model upgrade! A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
Objective verification of COSMO-LEPS • SYNOP on the GTS Main features: variable: 12h cumulated precip (18-06, 06-18 UTC); period : from Dec 2002 to Jul 2009; region: 43-50N, 2-18E (MAP D-PHASE area); method: nearest grid point; no-weighted fcst; obs: synop reports (about 470 stations/day); fcst ranges: 6-18h, 18-30h, …, 102-114h, 114-126h; thresholds: 1, 5, 10, 15,25, 50mm/12h; system: COSMO-LEPS; scores: ROC area, RPSS, Outliers, … both monthly and seasonal scores were computed A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
Time series of ROC area • Area under the curve in the HIT rate vs FAR diagram. • Valuable forecast systems have ROC area values > 0.6. • Poor performance of the system in Spring and Summer 2006 (both particularly dry), despite system upgrades. • Good performance during DOP 2007. • Small positive impact after 2007 system upgrades. • Scores in 2009 similar to those of last year (worsening for the highest threshold). A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
Seasonal scores of ROC area • Performance of the system assessed for the last 5 summers (JJA) and the last 5 autumns (SON). • Consider the “event” 10 mm/12h; valuable forecast systems have ROC area values > 0.6. • Need to take into account the different statistics for each season (JJA 2004 and SON 2006 less rainy than the others). • Good scores during the full DOP in 2007. • Best performance for JJA and SON 2008, more evident for longer ranges. • Higher scores in SON than in JJA. A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
Time series of Brier Skill Score • BSS is written as 1-BS/BSref. Sample climate is the reference system. Useful forecast systems if BSS > 0. • BS measures the mean squared difference between forecast and observation in probability space. • BS equivalent to MSE for deterministic forecast. • The improvement of model performance is detectable for all thresholds along the years. • Still problems with high thresholds for shorter ranges. • For the 78-90h range, a good trend has been evident since 2007: BSS has been positive for all thresholds in the last 12 months. A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
Ranked Probability Skill Score: time series + seasonal scores • A sort of BSS “cumulated” over all thresholds. RPSS is written as 1-RPS/RPSref. Sample climate is the reference system. RPS is the extension of the Brier Score to the multi-event situation. • Useful forecast systems, if RPSS > 0. • Performance of the system assessed as time series and for the last 5 autumns (SON). • the improvement of the system performance is detectable for all forecast ranges along the years; • RPSS positive for all forecast ranges in 2007 and since May 2008. • Best results for last autumn (almost no more 12-hourly cycle in the score). A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
Outliers: time series + seasonal scores • How many times the analysis is out of the forecast interval spanned by the ensemble members. • … the lower the better … • Performance of the system assessed as time series and for summers 2004-2008. • Evident seasonal cycle (more outliers in winter), but overall reduction of outliers in the years up to 2007. • Reduction of outliers from one summer to the other, related to the increase of ensemble size (evident for the 10m to 16m increase). • Need to take into account the different statistics for each season. • Best results for the last 3 summers. A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
Present activity • Implementation of COSMO-LEPS at 7 km • COSMO-LEPS for TIGGE-LAM • Multi-model clustering (“bridge-talk” WG4-CONSENSUS) A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
Implementation of COSMO-LEPS at 7 km New system (COSMO-LEPS_7) x = 7 km z = 40 ML t = 60 s ngp = 510x405x40 = 8.262.000 fcst range = 132h cost = 2050 BU x run elapsed time = 48 min/run initial conditions: interpolated from EPS members merged with surface and soil-layer fields produced at DWD for COSMO-EU Present system x = 10 km z = 40 ML t = 90 s ngp = 306x258x40 = 3.157.920 fcst range = 132h cost = 570 BU x run elapsed time = 25 min/run initial conditions: interpolated from EPS members COSMO-LEPS_7 running in experimental mode since 24 May 2009 (no merging yet) A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
cleps_7 going operational in late Autumn COSMO-LEPS 10 km COSMO-LEPS 7 km • the domain of COSMO-LEPS_7 is slightly larger than in the present configuration and fully covers Central and Southern Europe (British Isles and Iberia Peninsula included). • the grid points are in the locations as COSMO-EU grid. A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
Observations: SYNOP reports over both MAP D-PHASE region (450 reports/day) and the FULL-DOMAIN (1400 reports/day). • Method: nearest grid point; no-weighted fcst. COSMO-LEPS_10 (oper) vs COSMO-LEPS_7 (exp) • Deterministic verification of T2M ensemble mean • Variable: 2-metre temperature. • Period: 3 months, from 24/5 to 24/8/2009. • Forecast ranges: fc+6h, fc+12h, …, fc+132h. • Scores: root-mean-square error, bias. • Probabilistic verification of 12-hour cumulated precipitation • Variable:12h cumulated precipitation (18-06, 06-18 UTC). • Period: 2 months, June and July 2009. • Forecast ranges: fc 6-18h, fc 18-30h, …, fc 114-126h. • Scores: ROC area, BSS, RPSS, Outliers. • Thresholds: 1, 5, 10, 15, 25, 50 mm/12h. A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
Bias and rmse of T2M Ensemble Mean • Consider bias (the closer to zero, the better) and rmse (the lower the better) for June 2009. • T2m forecasts are corrected with height. MAPDOM FULLDOM • For both models, slightly better scores over the FULL domain in comparison with the MAP D-PHASE domain. • Bias closer to zero and lower rmse for the 7-km suite. • Improvement is not “massive”, but detectable for all forecast ranges, especially for day-time verification. • The signal is stable (similar scores for 1-month or 3-month verification). A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
ROC area, BSS, RPSS, OUTL for 12-hour tp (MAPDOM) • Consider the event “5 mm of precipitation in 12 hours” for ROC area and BSS in June 2009. ROC area BSS % of outliers RPSS • Slight better results for the 7-km suite, but the gap is very small (and significant?); outliers not reduced. • The improvement is detectable for all forecast ranges. • Similar scores also July 2009 (less rainy) and for higher thresholds. A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
ROC area, BSS, RPSS, OUTL for 12-hour tp (FULLDOM) • Consider the event “5 mm of precipitation in 12 hours” for ROC area and BSS in June 2009. ROC area BSS % of outliers RPSS • “Larger” (but still small) impact when verification is performed over the full domain. • Need to consider also rainy seasons. • The improvement is not “massive”, but is detectable for all forecast ranges. A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
Products: high-priority parameters (tp, t2m, td2m, u10, v10, gust10, mslp, orog, lsm) operationally generated for each ensemble member from fc+0h to fc+132h every 3h; produced, in GRIB2 format, over a regular lat/lon grid (0.1x0.1); saved at ARPA-SIMC; will be archived at ECMWF. COSMO-LEPS for TIGGE-LAM rotated lat/lon domainregular lat/lon domain grib2 (TIGGE-LAM regular grid) grib1 (original rotated grid) A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
Multi-model clustering (data from TIGGE portal) Cluster members chosen as representative members (RMs) initial conditions by EPS Possible evolution scenarios initial conditions by MOGREPS thesis by Andrea Corigliano • Consider both ECMWF EPS and UKMO MOGREPS and study properties of cluster analysis on multi-ensemble: • How many times do the 2 ensembles mix? • Where do the RMs come from? How to they score depending on their “origin”? • BEFORE DOWNSCSALING: is there added value with respect to single-model ensemble? • AFTER DOWNSCALING: is there added value with respect to single-model ensemble? A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
Future plans • COSMO-LEPS_7km: • run the suite for ~ 40 more days and hope for larger impact !!!!!! • use the soil moisture analysis fields provided by DWD or the fields provided by COSMO run in hindcast mode (“proxy-run”), • implement it operationally by November. • COSMO-LEPS for TIGGE-LAM: • develop coding, post-processing and archiving of COSMO-LEPS output files in GRIB2 format; • assistance to users. • Multi-model clustering: continue the work. • Gaining from COSMO-SREPS experience, introduce more model perturbations. • Support calibration and verification. A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
Thank you ! A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
COSMO-LEPS (developed at ARPA-SIM) What is it? It is a Limited-area Ensemble Prediction System (LEPS), based on COSMO-model and implemented within COSMO (COnsortium for Small-scale MOdelling, which includes Germany, Greece, Italy, Poland, Romania, Switzerland). Why? It was developed to combine the advantages of global-model ensembles with the high-resolution details gained by the LAMs, so as to identify the possible occurrence of severe and localised weather events (heavy rainfall, strong winds, temperature anomalies, snowfall, …) generation of COSMO-LEPS to improve the Late-Short (48hr) to Early-Medium (132hr) range forecast of severe weather events. A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
Operational set-up Additional products: • 1 deterministic run (ICs and 3-hourly BCs from the high-resolution deterministic ECMWF forecast) to “join” deterministic and probabilistic approaches: start at 12UTC; t = 132h; • 1 hindcast (or proxy) run (ICs and 3-hourly BCs from ECMWF analyses) to “downscale” ECMWF information: start at 00UTC; t = 36h. Core products: 16 perturbed COSMO-model runs (ICs and 3-hourly BCs from 16 EPS members) to generate, “via weights”, probabilistic output: start at 12UTC; t = 132h; A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
but … A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
Score dependence on the domain size (1) • Verification of COSMO-LEPS against synop reports over the MAP D-PHASE area (~ 470 stations; MAPDOM) and the full domain (~ 1500 stations; fulldom): different statistics of the verification samples; up to now, performance of the system over the 2 domains assessed only for 6 months (March-August 2007). difficult to draw general conclusions A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
OUTL RPSS ROC Score dependence on the domain size (2) • RPSS score… the higher the better… (and positive). • ROC area… the higher the better… (and above 0.6). • Outliers percentage … the lower the better. • Smoother transitions from month to month in “fulldom” scores. • Slightly better performance of COSMO-LEPS over the MAPDOM, but the signal varies from month to month. • Higher predictability with orographic forcing? Need to check individual regions and/or to stratify for type of stations. A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
Bias and rmse of T2M Ensemble Mean • Consider bias (the closer to zero, the better) and rmse (the lower the better). • Bias closer to zero (0.5 °C of decrease) and lower rmse for the 7-km suite. • The improvement is not “massive”, but detectable for all forecast ranges, especially for day-time verification. • The signal is stable (similar scores for 1-month or 3-month verification). • Need to correct T2M forecasts with height to assess the impact more clearly. A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
Overestimation of Td2m and soil moisture (1) • Verification period: MAM07 and MAM08. • Obs: synop reports (about 470 stations x day). • Region: 43-50N, 2-18E (MAP D-PHASE area). • Larger bias and larger rmse in MAM08 rather than in MAM07 for COSMO-LEPS deterministic run (in 2007, no multi-layer soil model). A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
Overestimation of Td2m and soil moisture (2) from Andrè Walser • COSMO-LEPS members have higher soil moisture than COSMO-7 run at Meteoswiss (case of 20/7), especially in the Alps. • For the future, do not simply interpolate from ECMWF IFS to COSMO model with the multi-layer soil model. generate “home-made” soil moisture analysis (from COSMO-EU, from a proxy-run, …) A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
Semi-diurnal cycle in COSMO-LEPS scores • BSS score … the higher the better … • Performance of the system assessed for 5 different Summers (JJA). BSS • Evident 12-hour cycle in BSS scores (the same holds for RPSS, while less evident for ROC area scores). • Better performance of the system for “night-time” precipitation, that is for rainfall predicted between 18Z and 6Z (ranges 30-42h, 54-66h, …). • The amplitude of the cycle is somewhat reduced throughout the years and with increasing forecast range. • The bad performance in Summer 2006 is confirmed. A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
Feasibility study of COSMOLEPS at 7 km (cleps_7) • “Keep the pace” with deterministic model (x~ 2-3 km): if the gap in resolutions between deterministic and probabilistic systems is too large, the two systems go for different solutions (that is, they forecast different weather!). Motivations: Provide a more detailed description of mesoscale processes by incresing the horizontal resolution. Do not lose a “reasonable advantage” against ECMWF EPS, which will go to x=25 km during 2009. from 10 to 7 km (plus small domain extensions) does not seem a lot A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
COSMO-LEPS at 7 km (cleps_7): the answer to forecasters’ dream? New system x = 7 km z = 40 ML t = 72 s ngp = 510x405x40 = 8.262.000 fcst range = 132h cost = 1925 BU x run elapsed time = 138 min Present system x = 10 km z = 40 ML t = 90 s ngp = 306x258x40 = 3.157.920 fcst range = 132h cost = 640 BU x run elapsed time = 45 min … cleps_7 is about 3 times more expensive than the present configuration new computer at ECMWF being installed Computer resources for each ECMWF member state will increase by a factor of 5 (five) and …. A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
The dream is possible COSMO-LEPS 10 km COSMO-LEPS 7 km • the grid of cleps_7 would be almost identical to that of COSMO-EU, this making easier and cleaner the use of initial fields provided by DWD (e.g. soil moisture analysis). A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
Main results • COSMO-LEPS system runs on a daily basis since November 2002 (6 “failures” in almost 6 years of activity) and it has become a “member-state time-critical application” at ECMWF ( ECMWF operators involved in the suite monitoring). • COSMO-LEPS products used in EC Projects (e.g. PREVIEW), scientific experiments (e.g. COPS, MAP D-PHASE, EFAS) and met-ops rooms across COSMO community. Time series scores cannot easily disentangle improvements related to COSMO-LEPS itself from those due to better boundaries by ECMWF EPS. • Nevertheless, positive trends can be identified: • increase in ROC area scores and reduction in outliers percentages; • positive impact of increasing the population from 5 to 10 members (June 2004); • some deficiency in the skill of the system were identified after the system upgrades occurred on Feb 2006 (from 10 to 16 members; from 32 to 40 ML + EPS upgrade!!!), but scores are encouraging throughout 2007 and, to a certain extent, 2008. • 2 more features: • marked semi-diurnal cycle in COSMO-LEPS scores (better skill for “night-time” forecasts); • better scores over the Alpine area rather than over the full domain (to be confirmed). A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
COSMO-LEPS 16-MEMBER EPS 51-MEMBER EPS MAM06 Average values (boxes 0.5 x 0.5) tp > 1mm/24h tp > 5mm/24h • As regards AVERAGE precipitation above these two threshols, the 3 systems have similar performance. A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
ENSEMBLE SIZE REDUCTIONIMPACT EVALUATED ON CASE STUDIES (1) A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
2003082512 Friuli(+72-+96) 5 RMs 10 RMs All 51 A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009 20 mm 100 mm
ENSEMBLE SIZE REDUCTIONIMPACT EVALUATED ON CASE STUDIES (2) Observed precipitation between 15-11-2002 12UTC and 16-11-2002 12 UTC Piedmont case A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
2002111212 Piedmont 5 RMs 10 RMs All 51 A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009 20 mm 150 mm
COSMO-LEPSReal time products A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
Why Limited Area Ensemble Prediction? • Global Ensemble Prediction Systems • have become extremely important tools to tackle the problem of predictions beyond day 2 • are usually run at a coarser resolution with respect to deterministic global predictions → skill inforecasting intense and localised events is currently still limited. A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
Why Limited Area Ensemble Prediction? (2) As regards high resolution deterministic forecast in the short range, where limited-area models play the major role, a “satisfactory” QPF is still one of the major challenges. The same can be said for other local parameters. This is due, among other reasons, to the inherently low degree of predictability typical of severe and localised events. Probabilistic/Ensemble approach is so required also for the short range at higher resolution A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
From Global EPS to LAM EPS • In the Limited-area ensemble systems, tailored for the short range, perturbations must be already “active” during the first hours of integration • The characteristic of the LAM ensemble are strongly dependent by the lateral boundaries forcing. • Due to the “regional” application of these Limited Area Ensembles, methodologies can be different in different geographycal regions. A pratical consideration: Global EPS ~ Big Centres Limited Area EPS ~ (also) Relatively Small Centres A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
OBJECTIVE VERIFICATION OF COSMO-LEPS In the last period the verification package is being developed keeping into account two measure of precipitation: • the cumulative volume of water deployed over a specific region • the rainfall peaks which occur within this region COSMO observations A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
Verification grid OBS MASK CLEPS EPS A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
COSMO-LEPS 16-MEMBER EPS 51-MEMBER EPS 3 sis Average values boxes 0.5x0.5 deg tp > 1mm/24h tp > 5mm/24h A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
COSMO-LEPS 16-MEMBER EPS ave 0.5 NOCC=2671 tp > 1mm/24h NOCC=1195 tp > 5mm/24h NOCC=610 tp > 10mm/24h A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
COSMO-LEPS 16-MEMBER EPS 51-MEMBER EPS 3 sis Maximum values boxes 0.5x0.5 deg tp > 1mm/24h tp > 5mm/24h tp > 10mm/24h A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009
COSMO-LEPS 16-MEMBER EPS Average values boxes 0.5x0.5 deg NOCC=227 tp > 20mm/24h A.Montani; The COSMO-LEPS system: present status and outlook COSMO meeting, Offenbach, 7-11 September 2009