720 likes | 728 Views
This presentation discusses the present status of COSMO-LEPS and COSMO-S14-EPS, including operational verification, calibrated precipitation, convection schemes, clustering technique, and future plans.
E N D
COSMO-LEPS and COSMO-S14-EPS:what is old, what is new.Andrea Montani,C. Marsigli, T. PaccagnellaARPA-SIMCHydroMeteoClimate Regional Service of Emilia-Romagna, Bologna, Italy COSMO General meeting Sibiu, 2-5 September 2013 A.Montani; The COSMO-LEPS system.
Outline • Present status of COSMO-LEPS: • about the operational verification, • about the calibrated precipitation, • about the convection schemes, • about the clustering technique, • about the future plans. • Present status of COSMO-S14-EPS: • about the main features, • about case-study assessment, • about the plans. A.Montani; The COSMO-LEPS system.
COSMO-LEPS suite @ ECMWF: present status 16 Representative Members driving the 16 COSMO-model integrations (weighted according to the cluster populations) using either Tiedtke or Kain-Fristch convection scheme (members 1-8 T, members 9-16 KF) + perturbations in turbulence scheme and in physical parameterisations 3 levels 500 700 850 hPa 4 variables Z U V Q d+3 d+4 d d+5 d+1 d+2 d-1 Cluster Analysis and RM identification Cluster Analysis and RM identification older EPS 00 2 time steps younger EPS 12 European area clustering period Complete Linkage • suite runs twice a day (00 and 12UTC) as a “time-critical application” managed by ARPA-SIMC; • Δx ~ 7 km; 40 ML; fc+132h; • COSM0 v4.26 since Jan 2013; • computer time (50 million BUs for 2013) provided by the ECMWF member states in COSMO. COSMO-LEPS Integration Domain COSMO-LEPS clustering area A.Montani; The COSMO-LEPS system.
16 January 2013: COSMO upgrade: 4.21 4.26; • int2lm upgrade: 1.18 1.20. • 17 January 2013: operational dissemination implemented for ARPA-Veneto. • 22 January 2013, technical changes at ECMWF: • change of ECMWF super-computer and of the user running the suite: itm zcl; • introduction of a new “dissemination stream” for COSMO-LEPS: “ad-hoc” initial and boundary conditions do not have to be retrieved any more, but are prepared on a dedicated file system; product dissemination starts about 40 minutes earlier than before (at 9UTC and 21UTC). • 7 May 2013: enriched test dissemination implemented for HNMS. • 13 May 2013: in the framework of GEOWOW research project, COSMO-LEPS was the first system (!!!) to populate TIGGE-LAM archive at ECMWF (high-priority parameters in grib2 format). • 25 June 2013: tests with Fieldextra 11.1.0 started. Main changes during the COSMO year A.Montani; The COSMO-LEPS system.
Outline • Present status of COSMO-LEPS: • about the operational verification, A.Montani; The COSMO-LEPS system.
SYNOP on the GTS Time-series verification of COSMO-LEPS Main features: variable: 12h cumulated precip (18-06, 06-18 UTC); period : from Dec 2002 to Jul 2013; region: 43-50N, 2-18E (MAP D-PHASE area); method: nearest grid point; no-weighted fcst; obs: synop reports (about 470 stations/day); fcst ranges: 6-18h, 18-30h, …, 102-114h, 114-126h; thresholds: 1, 5, 10, 15,25, 50mm/12h; system: COSMO-LEPS; scores: ROC area, BSS, RPSS, Outliers, … both monthly and seasonal scores were computed A.Montani; The COSMO-LEPS system.
Time series of ROC area (6-month running mean) • Area under the curve in the HIT rate vs FAR diagram; the higher, the better … • Valuable forecast systems have ROC area values > 0.6. • Highest scores in the 2nd part of 2011 and, for the highest threshold, in 2013. • Drier seasons during 2011 and 2012 with few heavy-precipitation events: limited significance of the results for the 15mm threshold. • fc 30-42h: ROC area is high for last winter and spring. Positive trend can be noticed. • fc 78-90h: the best scores date back to the end of 2011. • Limited loss of predictability with increasing forecast range. A.Montani; The COSMO-LEPS system.
Outline • Present status of COSMO-LEPS: • about operational verification (time-series scores show improvements), • about the calibrated precipitation; A.Montani; The COSMO-LEPS system.
about calibrated precipitation • For each COSMO-LEPS member, calibrated precipitation is operationally generated over Germany, Switzerland and Emilia-Romagna; the calibration technique is based on CDF-based corrections, making use of COSMO-LEPS reforecast. • For MAM2013, inter-comparison between raw and calibrated 24h TP forecast. Main features: variable: 24h cumulated precip (06-06 UTC); period : DJF 2012-13 and MAM 2013; region: Germany, Switzerland, Emilia-Romagna; method: nearest grid point; no-weighted fcst; obs: synop reports (about 300 stations/day); fcst ranges: 18-42h, 42-66h, 66-90h, 90-114h; thresholds: 1, 5, 10, 15,25, 50mm/12h; system: opecleps and Calibcleps; scores: ROC area, BSS, RPSS, Outliers, RelDiag, … A.Montani; The COSMO-LEPS system.
opecleps vs Calibcleps fc 42-66h; 10mm/24h A.Montani; The COSMO-LEPS system.
Outline • Present status of COSMO-LEPS: • about operational verification (time-series scores show improvements), • about calibration (positive impact, especially over Emilia-Romagna!); • about convection schemes, • members 1-8 use Tiedtke convection scheme (8TD), • members 9-16 use Kain-Fritsch (8KF). • MAM 2013: compare cleps16, 8TD, 8KF. A.Montani; The COSMO-LEPS system.
about the convection scheme ___ cleps16 ___ 8TD ___ 8KF BSS, tp > 10mm BSS, tp > 1mm • As expected, best performance by the full ensemble (cleps16). • Tiedtke-members better than Kain-Fritsch members, but NOT for all scores. ROC, tp > 1mm ROC, tp > 10mm A.Montani; The COSMO-LEPS system.
Outline • Present status of COSMO-LEPS: • about operational verification (time-series scores show improvements); • about calibration (positive impact, especially over Emilia-Romagna!); • about convection schemes (Tiedtke slightly superior to Kain-Fritsch); • about the clustering technique A.Montani; The COSMO-LEPS system.
about the clustering technique AIM: provide limited-area ensembles (either convection-parameterised or convection-permitting) with the best set of boundary conditions. • Consider distances between ECMWF EPS members according to “COSMO-LEPS metric” (Z, U, V, Q in the mid-lower troposphere over the clustering domain). • Look at distances between pairs of ECMWF EPS members: to what extent do these distances grow with forecast range, using “COSMO-LEPS metric”? • Study a number of seasons. • Compare against random choice. • Outcome: modifications to the number of clusters / number of EPS considered / clustering intervals. A.Montani; The COSMO-LEPS system.
Outline • Present status of COSMO-LEPS: • about operational verification (time-series scores show improvements), • about calibration (clear positive impact of calibration); • about convection schemes (Tiedtke slightly superior to Kain-Fritsch); • about the clustering technique (work in progress); • about the future plans. A.Montani; The COSMO-LEPS system.
Adapt COSMO-LEPS suite to ECWMF forthcoming upgrades: increase of vertical resolution in ECMWF-EPS: 62 91; change of Member-State server: ecaccess ecgb; change of super-computer: IBM Cray; Carry on study about the clustering methodology. Increase of COSMO-LEPS vertical resolution (40 50ML): tests start in October. LAMEPS_BC project: test with high-resolution ECMWF-EPS boundaries start by the end of 2013. Analysis of the performance of COSMO-HYBEPS (COSMO-LEPS + 2-3 COSMO runs nested on IFS/GME/GFS). Strengthen links with Fieldextra. about the future plans Any request for modifications to the present configuration of COSMO-LEPS? A.Montani; The COSMO-LEPS system.
Outline • Present status of COSMO-S14-EPS: • about the main features, • about case-study assessment, • about the plans. A.Montani; The COSMO-LEPS system.
Milestones of COSMO-S14-EPS • 11/3/2011. Nothing present. • 2/5/2011. Submission of a new ECMWF Special Project (Title: “Implementation of a limited-area ensemble prediction system for Sochi Olympic Games”; Project investigators: Majewski, Montani, Steiner; duration: 3 years) for provision of computer time to run the system on ECMWF super-computers; • 5-9/9/2011. Discussion during the COSMO meeting about the system set-up; • 6/12/2011. Approval of the Special Project by ECMWF Council; • 6-9/12/2011. Visit of Russian colleagues at ARPA-SIMC to define specifics of the new ensemble system; • 19/12/2011. Beginning of provision of COSMO-S14-EPS products on a daily basis. • 1/11/2012. Initial conditions of soil fields (temperature, moisture, snow cover) are no more interpolated from ECMWF EPS, but provided by COSMO run in hindcast mode (approach already tested for COSMO-LEPS in 2011) . A.Montani; The COSMO-LEPS system.
COSMO-S14-EPS @ ECMWF: present status 10 Representative Members driving the 10 COSMO-model integrations (weighted according to the cluster populations) employing either Tiedtke or Kain-Fristch convection scheme (randomly choosen) + perturbations in turbulence scheme and in physical parameterisations 4 variables Z U V Q 3 levels 500 700 850 hPa d d+1 d+2 d+3 Cluster Analysis and RM identification Cluster Analysis and RM identification ECMWF EPS 2 time steps clustering interval Black-Sea area Complete Linkage clustering area • Δx ~ 7 km; 40 ML; fc+72h; • initial time: 00/12 UTC; • computer time (~ 4.5 million BUs for 2013) is provided by an ECMWF Special Project; • contributions from Switzerland national allocation were needed; • suite managed by ARPA-SIMC. Integration Domain A.Montani; The COSMO-LEPS system.
Disseminated products • individual ensemble member runs (ICs and BCs from 10 selected EPS members): start at 00UTC and 12UTC; t = 72h; • 1 deterministic run (ICs and BCs from the deterministic ECMWF forecast) to “join” deterministic and probabilistic approaches: start at 00UTC and 12UTC; t = 72h; post-processing uses COSMO-software fieldextra: probability fields for the exceedance of thresholds for surface fields; ensemble mean and ensemble standard deviation for some fields; • provision of hourly boundary conditions (from fc+0h to fc+48h) for convective-resolving ensemble (RDP part); • provision of hourly boundary conditions (from fc+0h to fc+48h) for higher-resolution deterministic modelling (RDP part). A.Montani; The COSMO-LEPS system.
Outline • Present status of COSMO-S14-EPS: • about the main features, • about case-study assessment, A.Montani; The COSMO-LEPS system.
Case-study assessment • Heavy-precipitation event on 13 Jan 2013: 21 mm of rain during the day on the coast (Sochi/Adler) and 33 mm of snow-water equivalent during the day in the mountain (KrasnayaPolyana); • Foehn event on 14-15 Feb 2013: a sudden 10-degree warming, which forecasters regard as hardly predictable. • Sochi local time = UTC + 4 hours • UTC= Sochi local time – 4 hours The fields of the next slides are operationally delivered to FROST-server A.Montani; The COSMO-LEPS system.
Heavy precipitation event (1) runs start at 00UTC of 11/1/2013 and verify at 12UTC of 13/1/2013 (fc+48-60h) • OBS: 21 mm of rain during the day on the coast (Sochi/Adler) and 33 mm of snow-water equivalent during the day in the mountain (Krasnaya Polyana) tpp12h_gt_20mm sfp12h_gt_15cm rain12h_gt_20mm A.Montani; The COSMO-LEPS system.
Heavy precipitation event (2) runs start at 12UTC of 11/1/2013 and verify at 12UTC of 13/1/2013 (fc+36-48h) • OBS: 21 mm of rain during the day on the coast (Sochi/Adler) and 33 mm of snow-water equivalent during the day in the mountain (Krasnaya Polyana) tpp12h_gt_20mm rain12h_gt_20mm sfp12h_gt_15cm A.Montani; The COSMO-LEPS system.
Heavy precipitation event (3) runs start at 00UTC of 12/1/2013 and verify at 12UTC of 13/1/2013 (fc+24-36h) • OBS: 21 mm of rain during the day on the coast (Sochi/Adler) and 33 mm of snow-water equivalent during the day in the mountain (Krasnaya Polyana) tpp12h_gt_20mm rain12h_gt_20mm sfp12h_gt_15cm A.Montani; The COSMO-LEPS system.
Heavy precipitation event: consistency Fix event (probability of 12h snowfall exceeding 15 mm of equivalent water) and verification time (12UTC of 13/1/2013): consider model runs with different forecast ranges. • OBS: 21 mm of rain during the day on the coast (Sochi/Adler) and 33 mm of snow-water equivalent during the day in the mountain (Krasnaya Polyana) fc +48-60h fc +24-36h fc +36-48h A.Montani; The COSMO-LEPS system.
Outline • Present status of COSMO-S14-EPS: • about the main features, • about case-study assessment (good performance of the system), • about the plans. A.Montani; The COSMO-LEPS system.
“Survive” ECMWF upgrades (increase of vertical resolution, change of member-state server, change of super-computer). Develop new products “on demand” (in the next weeks, “multi-model” ensemble products will be tested). Use good-quality observations at high resolution (and there are) to perform statistical verification. about the plans • To what extent are forecasters using COSMO-S14-EPS products? • How to strengthen links between forecasters and ensemble developers? training with forecasters in Sochi next October A.Montani; The COSMO-LEPS system.
Thank you ! European Conference on Applications of Meteorology / EMS annual meeting 09 – 13 September 2013, Reading (UK) Session NWP4 (on 13 September): Probabilistic and ensemble forecasting at short and and medium-range http://www.ems2013.net/home.html A.Montani; The COSMO-LEPS system.
Extra slides on configuration A.Montani; The COSMO-LEPS system.
Dim 2 Possible evolution scenarios Cluster members chosen as representative members (RMs) Initial conditions Dim 1 LAM scenario Dim 2 LAM scenario LAM integrations driven by RMs LAM scenario Dim 1 Initial conditions COSMO-LEPS methodology ensemble size reduction A.Montani; The COSMO-LEPS system.
COSMO-HYBrid Ensemble Prediction System From the results of CONSENS PP, come to a synthesis with the different ensemble systems / strategies, considering scientific, implementation, solidity aspects. Generate 20-member hybrid ensemble (COSMO-HYBEPS) , where: 16 members comes from COSMO-LEPS, 1 member is nested on IFS (uses Tiedtke scheme), 1 member is nested on IFS (uses Kain-Fritsch scheme), 1 member is nested on GME, 1 member is nested on GFS. already existingtaken from CONSENS. All members have Δx ~ 7 km; 40 ML; fc+132h; Study performance of different members’ combinations with the same ensemble size. “20-members esuite” implemented on 7/9/2012; will be run up to the end of the year A.Montani; The COSMO-LEPS system.
COSMO-LEPS (developed at ARPA-SIM) What is it? It is a Limited-area Ensemble Prediction System (LEPS), based on COSMO-model and implemented within COSMO (COnsortium for Small-scale MOdelling, which includes Germany, Greece, Italy, Poland, Romania, Switzerland). Why? It was developed to combine the advantages of global-model ensembles with the high-resolution details gained by the LAMs, so as to identify the possible occurrence of severe and localised weather events (heavy rainfall, strong winds, temperature anomalies, snowfall, …) generation of COSMO-LEPS to improve the Late-Short (48hr) to Early-Medium (132hr) range forecast of severe weather events. A.Montani; The COSMO-LEPS system.
Operational set-up Additional products: • 1 deterministic run (ICs and 3-hourly BCs from the high-resolution deterministic ECMWF forecast) to “join” deterministic and probabilistic approaches: start at 12UTC; t = 132h; • 1 hindcast (or proxy) run (ICs and 3-hourly BCs from ECMWF analyses) to “downscale” ECMWF information: start at 00UTC; t = 36h. Core products: 16 perturbed COSMO-model runs (ICs and 3-hourly BCs from 16 EPS members) to generate, “via weights”, probabilistic output: start at 12UTC; t = 132h; A.Montani; The COSMO-LEPS system.
Types of perturbations • As for types and values, the results from CSPERT experimentation were followed (* denotes default values for COSMO v4.26 ): • convection_scheme: Tiedtke* (members 1-8), Kain-Fritsch (members 9-16), • tur_len (either 150, or 500*, or 1000), • pat_len (either 500*, or 2000), • crsmin (either 50, or 150*, or 200), • rat_sea (either 1, or 20*, or 40), • rlam_heat (either 0.1, or 1*, or 5), • mu_rain : either 0.5* (with rain_n0_factor =0.1) or 0 (with rain_n0_factor =1.0), • cloud_num (either 5x10^8* or 5x10^7). A.Montani; The COSMO-LEPS system.
convection scheme: T=Tiedtke KF=Kain-Fritsch; • tur_len: maximal turbulent length scale (default 500m); this parameter is used mainly in the calculation of the characteristic length scale for vertical mixing and thus into the calculation of the vertical transport momentum coefficient; • pat_len: length scale of thermal surface patterns (default 500m); this parameter is mainly used in the calculation of the large-scale part of the equation addressing the heat flux parameterisation; horizontal length; • rlam_heat: scaling factor of the laminar layer depth (default 1); it defines the layer with non-turbulent characteristics (molecular diffusion effects only); • rat_sea: ratio of laminar scaling factors for heat over sea (default 20); • crsmin: minimal stomata resistance (default 150); • Cloud_num: Cloud droplet number concentration; • Mu_rain: Exponent of the raindrop size distribution; • ( gscp: Switch on/off of the graupel scheme).
Main results • Time-series verification • ECMWF EPS changed substantially in the last years (more and more weight to EDA-based perturbations) and it is hard to disentangle improvements related to COSMO-LEPS upgrades from those due to better boundaries; nevertheless: • high values of BSS and ROC area for the probabilistic prediction of 12-h precipitation for autumn 2011; • poor performance in the first months of 2012, then recovery. Need to investigate what happened. Case-study verification Consistent signal for different forecast ranges of a high-impact weather event for the snowfalls of February 2012. A.Montani; The COSMO-LEPS system.
Extra slides on COSMO-S14-EPS A.Montani; The COSMO-LEPS system.
Important ingredients (from 1st and 2nd FROST meetings) • Provide reasonable “numbers”. addressed • Develop experience with probabilities. ? • Feedback on the top-priority products. being addressed • Snow analysis. ? • Soil-field initialisation. addressed • High-res obs to assess the quality of the system. being addressed • Computer time. addressed • Timeliness in product delivery. addressed • ...... anything to add/remove? A.Montani; The COSMO-LEPS system.
FROST-2014 vs SOCHMEL • Introduction to FROST-2014: • What is it? • What has to do with COSMO? • COSMO ensemble activities within FROST-2014: • introduction to SOCHMEL (the SOCHi-targeted Mesoscale EnsembLe system) • methodology; • phases of development; • planned activity. • Final remarks. A.Montani; The COSMO-LEPS system.
Extra slides on verification A.Montani; The COSMO-LEPS system.
Seasonal scores of ROC and BSS: last 4 springs • Fixed event (“12h precip > 10mm”): consider the performance of the system for increasing forecast ranges. • Valuable forecast systems have ROC area values > 0.6 and BSS > 0. • Need to take into account the different statistics for each season (MAM 2011 was the driest). • Best performance for the spring 2011 and 2013, but less marked diurnal cycle in 2013. • Spring 2013: BSS is positive for all forecast ranges • Similar results for the other thresholds (not shown). A.Montani; The COSMO-LEPS system.
Outliers: time series + ………seas scores (DJF)? • How many times the analysis is out of the forecast interval spanned by the ensemble members. • … the lower the better … • Performance of the system assessed as time series and for the last 4 winters. • Evident seasonal cycle (more outliers in winter). • Overall reduction of outliers in the years up to 2007; then, again in 2009 and 2010, but later. • Need to take into account the different statistics for each season. • In the short range, best results for winter 2010-2011. • For longer ranges, the performance of the system is “stable”. • Outliers before 10% from day 3 onwards. A.Montani; The COSMO-LEPS system.
Time series of Brier Skill Score • BSS is written as 1-BS/BSref. Sample climate is the reference system. Useful forecast systems if BSS > 0. • BS measures the mean squared difference between forecast and observation in probability space. • BS equivalent to MSE for deterministic forecast. DA FINIREEEEEEEEEEEEEEEEEE Month-to-month variability is higher than for the ROC area. • fc 30-42h:very good scores in 2010 and 2011; BSS positive for all thresholds since April 2009; fewer and fewer problems with high thresholds. • fc 78-90h: good trend in 2010 and 2011 for all thresholds. • In the last months, “spread” in BSS values for the different threshold values, possibly due to the lack of events. A.Montani; The COSMO-LEPS system.
Seasonal scores of BSS: ……last 4 winters • Fixed event (“12h precip > 10mm”): consider the performance of the system for increasing forecast ranges. • Fixed forecast range (fc 30-42h): consider the performance of the system for increasing thresholds. • Need to take into account the different statistics for each season (last DJF was the driest). • Fixed event: best performance for the last two winters (ECMWF EPS had a record performance for winter 2009-2010): BSS positive for all forecast ranges. • Fixed forecast range: similar results as before. • Similar results for longer forecast ranges and for higher thresholds. A.Montani; The COSMO-LEPS system.
Ranked Probability Skill Score: time series + …….. seasonal scores (MAM) • A sort of BSS “cumulated” over all thresholds. RPSS is written as 1-RPS/RPSref. Sample climate is the reference system. RPS is the extension of the Brier Score to the multi-event situation. • Useful forecast systems for RPSS > 0. • Performance of the system assessed as time series and for the last 4 springs (MAM). • the increase of the COSMO-LEPS skill is detectable for 3 out of 4 forecast ranges along the years, BUT • low skill in the first months of 2012 (the problem comes from MAM), then recovery. • Best results for MAM 2011; quick decrease of RPSS with fcst range for MAM 2012. A.Montani; The COSMO-LEPS system.
Bias and rmse of T2M Ensemble Mean • Consider bias (the closer to zero, the better) and rmse (the lower the better). • Bias closer to zero (0.5 °C of decrease) and lower rmse for the 7-km suite. • The improvement is not “massive”, but detectable for all forecast ranges, especially for day-time verification. • The signal is stable (similar scores for 1-month or 3-month verification). • Need to correct T2M forecasts with height to assess the impact more clearly. A.Montani; The COSMO-LEPS system.
Overestimation of Td2m and soil moisture (1) • Verification period: MAM07 and MAM08. • Obs: synop reports (about 470 stations x day). • Region: 43-50N, 2-18E (MAP D-PHASE area). • Larger bias and larger rmse in MAM08 rather than in MAM07 for COSMO-LEPS deterministic run (in 2007, no multi-layer soil model). A.Montani; The COSMO-LEPS system.
Score dependence on the domain size (1) • Verification of COSMO-LEPS against synop reports over the MAP D-PHASE area (~ 470 stations; MAPDOM) and the full domain (~ 1500 stations; fulldom): different statistics of the verification samples; up to now, performance of the system over the 2 domains assessed only for 6 months (March-August 2007). difficult to draw general conclusions A.Montani; The COSMO-LEPS system.
OUTL RPSS ROC Score dependence on the domain size (2) • RPSS score… the higher the better… (and positive). • ROC area… the higher the better… (and above 0.6). • Outliers percentage … the lower the better. • Smoother transitions from month to month in “fulldom” scores. • Slightly better performance of COSMO-LEPS over the MAPDOM, but the signal varies from month to month. • Higher predictability with orographic forcing? Need to check individual regions and/or to stratify for type of stations. A.Montani; The COSMO-LEPS system.