1 / 44

VERIFICATION Highligths by WG5

VERIFICATION Highligths by WG5. Outlook. Some focus on Temperature with common plots and Conditional Verification Some Fuzzy verification Long trends. 2. SON 2009. DJF 2009-2010. T2m: mean diurnal cycle (first 24h forecasts) domain Switzerland (hourly SYNOP‘s). Summer 2010.

kiefer
Download Presentation

VERIFICATION Highligths by WG5

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. VERIFICATIONHighligthsby WG5

  2. Outlook Some focus on Temperature with common plots and Conditional Verification Some Fuzzy verification Long trends 2

  3. SON 2009

  4. DJF 2009-2010

  5. T2m: mean diurnal cycle (first 24h forecasts)domain Switzerland (hourly SYNOP‘s) Summer 2010 Winter 2009/2010 OBS COSMO-7 COSMO-2 Autumn 2009 Spring 2010 P. Kaufmann, V. Stauch

  6. T2m COSMO-I7 00UTC: LAST YEAR

  7. WAM Conditional Verification Extracting information for relevant performance of weather parameters The input from modelers and forecasters is necessary for identifying and testing hypotheses. F. Gofa - HNMS WG5 COSMO General Meeting, Moscow 2010

  8. Temp in overcast conditions Fall Winter Spring Summer F. Gofa - HNMS

  9. Conditional VerificationTemp – TCC obs >=75% DJF SON Better behaviour for all the seasons Compare to no condition model MAM

  10. Temp in clear sky conditions Fall Winter Spring Summer F. Gofa - HNMS

  11. Conditional VerificationTemp – TCC obs <=35% DJF SON Worse behaviour for all the seasons Compare to no condition model MAM

  12. Temp in ‘calm’ conditions (<2 m/s) Fall Winter Spring Summer F. Gofa - HNMS WG5 COSMO General Meeting, Moscow 2010

  13. Temp in ‘high wind’ conditions >10m/s Fall Winter Spring Summer F. Gofa - HNMS WG5 COSMO General Meeting, Moscow 2010

  14. Some conclusion A problem with Temp is clear. RMSE between 2-3 °C it is not so small. Diurnal cycle too cold during the day and too warm during the night Clear different behaviour with conditions on TCC and with different wind conditions 14

  15. Outlook Some focus on Temperature with common plots and Conditional Verification Some Fuzzy verification Long trends 15

  16. Neighborhood verification for precipitation results for 2009 3h accumulated precipitation sums over the domain of the swiss radar composit models: COSMO-2 and COSMO-7 leadtimes 04 – 07h for all 8 daily forecast runs obervation precipitation estimates of the swiss radar composit in case of a missing value, the full date will not be evaluated(total of 28 days)

  17. good bad COSMO-7 better COSMO-2 better Neighborhood (fuzzy) verification 2009, FSS and UP T. Weusthoff Fractions Skill Score - = COSMO-2 - COSMO-7 COSMO-2 COSMO-7 Upscaling - =

  18. Neighborhood (fuzzy) verification: Spring 2010COSMO-2/COSMO-7: 3h acc, leadtime +4 to +6 for all models COSMO-2 COSMO-7 IFS Fractions Skill Score FSS Upscaling ETS Upscaling freq. bias FBI T. Weusthoff

  19. Outlook Some Common Plots (Task 6 Versus) Conditional Verification Some Fuzzy verification Long trends verification 19

  20. Introduction of new Z0

  21. Introduction of new Z0

  22. Introduction of new Z0

  23. Introduction of new Z0

  24. Valid time 00 UTC Total cloud cover _____ Cloud cover above 2 Octa (Cl.1) .......... Cloud cover above 6 Octa (Cl.2) Cloud cover of low clouds because incorporation of AWS

  25. Time series of the COSI: State May 2010 Stand Mai 2010

  26. Time series of the COSI: State May 2010 Stand Mai 2010

  27. Time series of the COSI: Temperature day 1

  28. Time series of the COSI: Temperature day 2 Stand Mai 2010

  29. Time series of the COSI: Temperature day 3 Stand Mai 2010

  30. Time series of the COSI: State May 2010 (STDV used for T2m instead of RMSE) Stand Mai 2010

  31. Long period verification (seasonal trend) (from djf’04 to mam’10) • Some Statistical indices for low thres (0.2mm/24h) • Some Statistical indices for high thres (20mm/24h) • Verification ovest last year (DJF 2009-MAM2010) • Driving model comparison: ecmwf/Cosmo-I7/Cosmo-I2 • Driving model comparison: ecmwf/Cosmo-ME/Cosmo-IT

  32. Seasonal trend - low thresholds • All the versions present a seasonal cycle with an overestimation during summertime (except COSMO-7 and I2) • COSMO-7 and I2 underestimate • Overestimation error decreases in D+2 (spin-up effect vanished) QPF verification of the 4 model versions at 7 km res. (COSMO-I7, COSMO-7, COSMO-EU, COSMO-ME) with the 2 model versions at 2.8 km res. (COSMO-I2, COSMO-IT) Dataset: high resolution network of rain gauges coming from COSMO dataset and Civil Protection Department  1300 stations Method: 24h/6h averaged cumulated precipitation value over 90 meteo-hydrological basins

  33. Seasonal trend - low thresholds • Very light improvement in trend • Seasonal error cycle: lower ets during winter and summertime • no significant differences between D+1 and D+2 • Last winter (very snowy particularly in Northern Italy): low ets value (D+1 and D+2) model error or lack of representativeness of the rain gauges over the plain during snowfall ?

  34. Driving model comparison: ECMWF/COSMO-ME/COSMO-IT, low thresholds • ECMWF tendency to forecast low rainfall amounts big overestimation, big false alarms, very low ets, quite good pod • Better prediction for COSMO-models (no strong differences between ME and IT) • Seasons DJF2009 – MAM2010

  35. Driving model comparison: ECMWF/COSMO-I7/COSMO-I2, low thresholds • ECMWF tendency to forecast low rainfall amounts big overestimation, big false alarms, very low ets, quite good pod • Better prediction for COSMO-models BUT bad performance during summertime • Seasons DJF2009 – MAM2010

  36. Precipitation (12h-sums +12 to +24h):Spring 2010 over Switzerland (SYNOP‘s)COSMO-7 & COSMO-2 V. Stauch

  37. Precipitation (12h-sums +12 to +24h):Spring 2010 over Switzerland (SYNOP‘s)COSMO-7 & IFS V. Stauch

  38. Seasonal trend - high thresholds • Slight bias reduction during latest seasons • Last winter: all the versions overestimate (probably due to lack of representativeness of the rain gauges over the plain during snowfall) • Strong COSMO-7 underestimation BUT slight improvement during latest seasons

  39. Seasonal trend - high thresholds • Low values during summertime • In general, quite stationary error since son2008 up to now • All the versions present a jump around son2008: ets increases from 0.2-0.4 up to 0.3-0.5 • Skill decreases with forecast time

  40. Driving model comparison: ECMWF/COSMO-ME/COSMO-IT, high thresholds • ECMWF difficulty to forecast high rainfall amounts bias around 1 BUT big false alarms, very low ets and pod • Better prediction for COSMO-models • Seasons DJF2009 – MAM2010

  41. Driving model comparison: ECMWF/COSMO-I7/COSMO-I2, high thresholds • ECMWF difficulty to forecast high rainfall amounts bias around 1 BUT big false alarms, very low ets and pod • Better prediction for COSMO-models • Seasons DJF2009 – MAM2010

  42. 12h Precipitation – Sep2009-Aug2010 ECMWF COSMO Really strong overestimation of lower preci amounts up to 3mm and lower ETS scores for ECMWF F. Gofa - HNMS WG5 COSMO General Meeting, Moscow 2010

  43. Some conclusion Long term trends show a general (sometimes light) improvements for all the considered models Comparison between COSMO models and IFS shows a general clear better behaviour for COSMO implementations 44

More Related