1 / 44

COSMO General Meeting 2011 WG5 Parallel Session 5 September 2011

Verification results at MeteoSwiss in the year 2011 Francis Schubiger, Vanessa Stauch, Tanja Weusthoff, Pirmin Kaufmann MeteoSwiss. COSMO General Meeting 2011 WG5 Parallel Session 5 September 2011. Content . results of the operational verification in 2011

jon
Download Presentation

COSMO General Meeting 2011 WG5 Parallel Session 5 September 2011

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Verification resultsat MeteoSwiss in the year 2011Francis Schubiger, Vanessa Stauch,Tanja Weusthoff, Pirmin KaufmannMeteoSwiss COSMO General Meeting 2011 WG5 Parallel Session 5 September 2011

  2. Content • results of the operational verification in 2011 • separate verification of all eight daily COSMO-2 forecasts • neighborhood verification (also weather type dependant) • COSMO-2 03 UTC forecasts up to +48h • extension of Beth Ebert‘s package: -> „fuzzy in time“

  3. Verification of COSMO-2, COSMO-7, COSMO-LEPS and IFS • Operational (COSMO-2, COSMO-7, COSMO-LEPS [mean], IFS): • Surface (3h-steps SYNOP and 1h-steps SMN [Swiss Met Net]) • psred, ps, T2m, Td2m, 10m-wind, cloud cover, gusts,precipitation (1h, 12h), global radiation • Upper-air (TEMP) • T, RH, wind, F • Quasi-operational (COSMO-2, COSMO-7, IFS): • precipitation with RADAR (neighborhood/fuzzy verification, Ebert’s package) also weather-type dependant • Monitoring: SMN (swiss SYNOPs) , RADAR, windprofilers • Ongoing in experimental modus: • windprofiler over Switzerland • fluxes at Payerne • New / Next developments: • VAD (locations of swiss radars) => All these packages should be replaced by VERSUS 2

  4. T2m: mean diurnal cycle (first 24h forecasts)domain Switzerland (hourly SYNOP‘s) Autumn 2010 Spring 2011 OBS COSMO-7 COSMO-2 Winter 2010/2011 Summer 2011 V. Stauch, P. Kaufmann

  5. T2m: mean diurnal cycle Spring 2011COSMO-7 vs IFSSYNOP‘s over COSMO-7 domain OBS COSMO-7 IFS V. Stauch, P. Kaufmann

  6. T2m: mean diurnal cycle Spring 2011COSMO-7 vs COSMO-LEPSSYNOP‘s over COSMO-7 domain OBS COSMO-7 COSMO-LEPS V. Stauch, P. Kaufmann

  7. Global radiation (inclined): Spring 2011COSMO-7 mean diurnal cycleSYNOP‘s over Switzerland V. Stauch, P. Kaufmann

  8. Global radiation (horizontal): Spring 2011COSMO-7 mean diurnal cycleSYNOP‘s over Switzerland V. Stauch, P. Kaufmann

  9. Precipitation: Spring 2011geographical distribution precipitation12h sums frequency bias(1 mm/12h) Spring 2011 COSMO-7+24 & +48h V. Stauch, P. Kaufmann

  10. Precipitation (12h-sums +12 to +24h):Spring 2011 over Switzerland (SYNOP‘s)frequency bias: COSMO-7 & COSMO-2 observed frequency V. Stauch

  11. Precipitation (12h-sums +36 to +48h):Spring 2011 over Switzerland (SYNOP‘s)frequency bias: COSMO-7 & IFS observed frequency V. Stauch

  12. Precipitation (12h-sums +12 to +24h):Spring 2011 over Switzerland (SYNOP‘s)COSMO-7 & COSMO-2 for both models mean over 9 gridpoints for each station V. Stauch

  13. Precipitation (12h-sums):Spring 2011 over Switzerland (SYNOP‘s)COSMO-7 & IFS V. Stauch

  14. COSMO-2 COSMO-7 August 2010 COSMO-7 COSMO-2 SMN RADAR fcst: +12..+24h fcst: +3..+6h 030915 21 00 Hourly precipitation: diurnal cycle Swiss SYNOPs (1h acc) CH-radarcomposit (3h acc) SMN COSMO-7 COSMO-7 July 2010 COSMO-2 COSMO-2 RADAR fcst: +12..+24h fcst: +3..+6h 030915 21 00 V. Stauch

  15. Verification of surface weather parameters • 2m-temperature: • cold bias in winter and warm bias in summer • positive bias more pronounced during nighttime • COSMO-2 ~0.5 K warmer (higher values) than COSMO-7 • 2m-dewpoint: • negative bias (~1 K) in the summer and autumn period • 10m-windspeed: • negative bias along the coast and on mountains, positive bias inland • total cloudiness: • mean daily cycle not well represented (mainly overestimation during night and underestimation during day) • precipitation: • overestimation (except in Summer ~ no bias) most pronounced in Winter (over COSMO-2 domain: 30-35% ; over Swizerland ~45%) • higher amounts in COSMO-2 / higher amounts over Alpine area • low amounts (0.1 mm/12h): overestimated in Winter (over the Alps all seasons) • high amounts (10 mm/12h): underestimated in COSMO-7 (over full domain by ~30% in summer, 20% in winter)

  16. TEMPS verification: Windspeed +24hWinter 10/11 all TEMPs COSMO-2 domain COSMO-7 COSMO-2

  17. Verification with vertical profiles: main results • temperature: cold bias (~ 0.5 K) in winter and warm bias (~ 0.5 K) in summer from ground up to 300 hPa • windspeed: positive bias in PBL (up to +1 m/s in winter) and slight negative bias above 400 hPa

  18. Separate verification of the eight daily COSMO-2 forecasts questions of interest: which is the best forecast for the afternoon/evening convection ? – is it the latest forecast or perhaps a forecast with a longer „spin up“ time ? are there significant differences between the eight forecasts ? period:COSMO-2 forecasts from 1 May to 31 July 2011 Verification with hourly obs over Switzerland

  19. COSMO-2: precipitation diurnal cycleforecasts of 00, 03, 06 and 09 UTC Vanessa Stauch

  20. COSMO-2: precipitation diurnal cycleforecasts of 12, 15, 18 and 21 UTC Vanessa Stauch

  21. COSMO-2: precipitation diurnal cycleall eight daily forecasts Vanessa Stauch

  22. COSMO-2: 2m-temperature diurnal cycleall eight daily forecasts Vanessa Stauch

  23. COSMO-2: 2m-temperature biasall eight daily forecasts Vanessa Stauch

  24. COSMO-2: 10m-wind speed diurnal cycleall eight daily forecasts Vanessa Stauch

  25. Neighborhood verification for precipitation results for 2010 3h accumulated precipitation sums over the domain of the swiss radar composit models: COSMO-2 and COSMO-7 for all 8 daily forecast runs, precipitation sums from +3 to +6h observation precipitation estimates of the swiss radar composit in case of a missing value, the full date will not be evaluated

  26. good bad COSMO-7 better COSMO-2 better Neighbourhood Verification, January–December 2010 3h sums (+3 ..+6h) Fractions Skill Score (top) and Upscaling (bottom) Fractions Skill Score - = COSMO-2 - COSMO-7 COSMO-2 COSMO-7 Upscaling - = T. Weusthoff

  27. COSMO-2 better COSMO-7 better FSS, COSMO-2 minus COSMO-7, 2010 Winter Spring numbers = FSS-Score of COSMO-2 colours = differences COSMO-2 minus COSMO-7 • COSMO-2: similar skill in all seasons • COSMO-2 better than COSMO-7 on almost all scales • COSMO-2 better than COSMO-7 especially in Winter and Summer Autumn Summer Tanja Weusthoff

  28. differences in Fractions Skill Score for weather-type dependant verif COSMO-2 minus COSMO-7 COSMO-7 better COSMO-2 better YEAR 2010 NE (11x) S (10x) F (78x) SW (49x) N (18x) H (73x) E (4x) NW (38x) W (56x) SE (4x) L (25x)

  29. Summary neighbourhood Verifikation precipitation in 2010 • COSMO-2 better than COSMO-7 on all scales,differences become less with increasing leadtime • good forecast of the spatial structure on higher scales • The skill of the models varies for different weather types and the differences between COSMO-2 and COSMO-7 varies also:- best skill: Autumn and Spring, south to northwest weather types- greatest difference COSMO-2 minus COSMO-7: Summer and Winter, north- and east types, convective cases Tanja Weusthoff

  30. COSMO-2 03 UTC up to +48h:Comparison COSMO-2 to COSMO-7 • period (full year): 1 June 2010 – 31 May 2011 • Main interest: • advance of COSMO-2 as compared to COSMO-7 • question: “are the COSMO-2 forecasts at +48hstill better than those of COSMO-7” ? • Verification with: • SYNOP over Switzerland (hourly):comparison +0 to +24h vs +24 to +48h • Swiss RadarComposit: neighborhood Verifikation for four 12h-Summen (0..+12, +12..+24, +24..+36, +36..+48h) • TEMPs for +21h and +45h

  31. Verification SYNOP-CH: 2m-temperatureCOSMO-2 vs COSMO-7 COSMO-2 COSMO-7 BIAS VAL advance of COSMO-2 also in the second day MAE STDE V. Stauch

  32. Verification SYNOP-CH: precipitation COSMO-2 vs COSMO-7 [+0 to +24h] 0.1 mm/h 1.0 mm/h Frequency bias (mm/h) 10 mm/h COSMO-2 COSMO-7 V. Stauch

  33. Verification SYNOP-CH: precipitation COSMO-2 vs COSMO-7 [+24 to +48h] 0.1 mm/h 1.0 mm/h Frequency bias (mm/h) 10 mm/h COSMO-2 COSMO-7 V. Stauch

  34. Verification with swiss radarcomposit daytime: 03-15 UTC nighttime: 15-03 UTC first forecast day +12..+24h +0.. +12h COSMO-2 COSMO-2 COSMO-7 COSMO-7 +24..+36h +36..+48h second forecast day COSMO-2 COSMO-2 COSMO-7 COSMO-7 T. Weusthoff

  35. COSMO-2 better COSMO-7 better FRACTIONS SKILL SCORE: COSMO-2 – COSMO-7precipitation sums of 12h + 0 .. + 12h + 12 .. + 24h + 36 .. + 48h + 24 .. + 36h T. Weusthoff

  36. COSMO-2 better COSMO-7 better UPSCALING: COSMO-2 – COSMO-7precipitation sums of 12h; score: ETS (equitable threat score) + 0 .. + 12h + 12 .. + 24h + 36 .. + 48h + 24 .. + 36h T. Weusthoff

  37. 3. „Fuzzy in Time“ Extension of the spatial window with a window in time volume (dx * dy * dt) Evaluation of the forecasts in this volume Time-window ntm = [1h,3h,5h,7h,9h] dx dy dt T. Weusthoff

  38. model dx dy t0+1 t0 e.g. FSS fraction blue pxiels model = 12/75 (dt=3), 5/25 (dt=1) fraction blue pxielsobs = 12/75 (dt=3), 3/25 (dt=1) t0-1 observation dx dy t0+1 t0 T. Weusthoff t0-1

  39. hourly accumulated precipitation Evaluation of 3h accumulations, time window ntm = 1 00 UTC 03 UTC 06 UTC 09 UTC 23 UTC 01 UTC 02 UTC 04 UTC Evaluation of 3h accumulations, time window ntm = 5 (t0 +-2h) 00 UTC 03 UTC 06 UTC 08 UTC 22 UTC 02 UTC 01 UTC 05 UTC 20 21 22 23 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 … 00-02 01-03 3 hourly accumulated precipitation 02-04 20-22 03-05 21-23 04-06 22-00 05-07 23-01 T. Weusthoff

  40. FSS for different time-windowsCOSMO-2, July 2010 T. Weusthoff

  41. FSS for different time-windowsCOSMO-7, July 2010 T. Weusthoff

  42. FSS for different time-windows COSMO-2 minus COSMO-7, July 2010 T. Weusthoff

  43. Summary „fuzzy in time“ FSS increases on all scales with increasing time-window greatest effect for small spatical scales lowest effect for high threshods Both models show a similar increase difference COSMO-2 minus COSMO-7 stays equal, resp. becomes littler for high time-windows For Upscaling the influence of a time tolerance is relatively low und restricted on low thresholds ( effect of the avergaing)  Application of time-windows on the gridsclae would make sense; simultaneous application with space tolerance brings no great change T. Weusthoff

More Related