1 / 32

Weather type dependant fuzzy verification of precipitation

Weather type dependant fuzzy verification of precipitation. Tanja Weusthoff. COSMO General Meeting, Offenbach, 07.-11.09.2009. Fuzzy Verification. fcst. obs. „multi-scale, multi-intensity approach“ „Fuzzy verification toolbox“ of B. Ebert Two methods Upscaling (UP)

andra
Download Presentation

Weather type dependant fuzzy verification of precipitation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Weather type dependant fuzzy verification of precipitation Tanja Weusthoff COSMO General Meeting, Offenbach, 07.-11.09.2009

  2. Fuzzy Verification fcst obs • „multi-scale, multi-intensity approach“ • „Fuzzy verification toolbox“ of B. Ebert • Two methods • Upscaling (UP) • Fraction Skill Score (FSS) • present output scale dependent • standard setting: 3h accumulations COSMO-2 (2.2km): leadtimes 03-06 COSMO-7 (6.6km): leadtimes 03-06,06-09,09-12,12-15 increasing box size increasing threshold

  3. … Upscaling (UP) (Atger, 2001) 1. Principle: Define box around region of interest and calculate the average of observation and forecast data within this box. 2. Contingency Table Event if Rave ≥ threshold No-Event if Rave < threshold Rave observation 3. Equitable Threat Score (ETS) forecast Q: Which fraction of observed yes - events was correctly forecast?

  4. … Fraction Skill Score (FSS) 1. Principle: Define box around region of interest and determine the fraction pj and oj of grid points with rain rates above a given threshold. (Roberts and Lean, 2005) 2. Probabilities 0 < pj < 1 = fraction of fcst grid points > threshold 0 < oj < 1 = fraction of obs grid points > threshold 3. Skill Score for Probabilities Q: On which spatial scales does the forecast resemble the observation? FBS worst  no colocation of non-zero fractions

  5. … Fraction Skill Score (FSS) 4. Useful Scales (Roberts and Lean, 2005) useful scales are marked in bold in the graphics

  6. good bad Upscaling and Fractions Skill ScoreJun – Nov 2007 - = Fractions skill score COSMO-2 COSMO-7 Difference - = Upscaling COSMO-2 better COSMO-7 better

  7. COSMO-2 better D I F F E R E N C E S COSMO-7 better abs(median) / 0.5(q95-q05) [ 10 , Inf ] q(50%) [ 5 , 10 [ q(5%) [ 2 , 5 [ q(95%) ¦ COSMO-2 – COSMO-7 ¦ [ 1 , 2 [ [ 0 , 1 [ 5% 95% COSMO-2 vs. COSMO-7 Values = Score of COSMO-2 • Size of numbers = abs(Median) / 0.5(q95-q05) • measure for significance of differences

  8. COSMO-2 better D I F F E R E N C E S COSMO-7 better Sensitivities Only 00 and 12 UTC model runs COSMO-2 & COSMO-7: leadtimes 3-6,6-9,9-12,12-15  absolute values of COSMO-2 slightly lower, but still the same pattern of differences

  9. Weather type verification: COSMO-2 Upscaling Window size: 3gp, 6.6 km Window size: 27gp, 60 km Fractions Skill Score Window size: 3gp, 6.6 km Window size: 27gp, 60 km Threshold [mm/3h] 34 7 # cases 4 7 5 5 7 45 31 23 15

  10. Smallest spatial scale [gridpoints] where the forecast has been useful regarding to FSS „useful scales“ definition. COSMO-2, gridpoints* 2.2 km COSMO-7, gridpoints* 6.6 km # cases - 7 4 + 23 + 5 7 45 5 31 - 15 34 7 16 14 10 7 5 2 0.5 <0.1 16 14 10 7 5 2 0.4 <0.1 % obs gridpts >= thresh (whole period)

  11. Fraction of observation gridpoints >= threshold  climatology

  12. 45 34 31 23 15 7 7 7 4 5 5 Frequency of Weather Classes, June – November 2007

  13. COSMO-2 better D I F F E R E N C E S COSMO-2 (wc) better D I F F E R E N C E S COSMO-7 better COSMO-2 (all) better Northerly Winds (NE,N) 11 days COSMO-2 (wc) vs.COSMO-2 (all) COSMO-2 vs.COSMO-7  COSMO-2 in northerly wind situations clearly worse than over whole period.

  14. COSMO-2 better D I F F E R E N C E S COSMO-2 (wc) better D I F F E R E N C E S COSMO-7 better COSMO-2 (all) better Southerly Winds (SE,S) 12 days COSMO-2 (wc) vs.COSMO-2 (all) COSMO-2 vs.COSMO-7  COSMO-2 in southerly wind situations clearly better than over whole period.

  15. COSMO-2 better D I F F E R E N C E S COSMO-2 (wc) better D I F F E R E N C E S COSMO-7 better COSMO-2 (all) better Northwesterly winds (NW) 23 days COSMO-2 (wc) vs.COSMO-2 (all) COSMO-2 vs.COSMO-7  COSMO-2 in nothwesterly wind situations at large thresholds clearly better than over whole period.

  16. COSMO-2 better D I F F E R E N C E S COSMO-2 (wc) better D I F F E R E N C E S COSMO-7 better COSMO-2 (all) better Flat (F) 15 days COSMO-2 (wc) vs.COSMO-2 (all) COSMO-2 vs.COSMO-7  COSMO-2 in flat pressure situations clearly worse than over whole period.

  17. Summary / Conlusions • Fuzzy verification of the D-PHASE operations period in 2007 has shown that COSMO-2 generally performed better than COSMO-7 on nearly all scales • part of this superiority is caused by the higher update frequency, but using same model runs still shows the same pattern of differences • the results for different weather types show large variations • best results were found for southerly winds and winds from Northwest and West, • Northeasterly winds as well as Flat pressure situations lead to worse perfomance of both models

  18. UP with other scores POD = H/(H+M) Probability of Detection (Perfect score: 1) FAR = FA/(H+FA) False Alarm Ratio (Perfect score: 0) POFD = FA/(CN+FA) Probability of false detection (Perfect score: 0) OR = (H*CN) / (M*FA) Odds Ratio (perfect score: infinity) BIAS = (H+FA) / (H+M) Frequency Bias (perfect score: 1) HK =POD – FAR True Skill Statistik (perfect score: 1)

  19. UP „useful scales“? COSMO-2, gridpoints* 2.2 km COSMO-7, gridpoints* 6.6 km How to define usefuls scales for ETS?

  20. Binary forecast Binary observation Fuzzy Verifikation Intensity Scale (IS) (Casati et al. 2004) • Transformation of Fcst and Obs into binaryimages on a rain/no-rain basis for the rainfall rate thresholds. • Difference between forecast and observation = binary error image. • Decomposition into the sum of components at different spatial scales by performing a two dimensional discrete wavelet decomposition. What is the relative improvement of the forecast over some reference forecast? Score:Mean squared error (MSE) and MSE skill score (SS) for each spatial scale component of the binary error image.

  21. Intensity Scale COSMO-2 COSMO-7

  22. Intensity Scale COSMO-2

  23. Outlook • operational Fuzzy verification is about to start, including Upscaling and Fraction Skill Score • season, year • weather-type dependant • Intensity scale results will further be investigated (new developments of B. Casati)

  24. All Weatherclasses • Overview over all weatherclasses – differences COSMO-2 minus COSMO-7 and absolute values of COSMO-2, here without bootstrapping

  25. COSMO-2 better D I F F E R E N C E S COSMO-7 better Northerly Winds (NE,N,NW) N NW NE 23 days 7 days 4 days

  26. COSMO-2 better D I F F E R E N C E S COSMO-7 better Southerly Winds (SE,S,SW) S SW SE 45 days 5 days 7 days

  27. COSMO-2 better D I F F E R E N C E S COSMO-7 better East and West (E,W) E W 5 days 31 days

  28. COSMO-2 better D I F F E R E N C E S COSMO-7 better Flat, High, Low (F,H,L) F H L 15 days 34 days 7 days

  29. Summary

  30. Summary Weatherclasses I

  31. Summary Weatherclasses II

  32. Summary Weatherclasses III

More Related