1 / 50

Venugopal, Basu, and Foufoula-Georgiou, 2005: New metric for comparing precipitation patterns…

This paper introduces a Forecast Quality Index for comparing precipitation patterns, utilizing surrogate fields and close forecasts. The study includes examples, surrogates, traditional skill scores, and expert rankings to evaluate forecast quality. Key components such as the universal image quality index and modified UIQI are discussed, alongside the UIQI properties and comparison to traditional methods. The paper highlights the importance of Hausdorff distance in pattern comparison and offers illustrative examples.

dlarry
Download Presentation

Venugopal, Basu, and Foufoula-Georgiou, 2005: New metric for comparing precipitation patterns…

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Venugopal, Basu, and Foufoula-Georgiou, 2005: New metric for comparing precipitation patterns… Verification methods reading group April 4, 2008 D. Ahijevych

  2. Forecast Quality Index • useful for ensembles • uses “surrogate fields” • accounts for “close” forecasts • One number

  3. Outline • Paper overview • universal image quality index (UIQI) and modified UIQI • components of forecast quality index (FQI) • Geometric examples (from Sukanta and Efi) • Perturbed “fake” examples (also from S and E) • Cases from SPC Spring 2005 • surrogates • traditional skill scores • expert rankings

  4. Paper overview – forecast ensembles • filter out similar members, and keep just enough to characterize the probability structure of forecast • find “best” member and propagate it forward • single measure (like RMSE and EqTh) but has important additional information

  5. Paper overview - UIQI • R1 and R2 are fields being compared • 3 terms: • covariance • means • standard deviations • 3 properties: • correlation • brightness (bias) • distortion (variability)

  6. Paper overview – UIQI, Hausdorff • UIQI • entirely amplitude-based measure • not efficient at telling difference between displaced patterns and amplitude error • Distance-based measures • Hausdorff distance

  7. Paper Overview - Hausdorff A h(A,B) forward distance B

  8. Paper Overview - Hausdorff A B h(B,A) backward distance

  9. Paper Overview - Hausdorff A h(B,A) backward distance B h(A,B) forward distance

  10. Paper Overview - Hausdorff A H(A,B) B

  11. Paper Overview – partial Hausdorff A h(A,B) B ?

  12. A B Paper overview - Hausdorff a1 b1 h(A,B) forward distance a3 b2 a2

  13. Paper overview - FQI

  14. Paper overview - FQI

  15. Paper Overview - surrogates

  16. 2 1 0 Paper overview – illustrative example

  17. Geometric examples CSI = 0 for first 4; CSI > 0 for the 5th

  18. mod. UIQI, including zero pixels PHD75 mod. UIQI

  19. when I did 10 surrogates <HS> = 271 +/-27

  20. Perturbed fake cases • 3 pts right, -5 pts up • 6 pts right, -10 pts up • 12 pts right, -20 pts up • 24 pts right, -40 pts up • 48 pts right, -80 pts up • 12 pts right, -20 pts up, times 1.5 • 12 pts right, -20 pts up, minus 0.05”

  21. Spring 2005 SPC cases • surrogates • pictures • example of distribution of forward and backward Hausdorff distances • comparison to traditional methods • comparison to expert scores

  22. 75th percentile 100 surrogates – distribution of Hausdorff distance, solid/forward, dash/backward count Hausdorff distance (in grid spacing units)

  23. surrogate mean PHD75 standard error mod. UIQI FQI: 0.47-0.49 PHD75

  24. 0.26-0.28 0.25-0.27 0.34-0.37

  25. 0.21-0.23 0.22-0.23 0.30-0.31

  26. 0.21 0.25 0.24

  27. 0.30 0.31 0.19

  28. 0.51 0.69 0.42

  29. 0.27 0.30 0.37

  30. 0.37 0.40 0.33

  31. 0.34 0.33 0.49

  32. 0.42 0.48 0.54

  33. r = w/o 1st case first case really bad; experts start out too generous?

  34. expert scores vs grid stats grid stats agree: first case was bad

  35. Pearson correlation coefficient and Spearman rank correlation coefficient

  36. FQI Discussion • application to ensembles • adding to MET • . . .

More Related