Verification results from LAPS 0-6h QPFs

1. Verification results from LAPS 0-6h QPFs • Model and data • Evaluation of forecast skill Kuiper score RRMSE Correlation • Evaluation of forecast value • Comparison of STMAS-WRF and CWB-WRF models • Summary

2. Model and data

3. Local Analysis and Prediction System(LAPS)

4. LAPS models (total 8 models)

5. Observation data for precipitation verification • Observation data (ground truth)：QPEs of QPESUMS • Resolution：1.25km. • Calibration : • land area : calibrated with gauges. • sea area : no calibration. LAPS / STMAS domain QPESUMS domain (Shading area : the radar coverage)

6. Validation data • Typhoon cases in 2013 (4 typhoons; total 78 QPFs)

7. Evaluation of forecast skill

8. Forecast skill • Kuiper score (KS) = Hanssen-Kuiper Skill Score (KSS) Hit rate: False alarm rate: c KS range : -1 ~ 1 fcst obs f m h KS = 0 for random or constant forecasts = 1 for perfect forecasts > 0 skillful forecasts

9. Kuiper score (TY Trami + Kong-Rey + Usagi + Fitow) land area

10. Verification Methods • Root mean square error (RMSE) Answers the question: What is the average magnitude of the forecast errors? Range : 0 to infinity perfect score: 0 • Relative root mean square error (RRMSE) Answers the question: What is the average magnitude of the normalized forecast errors? Range : 0 to infinity perfect score: 0

11. Verification Methods • Correlation coefficient (r) Answers the question : How well did the forecast values correspond to the observed values? Range : -1 to 1 1: perfect score • Good measure of linear association or phase error. • Dose not take forecast bias into account – it is possible for a forecast with large errors to still have a good correlation coefficient with the observations.

12. RRMSE (TY Trami + Kong-Rey + Usagi + Fitow)

13. Correlation (TY Trami + Kong-Rey + Usagi + Fitow)

14. Evaluation of forecast value

15. Forecast value-Cost-loss ratio decision model • Economic value (V) of a forecast system Range of V : -∞ ~ 1 V = 1: perfect value V > 0 if Eforecast < Eclimate Eclimate: Expected expense (E) using climatology information. Eforecast: E using a forecast system. Eperfect:E using a perfect deterministic forecast system.

16. Cost-loss ratio decision model Assumes a decision maker takes action totally depending on forecast information. H, m, f, andcare the relative frequencies (< 0) of four outcomes (h+m+f+c=1). C : cost of protection L = Lp + Lu : total loss Lp : protectable loss Lu : unprotectable loss α = C / Lp : cost-loss ratio (o < α < 1 since Lp > C )

17. How to calculate the expected expense ? h+m=o f+c=1-o Expected expense using a forecast system: (N = 0) 2. Expected expense using a perfect forecast system (m=0, f=0)： (since h+m=o  h=o for a perfect forecast system )

18. How to calculate the expected expense ? h+m=o f+c=1-o 3. Expected expense using climatological information only: always take protection never take protection

19. Cost-loss ratio decision model • Economic value (V) of a forecast system h, m and f : forecast performance parameters o :climatological frequency α :cost-loss ratio

20. Forecast value (TY Trami + Kong-Rey + Usagi + Fitow) land area Event: precipitation > 30mm/6h

21. Comparison of STMAS-WRF and CWB-WRF models

22. Schematic diagram of partial cycling for CWB-WRF Analysis time Partial cycling T0 - 12h T0 - 6h T0 time CWB-WRF Forecast GFS forecast as first guess 6h CWB- WRF forecast as first guess 6h CWB- WRF forecast as first guess

23. Comparison of STMAS and CWB-WRF models

24. Conclusion • The STMAS-WRF(GFS) model has greater forecast skill and can offer more economic value to a wider range of users than the other LAPS models. • Comparison of STMAS-WRF and CWB-WRF models • For 0-6h QPFs, the STMAS-WRF(GFS) model has higher • POD and lower FAR; and hence higher ETS. However, Its • wet bias is slightly larger.