Diagnostic verification and extremes: 1 st Breakout

1. Diagnostic verification and extremes: 1st Breakout • Discussed the need for toolkit to build beyond current capabilities (e.g., NCEP) • Identified (and began to address) 3 major questions: • How should confidence intervals and hypothesis tests be computed, especially when there are spatial and temporal correlations? • What methods should be included for evaluating extremes? • What diagnostic approaches should be included initially; in a 2nd tier; in a 3rd tier

2. Confidence intervals and hypothesis tests • Need to appropriately take into account autocorrelations • Reduce sample by eliminating cases • Block re-sampling (Candille results indicate spatial correlation may have more impact than temporal, at least for upper air) • Identify situations when parametric approaches are “ok” • Bootstrapping approaches are computer-intensive and require lots of data storage • May not always be practical in operational settings

3. Methods for evaluation of extremes • Need to distinguish (in our minds) between extremes and high impact weather • User should define thresholds for extremes • May be based on quantiles of sample distribution • Could use extreme value theory to help with this (e.g., return level methods) • Extreme dependency score is appropriate in many cases • Also compute standard scores: Yule’s Q; odds ratio; ORSS; ETS, etc.

4. Diagnostic methods • Goal: Identify different tiers of methods/capabilities that will be implemented over time, starting with Tier 1 in 1st release • Initial discussion: Stratification • Friday discussion: Specific methods

5. Stratification • Tier 1: Based on meta-data, including time-of-day, season, location, etc. • Tier 2: Based on other information from the model, such as temperature, wind direction, etc. • Tier 3: Based on feature such as location or strength of jet core; cyclone track; etc.