Developing of Evaluation Metrics and Indices for Applications

1. Developing of Evaluation Metrics and Indices for Applications GaliaGuentchev and the NCPP Core and Tech team

2. Community and Collaboration NCPP mission - to accelerate the provision of climate information on regional and local scale for use in adaptation planning and decision making through collaborative participation of a community of scientists and practitioners.

3. Applications related groups

4. Applications Related Metrics and Indices Major focus - the development of a capability for objective and quantitative evaluation of downscaled climate information in support of applications.

5. Example of a Health Impacts Case Study • Focus: Historical changes in the temperature related triggers used in Heat Health Warning Systems (HHWS) – May – September 1971-2000 • Variables – tmax, tmin • The temperature/mortality relationships, when used as trigger setting approaches for HHWS, have been found to most closely identify the most dangerous days in terms of excess mortality - Hajat et al 2010 • Indices: • 92.5th, 95th and 97.5 percentile exceedances of Tmax and Tminwith a minimum duration of 2 days (based on relative threshold exceedances for each summer day with a 5 day window); • Parameters – frequency, timing of occurrence, duration • Areas of interest – 3 metropolitan areas – Boston, Indianapolis, Washington DC; N Carolina counties

6. Information that can be obtained from the Advanced Search capability • Comparisons of downscaled data vs Maurer02v2 • Average summer maximum and minimum temperature comparisons • 95th percentile of maximum or minimum temperature • Annual number of tropical nights (tmin>20degC) • Average, median, maximum, minimum

7. Average summer minimum temperature Minimal differences in the order of -0.5 to 0.5 degrees C regardless of downscaling method or model (some variation in spatial pattern for the ARRM MIROC MEDRES data).

8. 95th percentile of summer minimum temperature

9. Average annual number of tropical nights Large differences between the methods and the models used.

10. Maximum annual number of tropical nights Even larger divergence between the method and model used.

11. Information that can be obtained from the Open Climate GIS capability • Observed and downscaled minimum and maximum temperature • To be used to calculate exceedances above a threshold using dynamical percentiles or a simple absolute threshold • For area of interest – the 3 metropolitan areas and the North Carolina counties • Pre-calculated indices results from the NCPP evaluation • To be used to extract evaluation results for area of interest

13. Metrics calculated on Tmax or Tmin for summer months: • Mean • Median • Maximum • Minimum • Standard Deviation • Absolute threshold exceedances • Number of values between thresholds • Consecutive occurrences above/below an absolute threshold • Frequency of spells • Exceedances using dynamical percentiles for each day – 90, 92.5, 95, 97.5 percentiles

14. Comparison between observed and downscaled data for exceedances above the 97.5 dynamical percentile

15. Need for Evaluation of Indices • General evaluation based only on distributional characteristics of variables of interest does not contain sufficient usable information for applications • There is a need to evaluate applications related indices and metrics • Following good practices to incorporate uncertainty is important – use of various downscaling methods and an array of GCMs and emissions scenarios is recommended (STARDEX, 2005; TGICA 2004)

16. During our working times with the applications groups we will be working with the tools, and will be discussing what is usable interpretation of these evaluation results and what guidance is needed on the use of climate information in their work. END

17. Summer Ave Tmax

18. 95th percentile of summer tmax

19. Minimum annual number of tropical nights