1 / 37

Preliminary report on verification scores for malaria and the climate drivers DEMETER to ENSEMBLES

Preliminary report on verification scores for malaria and the climate drivers DEMETER to ENSEMBLES. Andy Morse , Cyril Caminade and Anne Jones Department of Geography, University of Liverpool, Liverpool, United Kingdom A.P.Morse@liv.ac.uk. Running Order. Introduction

katen
Download Presentation

Preliminary report on verification scores for malaria and the climate drivers DEMETER to ENSEMBLES

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Preliminary report on verification scores for malaria and the climate drivers DEMETER to ENSEMBLES Andy Morse, Cyril Caminade and Anne Jones Department of Geography, University of Liverpool, Liverpool, United Kingdom A.P.Morse@liv.ac.uk WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  2. Running Order • Introduction • Malaria verification – ENSEMBLES stream II –limited area • Climate verification – • ENSEMBLES stream II - all Africa • Summary and Way Forward WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  3. Introduction • Normally use BSS and ROCSS to work with probabilistic data • Running RMSE, ACC and MSSS deterministically • on ensemble mean • Developed/modified scripts for skill scores • in NCAR Command Language (NCL) – share & cross validate? • Work not complete – will report via ENSEMBLES website WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  4. Introduction – ROCSS malaria modelling Tier-2 malaria runs - ROC Skill Scores Above Median Event May 4-6 JAS Nov 4-6 FMA DEMETER driven LMM. Areas of high interannual variability were selected and persisted forecast skill was removed from the scores. Jones, A. and Morse, A. (2007) CLIVAR Exchanges, 43 WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  5. Validation of Liverpool Malaria Model (LMM) runs forced by ENSEMBLES stream2 seasonal forecasts WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  6. ENSEMBLES –stream 2 Table of experiments: LMM dynamic malaria mode forced with daily Tx and pptn. The domain: Botswana: [27.5°S-17.5°S] [17.5°E-30°E] WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  7. ENSEMBLES –stream 2 Scores: • ACC: anomaly correlation coefficients • Addresses the question: How well did the forecast anomalies correspond to the observed anomalies? • RMSE: Root mean square errors • Answers the question: What is the average magnitude of the forecast errors? • MSSS: Mean square skill score • with WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  8. ENSEMBLES –stream 2 Results 1961-2000 NDJF: RMSE for Malaria Incidence over Botswana (NDJF 1961-2000). The reference dataset is a companion run forced by ERA40 reanalysis, and the analysis is applied to the ensemble means. WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  9. ENSEMBLES –stream 2 Results 1961-2000 NDJF: ACC for Malaria Incidence over Bostwana (NDJF 1961-2000). The reference dataset is a companion run forced by ERA40 reanalysis, and the analysis is applied to the ensemble means. The dotted area display significant correlations at the 95% confidence interval WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  10. ENSEMBLES –stream 2 Results 1961-2000 NDJF: MSSS for Malaria Incidence over Botswana (NDJF 1961-2000). The reference dataset is a companion run forced by ERA40 reanalysis, and the analysis is applied to the ensemble means. WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  11. ENSEMBLES –stream 2 Results 1981-2000 NDJF: RMSE for Malaria Incidence over Botswana (NDJF 1981-2000). The reference dataset is a companion run forced by ERA40 reanalysis, and the analysis is applied to the ensemble means. WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  12. ENSEMBLES –stream 2 Results 1981-2000 NDJF: ACC for Malaria Incidence over Botswana (NDJF 1981-2000). The reference dataset is a companion run forced by ERA40 reanalysis, and the analysis is applied to the ensemble means. The dotted area display significant correlations at the 95% confidence interval WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  13. ENSEMBLES –stream 2 Results 1981-2000 NDJF: MSSS for Malaria Incidence over Bostwana (NDJF 1981-2000). The reference dataset is a companion run forced by ERA40 reanalysis, and the analysis is applied to the ensemble means. WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  14. Validation climate drivers ENSEMBLES stream2 1981-2000 seasonal forecasts WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  15. ENSEMBLES –stream 2 Results 1981-2000 NDJF: RMSE for tasmax (NDJF 1981-2000). The reference dataset is a NCEP reanalysis, and the analysis is applied to the ensemble means. WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  16. ENSEMBLES –stream 2 Results 1981-2000 NDJF: ACC for tasmax (NDJF 1981-2000). The reference dataset is a NCEP reanalysis, and the analysis is applied to the ensemble means. WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  17. ENSEMBLES –stream 2 Results 1981-2000 NDJF: MSSS for tasmax (NDJF 1981-2000). The reference dataset is a NCEP reanalysis, and the analysis is applied to the ensemble means. WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  18. ENSEMBLES –stream 2 Results 1981-2000 NDJF: RMSE for prlr (NDJF 1981-2000). The reference dataset is a GPCP reanalysis, and the analysis is applied to the ensemble means. WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  19. ENSEMBLES –stream 2 Results 1981-2000 NDJF: ACC for prlr (NDJF 1981-2000). The reference dataset is a GPCP reanalysis, and the analysis is applied to the ensemble means. WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  20. ENSEMBLES –stream 2 Results 1981-2000 DJF: MSSS for prlr (NDJF 1981-2000). The reference dataset is a GPCP reanalysis, and the analysis is applied to the ensemble means. WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  21. Summary& Way Forward? • Experience in BSS and ROCSS (probabilistic) • Question non probabilistic nature/use of RMSE, • ACC and MSSS for EPS verification • All data should come with sets of verification plots • e.g. ECMWF DEMETER • http://www.ecmwf.int/research/demeter/d/charts/verification • Library of scripts? Or portal to where they exsist WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  22. Questions? WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  23. WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  24. Validation climate drivers ENSEMBLES stream1 1991-2001 seasonal forecasts also have for DEMETER equivalent years WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  25. ENSEMBLES –stream 1 Results 1991-2000 DJF: RMSE for tas (DJF 1991-2000). The reference dataset is a ERA40 reanalysis, and the analysis is applied to the ensemble means. WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  26. ENSEMBLES –stream 1 Results 1991-2000 DJF: RMSE for tas (DJF 1991-2000). The reference dataset is a NCEP reanalysis, and the analysis is applied to the ensemble means. WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  27. ENSEMBLES –stream 1 Results 1991-2000 DJF: ACC for tas (DJF 1991-2000). The reference dataset is a ERA40 reanalysis, and the analysis is applied to the ensemble means. WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  28. ENSEMBLES –stream 1 Results 1991-2000 DJF: ACC for tas (DJF 1991-2000). The reference dataset is a NCEP reanalysis, and the analysis is applied to the ensemble means. WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  29. ENSEMBLES –stream 1 Results 1991-2000 DJF: MSSS for tas (DJF 1991-2000). The reference dataset is a ERA40 reanalysis, and the analysis is applied to the ensemble means. WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  30. ENSEMBLES –stream 1 Results 1991-2000 DJF: MSSS for tas (DJF 1991-2000). The reference dataset is a NCEP reanalysis, and the analysis is applied to the ensemble means. WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  31. ENSEMBLES –stream 1 Results 1991-2000 DJF: RMSE for prlr (DJF 1991-2000). The reference dataset is a GPCP reanalysis, and the analysis is applied to the ensemble means. WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  32. ENSEMBLES –stream 1 Results 1991-2000 DJF: ACC for prlr (DJF 1991-2000). The reference dataset is a GPCP reanalysis, and the analysis is applied to the ensemble means. WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  33. ENSEMBLES –stream 1 Results 1991-2000 DJF: MSSS for prlr (DJF 1981-2000). The reference dataset is a GPCP reanalysis, and the analysis is applied to the ensemble means. WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  34. ENSEMBLES –stream 1 Results 1991-2000 JAS: RMSE for prlr (JAS 1991-2000). The reference dataset is a GPCP reanalysis, and the analysis is applied to the ensemble means. WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  35. ENSEMBLES –stream 1 Results 1991-2000 JAS: ACC for prlr (JAS 1991-2000). The reference dataset is a GPCP reanalysis, and the analysis is applied to the ensemble means. WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  36. ENSEMBLES –stream 1 Results 1991-2000 JAS: MSSS for prlr (JAS 1991-2000). The reference dataset is a GPCP reanalysis, and the analysis is applied to the ensemble means. WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

  37. WGSIP12 – Miami - January 2009 – Impacts verification - Andy Morse

More Related