160 likes | 168 Views
This review discusses the validation and verification of Australian outlook models for rainfall, temperature, and El Nino. It includes summaries, terminology, and the performance of different models over time.
E N D
A Review of Bureau of Meteorology Outlooks for Australia Andrew Watkins National Climate Centre Bureau of Meteorology, Melbourne 31st Climate Diagnostics and Prediction Workshop, Boulder, Colorado, October 23-27 2006
Introduction • Australian Outlook Models (Rain/Temp/Nino) • Validation (1950-99) • Verification (2000-2006) • 2005/06 verification • Summary
Terminology – this talk • Validation • assessment of skill by scoring cross-validated hindcasts • essential for assessing new models and expected future performance of current models • Verification • assessment of skill by scoring independent real-time forecasts • used for assessing how a model has performed • undertaken for accountability purposes • limited use for assessing expected future performance due to small sample size
Seasonal Outlook Model Percent consistent • Validation 1950-1999 Rainfall Tmax Tmin http://www.bom.gov.au/silo/products/verif/
Observed Deciles • Observations 2000-2006 Rainfall Tmax Tmin
Seasonal Outlook Model Percent consistent • Verification 2000-2006 Rainfall Tmax Tmin
Seasonal Outlook Model Reliability Rainfall • Verification 2000-2006 Tmax Tmin
Seasonal Outlook Model LEPS score • 2005-06 verification All Australia, all Outlooks Rainfall
Seasonal Outlook Model • 2005-06 verification
Australian Models - POAMA • Global coupled model seasonal forecasting system • Components AGCM (T47L17) + OGCM (0.5x2.0xL25) + OASIS Coupler. OGCM is Australian Community Ocean Model • Operational since Oct 2002. Daily 9-months forecasts produced • Operational products issued by the BoM National Climate Centre www.bom.gov.au/bmrc/ocean/JAFOOS/POAMA www.bom.gov.au/climate/coupled_model/poama.shtml
POAMA (dynamical) Model 2005-06 verification
Summary • Validation of Empirical model shows skill above climatology (an unskilled model) • Verification shows skilful Temp forecasts, but less skilful Rainfall forecasts • Bias tends to be fairly conservative • Past year has shown skill in the southern states, and in the east in recent months • POAMA Dynamical model suffered a cool bias earlier in the year – problem fixed; did predict a warming Pacific
The End “Whatever may be the progress of sciences, never will observers who are trustworthy, and careful of their reputation, venture to foretell the state of the weather.” François Arago in the Annual Report of the Paris (astronomy) observatory, 1846 (as quoted in Storm Watchers, by John D Cox)
References • Fawcett R J B, Jones D A and Beard G S. 2005. A verification of publicly issued seasonal forecasts issued by the Australian Bureau of Meteorology: 1998-2003. Australian Meteorological Magazine, 54, 1-13. • Drosdowsky and Chambers – Journal of Climate 14, 1677-1687 (2001) • validation results for operational SST model (all 4 SST predictors) – 44 year climatology (1950-1993) • verification results for 5 years of independent hindcasts (1994-1998) • Jones – BMRC Research Report No. 70 (1998) • validation results for operational SST model (all 4 SST predictors) – 45 year climatology (1950-1994) • verification results for 3.5 years of independent hindcasts (1995-1998) • http://www.bom.gov.au/climate/ahead/rr70/ • Validation results for current operational models (rain & temps) on web: http://www.bom.gov.au/silo/products/verif/
Seasonal Outlook Model Brier Score • Verification 2000-2006 Rainfall Tmax Tmin