250 likes | 462 Views
Bookmaker or Forecaster?. By Philip Johnson. Jersey Meteorological Department. . Why use probabilities?. What was wrong with the “Old” methods. Can categorical forecasts be improved? Do the public understand percentages? How can we produce the odds? Do we know if we are getting it right.
E N D
Bookmaker or Forecaster? By Philip Johnson. Jersey Meteorological Department.
Why use probabilities? • What was wrong with the “Old” methods. • Can categorical forecasts be improved? • Do the public understand percentages? • How can we produce the odds? • Do we know if we are getting it right.
Producing the forecast. • Various model outputs from different centres • MOS (model output statistics) • Climatology • Local knowledge • Feedback of forecast assessment
American MRF model
Verification • To assess the output • To provide quality assurance • Feedback of results to improve forecast
Verification Techniques • Reliability Diagrams • Brier Scores • Skill Scores
Reliability Diagrams • Number of forecasts • Brier % score • Under and over forecasting
Brier and Skill Scores • Brier Scores use same information as reliability diagrams. • Shows a mean square error of the forecast. • 0 is perfect, 1 is worst possible score. • Skill Score compares two Brier scores.
Hedging your bets. • What if you only forecast the climate value? • All forecasts around 28% and you must be right!
Only 25% forecast. • Reliable forecast. • No sharpness.
Single value forecast. • Brier goes up. • Skill goes down.
What results have we had? • Over the last 6 years we have assessed our forecasts. • These have been compared with MOS output. • Sample and fixed Climatology has been used for evaluation.
Brier Results. • These show that the forecaster used a good range of % values, and good skill levels especially up to T +78 • From T+90 the forecasts showed an increase in Brier Scores inline with model variability.
Skill scores • The forecast skill when compared with climate shows good positive values • Model and forecast skill deteriorate noticeably from T+90
Brier and Skill Score v Climate Brier score for fixed 25% forecast
What next? • Continued feedback of results • Improved modeling • More use of Ensemble Forecasts • Neural networks
Bookmaker or Forecaster? By Philip Johnson. Jersey Meteorological Department.