1 / 25

Bookmaker or Forecaster?

Bookmaker or Forecaster?. By Philip Johnson. Jersey Meteorological Department. . Why use probabilities?. What was wrong with the “Old” methods. Can categorical forecasts be improved? Do the public understand percentages? How can we produce the odds? Do we know if we are getting it right.

paul2
Download Presentation

Bookmaker or Forecaster?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Bookmaker or Forecaster? By Philip Johnson. Jersey Meteorological Department.

  2. Why use probabilities? • What was wrong with the “Old” methods. • Can categorical forecasts be improved? • Do the public understand percentages? • How can we produce the odds? • Do we know if we are getting it right.

  3. The product that started us in Probability forecasting.

  4. Producing the forecast. • Various model outputs from different centres • MOS (model output statistics) • Climatology • Local knowledge • Feedback of forecast assessment

  5. UKMO forecast chart

  6. American MRF model

  7. American MRF model

  8. MSL pressure from an American ensemble forecast.

  9. Verification • To assess the output • To provide quality assurance • Feedback of results to improve forecast

  10. Verification Techniques • Reliability Diagrams • Brier Scores • Skill Scores

  11. Reliability Diagrams • Number of forecasts • Brier % score • Under and over forecasting

  12. Brier and Skill Scores • Brier Scores use same information as reliability diagrams. • Shows a mean square error of the forecast. • 0 is perfect, 1 is worst possible score. • Skill Score compares two Brier scores.

  13. Brier and Skill Score v Sample Climate Brier score

  14. Hedging your bets. • What if you only forecast the climate value? • All forecasts around 28% and you must be right!

  15. Only 25% forecast. • Reliable forecast. • No sharpness.

  16. Single value forecast. • Brier goes up. • Skill goes down.

  17. What results have we had? • Over the last 6 years we have assessed our forecasts. • These have been compared with MOS output. • Sample and fixed Climatology has been used for evaluation.

  18. Brier Results. • These show that the forecaster used a good range of % values, and good skill levels especially up to T +78 • From T+90 the forecasts showed an increase in Brier Scores inline with model variability.

  19. Skill scores • The forecast skill when compared with climate shows good positive values • Model and forecast skill deteriorate noticeably from T+90

  20. Brier and Skill Score v Climate Brier score for fixed 25% forecast

  21. Did the forecaster make it as a successful Bookmaker? YES

  22. What next? • Continued feedback of results • Improved modeling • More use of Ensemble Forecasts • Neural networks

  23. Bookmaker or Forecaster? By Philip Johnson. Jersey Meteorological Department.

More Related