1 / 31

Short Range Ensemble Prediction System Verification over Greece Petroula Louka, Flora Gofa

Short Range Ensemble Prediction System Verification over Greece Petroula Louka, Flora Gofa Hellenic National Meteorological Service. HNMS involvement in COSMO-SREPS. Verification of LM-COSMO ensemble forecasts for MAPD-Phase (06-12/2007) Cases of 72-hour forecast horizon, 16 members

holt
Download Presentation

Short Range Ensemble Prediction System Verification over Greece Petroula Louka, Flora Gofa

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Short Range Ensemble Prediction System Verification over Greece Petroula Louka, Flora Gofa Hellenic National Meteorological Service

  2. HNMS involvement in COSMO-SREPS • Verification of LM-COSMO ensemble forecasts for MAPD-Phase (06-12/2007) • Cases of 72-hour forecast horizon, 16 members • Verification domain: Greece • Data used: SYNOP data covering Greece • Parameters verified: • 2m temperature • Mean Sea Level Pressure (MSLP) • Precipitation • Statistical analysis of the results 10th COSMO General Meeting

  3. COSMO-SREPS domain 10th COSMO General Meeting

  4. Greek SYNOP stations 10th COSMO General Meeting

  5. COSMO-SREPS members 10th COSMO General Meeting

  6. Statistical analysis methods • For continuous parameters such as Temperature and MSLP • Bias • RMSE • For non-continuous parameters (precipitation) • Deterministic approach • Multi-category contingency tables • POD, FAR, ETS • Probabilistic approach (e.g., BSS, ROC diagrams, etc) 10th COSMO General Meeting

  7. IFS GME NCEP UKMO 2mTemperature By Member • Underestimation of Temperature. • It seems that GME driven members have better skill than the others. 10th COSMO General Meeting

  8. 2m TemperatureBy Month • Underestimation of summer temperatures • June 2007 was exceptionally warm with strong heat waves • The maximum temperature in Athens reached 46.2 ºC ! • October to December RMSE statistically acceptable. • Different pattern between summer and autumn/winter months 10th COSMO General Meeting

  9. IFS GME NCEP UKMO POD plots dependence on Driving Model 10th COSMO General Meeting

  10. Tiedtke1 Kain-Fritsch1 Tiedtke2 Tiedtke3 POD plots dependence on convective scheme etc. 10th COSMO General Meeting

  11. IFS GME NCEP UKMO FAR plots dependence on Driving Model 10th COSMO General Meeting

  12. Tiedtke1 Kain-Fritsch1 Tiedtke2 Tiedtke3 FAR plots dependence on convective scheme etc. 10th COSMO General Meeting

  13. Brier Skill Score • BSS measures the improvement of the probabilistic forecast relative to the sample climatology = total frequency of the event (sample climatology) • The forecast system has predictive skill if BSS is positive (better than climatology), a perfect system having BSS = 1 10th COSMO General Meeting

  14. BSS plots • The predictive skill is good (positive BSS) for the smaller precipitation thresholds • The first day shows better scores than the other two (especially when compared to the third) • The size of the sample affects the score for the larger precipitation thresholds 10th COSMO General Meeting

  15. BSS plots dependence on Driving Model • It seems that for the 1st day all members group together • Some members show negative BSS for low thresholds • For the 2nd and 3rd days IFS seems to provide better score • The size of the sample affects the score 10th COSMO General Meeting

  16. BSS plots dependence on Physical parameterisations • Members with Tiedtke convective scheme group together • The perturbations of the particular parameters for turbulent and length scale are less important than convective schemes • It seems that the members with Kain-Fritsch convective scheme have worse performance than the others 10th COSMO General Meeting

  17. Reliability Diagrams • The Frequency of an observed event is plotted against the forecast probability of the event. • If the curve lies below the 45° line, the probabilities are overestimated • Points between the "no skill" line and the diagonal contribute positively to the BSS (resolution > reliability). 10th COSMO General Meeting

  18. No skill Climatology – No resolution Reliability Diagrams48hr • Overestimation of the probability especially for the larger threshold, although • Small sample for large thresholds 10th COSMO General Meeting

  19. Relative Operating Characteristic • ROC is a measure of forecast skill. • ROC is a tool that permits to evaluate the ability of the forecast system to discriminate between occurrence and non-occurrence of a precipitation event (to detect the event) • It measures resolution (YES or NO event), but not reliability (e.g. biased forecast) • ROC area < 0.5 indicates no skill. 10th COSMO General Meeting

  20. ROC Area • ROC area values are generally high for the lower thresholds • The ensemble can discriminate those events • It seems that for the first predictive period (24hr) ROC area has larger values for more precipitation thresholds (up to 15mm/day) compared with the other two periods 10th COSMO General Meeting

  21. ROC Areadependence on Driving Model • In general all members contribute similarly • Small sample for larger thresholds 10th COSMO General Meeting

  22. ROC Areadependence on Physical parameterisations • Members with Tiedtke convective scheme group together • The perturbations of the particular parameters for turbulent and length scale are less important than convective schemes • The members with Kain-Fritsch convective scheme seem to perform better => better resolution (but not good reliability, c.f. BSS diagrams) 10th COSMO General Meeting

  23. ROC Curves • Hit rates are plotted against the corresponding false alarm rates to generate the ROC Curve. • The area under the ROC curve is used as a statistical measure of forecast usefulness. 10th COSMO General Meeting

  24. ROC Curves 10th COSMO General Meeting

  25. Remarks on Temperature • Temperature is strongly (~5°C) underestimated during the summer months • Both Bias and RMSE exhibited a diurnal cycle with the (absolute) maxima being during the hottest summer hours, while for the autumn/winter months the diurnal variation had the opposite behaviour • The initial and boundary condition perturbations contribute to the BIAS and RMSE more than the physical parameter perturbations 10th COSMO General Meeting

  26. Remarks on Precipitation • Precipitation amount is overestimated • It is not evident a consistent attitude for the forecasted precipitation driven by a certain initial and boundary condition model • Perturbations of the convective schemes are more important than the perturbations of the particular parameters for turbulent and length scales used • It seems that the members with Kain-Fritsch convective scheme have better resolution but worse reliability compared to the members with Tiedtke convective scheme 10th COSMO General Meeting

  27. 10th COSMO General Meeting

  28. MSLPBy Month 10th COSMO General Meeting

  29. Tiedtke1 Kain-Fritsch1 Tiedtke2 Tiedtke3 2mTemperature By Member • Underestimation of Temperature. • No particular effect of the scaling parameters is evident. 10th COSMO General Meeting

  30. Reliability Diagrams24hr No skill Climatology – No resolution 10th COSMO General Meeting

  31. Reliability Diagrams72hr No skill Climatology – No resolution 10th COSMO General Meeting

More Related