slide1 n.
Skip this Video
Loading SlideShow in 5 Seconds..
D.Kiktev, E.Astakhova, A.Muravyev, M.Tsyrulnikov PowerPoint Presentation
Download Presentation
D.Kiktev, E.Astakhova, A.Muravyev, M.Tsyrulnikov

Loading in 2 Seconds...

play fullscreen
1 / 25

D.Kiktev, E.Astakhova, A.Muravyev, M.Tsyrulnikov - PowerPoint PPT Presentation

  • Uploaded on

D.Kiktev, E.Astakhova, A.Muravyev, M.Tsyrulnikov. Perfomance of the WWRP project FROST-2014 forecasting systems: Preliminary assessments (FROST = Forecast and Research in the Olympic Sochi Testbed). WWOSC-2014, 16-21 August 2 014. PRESENTED by S. BELAIR (with additions).

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'D.Kiktev, E.Astakhova, A.Muravyev, M.Tsyrulnikov' - keona

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
D.Kiktev, E.Astakhova, A.Muravyev, M.Tsyrulnikov

Perfomance of the WWRP project FROST-2014 forecasting systems: Preliminary assessments (FROST = Forecast and Researchin the Olympic Sochi Testbed)

WWOSC-2014, 16-21 August 2014

PRESENTED by S. BELAIR (with additions)


Items in this presentation:

Brief reminder

Online monitoring, evaluation (deterministic and ensemble)

Interesting cases

Integrated forecasting

What’s next

goals of wmo wwrp rdp fdp frost 2014

• To develop a comprehensive information resource of alpine winter weather observations;

  • To improve and exploit:– high-resolution deterministic mesoscale forecasts of meteorological conditions in winter complex terrain environment;

– regional meso-scale ensemble forecast products in winter complex terrain environment;

– nowcast systems of high impact weather phenomena (wind, precipitation type and intensity, visibility, etc.) in complex terrain.

• To improve the understanding of physics of high impact weather phenomena in the region;

• To deliver deterministic and probabilistic forecasts in real time to Olympic weather forecasters and decision makers.

• To assess benefits of forecast improvement (verification and

societal impacts)

3 rd meeting of the project participants 10 12 april 2013

International participants of the FROST-2014 project

3rd meeting of the project participants (10-12 April 2013)
  • COSMO,
  • EC,
  • FMI,
  • KMA,
  • NOAA,
  • ZAMG
  • under supervision of the WWRP WGs on Nowcasting, Mesoscale Forecasting, Verification Research

Observational network in the region of Sochi

  • About 50 AMS;
  • C-band Doppler Radar WRM200; - Temperature/Humidity profiler – HATPRO; - Wind – Scintec-3000 Radar Wind Profiler;- Two Micro Rain vertically pointing Radars (MRR-2);
  • -4 times/day upper air sounding in Sochi

Forecasting systems participating in RDP/FDP FROST-2014

Deterministic NWP:

COSMO-RU with grid spacing 1km, 2.2km, 7km;

GEM with grid spacing 2.5 km, 1 km, 0.25 km;

NMMB – 1 km; HARMONIE – 1 km; INCA – 1 km

Ensemble NWP: COSMO-S14-EPS (7km),

Aladin LAEF (11km), GLAMEPS (11km), NNMB-EPS (7km ), COSMO-RU2-EPS (2.2km), HARMON-EPS (2.5km)

Nowcasting: ABOM, CARDS, INCA, INTW, MeteoExpert, Joint (Multi-system forecast integration)


FROST-2014 Online Monitoring of Forecast Quality:

Role of resolution - GEM-2.5 km vs GEM-1 km vs GEM-250 m

Forecast mean absolute errors

as a function of forecast lead


Location: Mountain skiing finish

(Roza-Khutor-7 station).


15 Jan – 15 March

Effect is not straightforward.

It depends on meteorological variable, location, lead time etc.


Some examples of diagnostic verification: Role of spatial resolution.

COSMO-S14-EPS (7km grid spacing) vs COSMO-RU2-EPS (2km grid spacing)

Parameter: T2m, Location: Biathlon Stadium (1075m),

Verification Period: 15.1.2014-15.3.2014, Verification approach: Nearest point


(7km grid spacing)


(7km grid spacing)


(2km grid spacing)


(2km grid spacing)

Better shape on Q-Q-plots and higher variability for the downscaled ensemble forecasts.


Role of spatial resolution for ensemble forecasts – continued

COSMO-S14-EPS (7km grid spacing) vs COSMO-RU2-EPS (2km grid spacing)

Verifications for ensemble mean

Verification Period: 15.1.2014-15.3.2014

  • T2m: Some positive effect of downscaling from 7 to 2 km resolution.
  • Wind Speed: No positive effect of dynamical downscaling was found.

Some ensemble verifications:

ROC Area, Brier Skill Score, and Brier Score for Precip > 0.01 mm/3h




COSMO-S14-EPS – red

COSMO-RU2-EPS – orange

LAEF-EPS – brown

NMMB-EPS – black


GLAMEPS – green

Verification approach: 13 stations in the area of Krasnaya Polyana were clustered for matching to forecasts.

Lead time

COSMO-S14-EPS, NMMB-EPS and COSMO-RU2-EPS look most informative.


ROCA, BSS, and BS scores for Precip > 5 mm/3h




COSMO-S14-EPS – red

COSMO-RU2-EPS – orange

LAEF-EPS – brown

NMMB-EPS – black


GLAMEPS – green

For higher Precip threshold (w.r.t the low threshold):

= COSMO-S14-EPS, NMMB, and HARMON-EPS become worse.

= In contrast, LAEF and GLAMEPS become better.


FROST-2014 experience demonstrates that direct forecast of visibility

is a serious challenge. However, some results were encouraging.

Example: 17 February 2014, 11:00-12:00 UTC (Biathlon venue) –

Forecast of time slot for competitions during the 3-days period with low visibility.

Forecast of wind direction and relative humidity (as proxy of visibility)

byCOSMO-Ru1 (1km grid spacing)

RH at 2m:

Forecast and observations

17.02.2014 . Camera shots from Gornaya Carousel-1500

11:00 UTC

11:30 UTC

12:00 UTC

Wind and RH at 850 hPa. Forecast from 12 UTC 16.02.2014

11:00 UTC

12:00 UTC

13:00 UTC

Biathlon Stadium

Biathlon Stadium

Biathlon Stadium


It was not simple for forecasters to deal with such an amount of information under the operational time constraints => Integrated Forecast

  • F. Woodcock and C. Engel: Operational Consensus Forecasts,
  • Weather and Forecasting, 2005;
  • L.X. Huang and G.A. Isaac: Integrating NWP Forecasts and Observation Data to Improve Nowcasting Accuracy, Weather and Forecasting, 2012

F(t) –integrated forecast (t – forecast time);

O – last available observation;

fi(t)– forecast of i-thparticipating forecasting system;

α(t), βi(t) - weights;

bi(t) - biasfori-thforecasting system


FROST-2014 weather data feedfor the Olympic information system

Integrated objective multi-model forecasts served as a first guess for preparation of the “official forecasts” for the Olympic information system.Web-editor was developed for forecasters for correction of objective forecasts. ATOS Requrements: - 1-hour update frequency;- Temporal resolution: for a current day – 1 hour; for subsequent days – 3 hours; - Forecast outlooksfor a current day and next 5 days;- Alert Warnings


Project Social and Economic Impacts

  • Socially significant project application areas:
  • Education
  • Understanding
  • Transfer of technologies
  • Practical forecasting – first guess for operational official forecasts.
  • Integrated project forecasts were used as a first guess for the data feed to the Olympic information system.

Further steps

  • Enhanced quality control of the project observations archive.
  • Additional diagnostic tools and export facilities on the project web-site http://frost2014.
  • Open access for international research community.
  • Validation and intercomparison of the participating forecasting systems, case studiesand numerical experiments, assessments of predictability of various weather elements;
  • Update of developed technologies and transfer of positive experience into operational practice.

What’s next…

Workshop this Fall in Moscow (difficult to attend for some of the participants)

Interesting case studies for the international community

Joint paper?

Lessons learned (after Vancouver and in preparation for upcoming events)

http frost2014 meteoinfo ru
Thank you!

Gratitude to all the participants !

Along with traditional verification measures some new scores were implemented.

EDI - Extremal Dependence Index

EDI = (logF – logH) / (logF+logH)

EDI is especially recommended for low base-rate thresholds, but it will give a good comparative estimate of accuracy for all thresholds (“Suggested methods for the verification of precipitation forecasts against high resolution limited area observations” by the JWGFVR (Laurie Wilson, Beth Ebert et al.)

NOTES: - Pictures will refer to thresholds 0.01 and 3, and the last threshold at which any of the three EDI curves remains not interrupted in the 0-36h interval

- The base rate has the following approximate values:

P(0.01mm/3h)=0.3; P(1mm/3h)=0.2; P(2mm/3h)=0.15; P(3mm/3h)=0.1; P(4mm/3h)=0.055;P(5mm/3h)=0.05



Highest threshold: 8 mm/3h

Lower decision-making


Blue: EDI for 50%

probability threshold

Green: for 66%

Red: for 90%

conclusions edi
Conclusions, EDI

Extremal Dependence Index, EDI, can be used for decision making, especially for rare events when other scores, such as PSS, approach zero. Constructing EDI for different probability decision levels (50, 66, and 90%) showed that the participated EPSs demonstrate skill for all these levels up to the following precipitation thresholds:

COSMO-S14-EPS and NMMB-EPS – informative up to 8mm/3h;

COSMO-RU2-EPS, Harmon-EPS, ALADIN LAEF - informative up to 6mm/3h;

GLAMEPS – informative up to 4mm/3h.

Sampling effects are evident for all the models, especially for higher thresholds of variables.

It is not possible to single out “the best ensemble producing system”, but still some conclusions can be drawn.