consens priority project status report cosmo year 2008 2009
Download
Skip this Video
Download Presentation
CONSENS Priority Project Status report COSMO year 2008/2009

Loading in 2 Seconds...

play fullscreen
1 / 64

CONSENS Priority Project Status report COSMO year 2008/2009 - PowerPoint PPT Presentation


  • 107 Views
  • Uploaded on

CONSENS Priority Project Status report COSMO year 2008/2009. Involved scientists: Chiara Marsigli, Andrea Montani, Tiziana Paccagnella, Tommaso Diomede (ARPA-SIMC) Flora Gofa, Petroula Louka (HNMS) Felix Fundel (MeteoSwiss). Overview. Task 1: Running of the COSMO-SREPS suite

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'CONSENS Priority Project Status report COSMO year 2008/2009' - graceland


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
consens priority project status report cosmo year 2008 2009

CONSENS Priority ProjectStatus report COSMO year 2008/2009

Involved scientists:

Chiara Marsigli, Andrea Montani, Tiziana Paccagnella, Tommaso Diomede (ARPA-SIMC)

Flora Gofa, Petroula Louka (HNMS)

Felix Fundel (MeteoSwiss)

overview
Overview
  • Task 1: Running of the COSMO-SREPS suite
    • suite maintenance
    • implementation of the back-up suite
  • Task 2: Model perturbations
    • perturbation of physics parameters
    • perturbation of soil fields
  • Task 3: Ensemble merging
    • Multi-clustering
  • Task 4: Calibration
the cosmo sreps ensemble
The COSMO-SREPS ensemble
  • COSMO-SREPS has been developed within the SREPS PP, aiming at the development of a Short-Range Ensemble Prediction System
  • 3 days forecast range, 10 km of horizontal resolution
  • COSMO-SREPS provides boundary conditions for COSMO-DE-EPS, the 2.8 km ensemble system under development at DWD
  • Application: test of the use of COSMO-SREPS to estimate a flow-dependent B matrix in a 1D-Var DA of satellite data
cosmo sreps
IFS – ECMWF global

COSMO at 25 km on IFS

P1: control

P2: physics pert p2

P3: physics pert p3

P4: physics pert p4

COSMO-SREPS

GME – DWD global

COSMO at 25 km on GME

by AEMET Spain

UM – UKMO global

COSMO at 25 km on UM

  • COSMO (v 4.7)
  • 00 UTC and12 UTC
  • 10 km
  • 40 levels
  • 16 members
  • 72 h

GFS – NCEP global

COSMO at 25 km on GFS

1 running of the cosmo sreps suite arpa simc
1. Running of the COSMO-SREPS suiteARPA-SIMC
  • Maintenance of the COSMO-SREPS suite at ECMWF
  • Adaptation of the data output for COSMO-DE-EPS
  • Implementation of a 12 UTC run (beside the 00 UTC one)
  • Implementation of the back-up suite: delayed (9 months)
    • The work involves also DWD, even if implicitly!
    • AEMET has provided the int2lm code adapted for the NCEP and UKMO models
    • An agreement with UKMO has been signed, in order to receive regularly the boundary conditions from the UM
2 1 model perturbations parameters cspert test suite arpa simc hnms
2.1 Model perturbations: parametersCSPERT test suiteARPA-SIMC - HNMS
  • In order to study new parameter perturbations, a test suite (CSPERT) was already implemented at ECMWF, by ARPA-SIMC, during the SREPS PP. Results for SON 2007 can be found in the SREPS final report
  • According to the outcome of the SREPS PP, it was decided to analyse the impact of these perturbations on a dry season as well
  • New runs of the CSPERT suite were performed in autumn 2008, for the JJA 2008 period
  • Analysis of the results completed in May 2009
the cspert suite
IFS – ECMWF globalThe CSPERT suite

P1: control (ope)

P2: conv. scheme (KF)

P3: parameter 1

P4: parameter 2

P5: …

16 LM runs at 10 km

SON 07 + JJA 08

slide11
=

-

--

+

++

Summary of the perturbation impact

remarks from the cspert suite
Remarks from the CSPERT suite
  • The effect of perturbing each physics parameter on improving or worsening the statistical values of the results in comparison to the corresponding control was investigated
  • Based on these results, the next step was to explore the importance and the effect of selected physical perturbations further
  • It seems that the particular parameter perturbations do not influence greatly the mean horizontal wind apart from a few exceptions. Possibly looking at the vertical wind component would make the effects more apparent for some parameters
remarks cont
Remarks (cont)
  • Looking separately at each parameter perturbation compared to the control run:
  • scaling factors related to the laminar layer (rlam_heat, rat_sea), turbulent length scale (tur_len) and evapotranspiration (crsmin), all associated with the development of the turbulent surface layer, are the physical parameters on which the main focus is given
2 1 model perturbations parameters the new cosmo sreps configuration arpa simc hnms
2.1 Model perturbations: parametersthe new COSMO-SREPS configurationARPA-SIMC - HNMS
  • On the basis of the analysis of these results, a new configuration of the COSMO-SREPS suite has been implemented in May 2009
  • An analysis of its performance over summer 2009 (JJA) has been carried out:
    • in terms of 2m temperature only over the Alpine area
    • In term of the continuous parameters (T, U and Td) over Greece
    • Precipitation has not been considered up to now mainly due to the summer season
cosmo sreps new configuration from the 5th of may 2009
convection scheme:

0 Tiedtke

1 Kain-Fritsch

maximal turbulent length scale

length scale of thermal surface patterns

scaling factor of the laminar layer depth

ratio of laminar scaling factors for heat over sea

minimal stomata resistance

COSMO-SREPS new configuration (from the 5th of May 2009)
slide16
Tiedtke

Kain-Fritsch

Tiedtke

Kain-Fritsch

rlam_heat <

IFS

tur_len >

pat_len >

tur_len <

crsmin >

rat_sea <

GME

rat_sea <

pat_len >

rlam_heat <

crsmin >

NCEP

rlam_heat >

rlam_heat >

tur_len >

tur_len <

UM

rat_sea >

rat_sea >

crsmin <

crsmin <

relationship between error and spread
Relationship between error and spread

t2m

JJA09

Small sample, 30 days only

SYNOP over the MAP D-PHASE domain

SYNOP over the whole domain

Nearest grid point

relationship between error and spread1
Relationship between error and spread

JJA09

t2m

12 UTC

00 UTC

SYNOP over the whole domain - Nearest grid point

2m t deterministic scores global model
2.0

3.0

1.5

2.8

1.0

2.6

0.5

2.4

0.0

2.2

-0.5

2.0

-1.0

1.8

ecmwf

gme

ncep

ukmo

JJA09

2m T – deterministic scoresglobal model

BIAS

MAE

SYNOP over the MAP D-PHASE domain

Nearest grid point

2m t deterministic scores convection scheme
2.0

3.0

1.5

2.8

1.0

2.6

0.5

2.4

0.0

2.2

-0.5

2.0

-1.0

1.8

Tiedtke

Kain-Fritsch

JJA09

2m T – deterministic scoresconvection scheme

BIAS

MAE

SYNOP over the MAP D-PHASE domain

Nearest grid point

2m t deterministic scores tur len
2.0

3.0

1.5

2.8

1.0

2.6

0.5

2.4

0.0

2.2

-0.5

2.0

-1.0

1.8

tur_len=150 – ecmwf T

tur_len=1000 – ecmwf KF

tur_len=1000 – ncep T

tur_len=150 – ncep KF

JJA09

2m T – deterministic scorestur_len

BIAS

MAE

SYNOP over the MAP D-PHASE domain

Nearest grid point

2m t deterministic scores pat len
2.0

3.0

1.5

2.8

1.0

2.6

0.5

2.4

0.0

2.2

-0.5

2.0

-1.0

1.8

pat_len=10000 – ecmwf KF

pat_len=10000 – gme T

JJA09

2m T – deterministic scorespat_len

BIAS

MAE

SYNOP over the MAP D-PHASE domain

Nearest grid point

2m t deterministic scores rlam heat
2.0

3.0

1.5

2.8

1.0

2.6

0.5

2.4

0.0

2.2

-0.5

2.0

-1.0

1.8

rlam_heat=0.1 crsmin=200 – ecmwf T

rlam_heat=0.1 – gme KF

rlam_heat=10 – ncep T

rlam_heat=10 – ncep KF

JJA09

2m T – deterministic scoresrlam_heat

BIAS

MAE

SYNOP over the MAP D-PHASE domain

Nearest grid point

2m t deterministic scores crsmin
2.0

3.0

1.5

2.8

1.0

2.6

0.5

2.4

0.0

2.2

-0.5

2.0

-1.0

1.8

rlam_heat=1 crsmin=200 – ecmwf T

rat_sea=1 crsmin=200 – gme T

crsmin=50 – ukmo T

crsmin=50 – ukmo KF

JJA09

2m T – deterministic scorescrsmin

BIAS

MAE

SYNOP over the MAP D-PHASE domain

Nearest grid point

remarks
Remarks
  • Some of the perturbations produce common effects on both regions (e.g. rlam_heat, crsmin, tur_len)
  • However, the impact of some of the physical perturbations (e.g. rat_sea) depends on the geographical characteristics of the region
  • Large values of rlam_heat produce an increase in the error, implying that, theoretically, a deeper laminar layer suppresses the vertical fluxes
  • The value of pat_len will be decreased in the new implementation to be more consistent
  • A paper about the SREPS outcomes is in preparation!
test of new parameter perturbations new cspert suite
Test of new parameter perturbations(new CSPERT suite)

15: ctrl T

Nov 08 - MAMJ 09

16: ctrl KF

2 2 model perturbations developing perturbations for the lower boundary hnms
2.2 Model perturbations: Developing perturbations for the lower boundaryHNMS

Aim

Implement a technique for perturbing soil moisture

conditions and explore its impacts

Reasoning

The lack of spread is typically worse near the surface rather than higher in the troposphere. Also, soil moisture is of primary importance in determining the partition of energy between surface heat fluxes, thus affecting surface temperature forecasts

soil perturbation method
Soil Perturbation method

Based on the method proposed by Sutton and Hamill (2004)

  • Select a period that provides variability in soil moisture e.g. spring
  • Use of data from a land–surface model analysis for the defined period for a few years in order to create some “climatology” (DWD SMA)
  • Implement the EOF (Empirical Orthogonal Function – Principal Component Analysis) to the data in order to generate random perturbations while retaining the spatial structure of the field
  • Define the number of perturbations that will be initially used
  • Test the impact of the perturbation within the COSMO-SREPS suite
3 ensemble merging development of the cosmo leps clustering a montani a corigliano
initial conditions by EPS

initial conditions by MOGREPS

3. Ensemble merging: development of the COSMO-LEPS clustering(A. Montani, A. Corigliano)
  • A dynamical downscaling where driving members for COSMO are taken from different global ensembles is under testing
  • The cluster analysis is applied on a large set of members coming from different global ensembles
  • Up to now, ECMWF EPS and UKMO MOGREPS have been considered
issues
Issues
  • Consider both ECMWF EPS and UKMO MOGREPS and study the properties of the cluster analysis on multi-ensemble:
    • How many times do the 2 ensembles mix?
    • Where do the RMs come from? How to they score depending on their “origin”?
    • Is there added value with respect to single-model ensemble:
      • BEFORE dowscaling
      • AFTER downscaling
forecast and analysis datasets
data from TIGGE-PORTAL (everything in GRIB2)
  • 90 days (MAM09) of ECMWF-EPS and UKMO-MOGREPS run at 00 and 12 UTC
  • use Z500 at fc+96h as clustering variable;
  • for verifying analysis (at 00 and 12 UTC), consider Z500:
    • “consensus analysis” (average of UKMO and ECMWF high-res analyses),
    • independent analysis (e.g. from NCEP);
  • generate the following global ensembles:
    • EPS (50+1): 51 members
    • MOGREPS (23+1): 24 members
    • MINI-MIX (EPS24 + MOGREPS24): 48 members
    • MEGA-MIX (EPS51 +MOGREPS24): 75 members

Forecast and analysis datasets

strategy
perform cluster analysis with 16 clusters and select RMs (like in operations);
  • generate 16-member global ensembles (EPS_REDU, MOGREPS_REDU, MINI_REDU, MEGA_REDU).
  • How do “REDUs” ensembles rank with respect to EPS, MOGREPS, MINI-MIX, MEGA-MIX?
  • Where do the best (and the worst) elements of REDU ensembles come from?
  • How do they score depending on their “origin”?
  • BEFORE DOWNSCALING: is there added value with respect to single-model ensemble?

Strategy

future plans

Future plans

finish by March 2010!

Future future plans

  • Implement dynamical downscaling: nest COSMO model in the selected RMs and generate “hybrid” COSMO-LEPS using boundaries from members of different global ensembles.
  • For a number of case, compare operational COSMO-LEPS and “hybrid” COSMO-LEPS.
summary results
Summary results
  • The availability of the COSMO-SREPS suite has been around 90% during this year, but the system is complete only about 50-60% of the times -> back-up suite!
  • The analysis of the parameter perturbations introduced in the SREPS PP has been completed in Spring, and new selected perturbations have been introduced in the COSMO-SREPS suite in May
  • There is a good impact of the new perturbations on the spread of the system
  • A new set of perturbations, also for the microphysics scheme, is currently under testing
  • A methodology for soil moisture perturbation has been selected and is being implemented at HNMS
  • The work on multi-clustering has started, using the GRIB2 fields of the TIGGE-PORTAL
4 calibration arpa simc meteoswiss
4. CalibrationARPA-SIMC - MeteoSwiss
  • At MeteoSwiss (F. Fundel, Sep 08-Feb 09):
    • Sensitivity tests
    • Documentation/paper
  • At ARPA-SIMC (T. Diomede):
    • Data collection:
      • observations
      • MeteoSwiss reforecast
      • COSMO-LEPS forecasts
    • Choice of the methods
    • Code implementation
    • Evaluation

Preparatory step: visit of Tom Hamill and Felix Fundel at ARPA-SIMC, June 2008

calibration method
Calibration Method

30 years COSMO-LEPS reforecasts (1971-2000)

Observations (stations, gridded fields)

CDF (for one grid point)

Observations

Reforecasts

Reforecasts

Return Period

Return Period

x

x

verification results i
Verification Results I

raw forecasts are overconfident

calibrated forecasts nearly perfect reliable

strong improvements during winter

summer forecast already are reliable, only

little improvement possible

sensitivity study precip
Sensitivity Study (precip)

rel. improvement in RPS over 16 Member CLEPS DMO

  • Cost for 1 member is ~equal to 2 reforecasts
  • 15% improvement (over 16 member CLEPS DMO) using 11-12 members and calibrate with 8-10 years reforecasts
  • Depending on season:
  • more improvement during winter
  • less improvement during summer

current setup (18% improvement)

best, cheap setup (15% improvement)

slide43
158 raingauges

281 COSMO-LEPS grid points

Calibration – data collection

[m]

Emilia-Romagna

Region

(22000 km2)

  • Observations
    • Emilia-Romagna Region
    • 24-h precipitation (08-08 UTC), 1970-2007
  • COSMO-LEPS reforecasts (done by MeteoSwiss)
    • 30 years: 1971-2000
    • 1 member, nested on ERA40, COSMO v4.0
    • 1 run every third day (+90h)
  • COSMO-LEPS QPFs operational
    • 5 years: 2003-2007
calibration choice of the methods
Calibration – choice of the methods
  • choice of methodologies which enable a calibration of the quantitative precipitation forecasts, not only of the probabilities of exceeding a threshold
  • aim:
    • improve COSMO-LEPS output (QPF)
    • hydrological applications
  • chosen methods up to now:
    • Cumulative Distribution Function (CDF) based
    • Linear regression
    • Analogues, based on the similarity of forecast fields:
      • precipitation
      • geopotential height
cdf based corrections
CDF-based corrections

Calibration methodologies

  • For each model grid point:
  • blue line  CDF of COSMO-LEPS reforecasts
  • red line  CDF of historical observations
  • “raw forecast”  each member of the operational COSMO-LEPS

Ref: Zhu and Toth, 2005 AMS Annual Conf., and many others

linear regression
Linear Regression

Calibration methodologies

For each model grid point:

x-axis: COSMO-LEPS reforecasts

y-axis: historical observations

Ref: any applied statistics textbook

slide47
Calibration methodologies

Analogues

For each ensemble member’s forecast and 24-h forecast period (+ 20-44h , 44-68h , 68-92h , 92-116h):

- the analog search is performed in terms of 24-h rainfall pattern over the Emilia-Romagna Region

- the root-mean-square (rms) difference between the current forecast and each reforecast is computed, over all the grid points of the Emilia-Romagna Region

- the historical date with the smallest rms difference is chosen as the date of the analog, then the past raingauge recordings are used as the calibrated forecast

1 analog date for the whole Emilia-Romagna Region

and for each 24-h forecast period

slide48
Calibration – analogues

domain used for the analogue search

example on the methodology used for the analogue search in terms of geopotential at 700 hPa

slide49
Method comparison

+20-44h

autumn

threshold: 5 mm/24 h

threshold: 20 mm/24 h

slide50
Method comparison

+68-92h

autumn

threshold: 5 mm/24 h

threshold: 20 mm/24 h

slide51
Method comparison

+20-44h

spring

threshold: 5 mm/24 h

threshold: 20 mm/24 h

slide52
plain

Different sub-areas

+20-44h

20 mm/24 h

mountain

slide53
mm/24h

underestimation

overestimation

Spatial variability of the Mean Error

Mean Error of the ensemble mean

autumn

+20-44h

raw

calibrated (CDF)

slide54
Spatial variability of the Mean Error

Mean Error of the ensemble mean

spring

+20-44h

raw

mm/24h

underestimation

overestimation

calibrated CDF

slide55
mm/24h

underestimation

overestimation

Spatial variability of the Mean Error

Mean Error of the ensemble mean

summer

+20-44h

raw

calibrated CDF

slide56
mm/24h

underestimation

overestimation

Spatial variability of the Mean Error

Mean Error of the ensemble mean

winter

+20-44h

raw

calibrated CDF

slide57
plain

Flow direction

+20-44h

20 mm/24 h

mountain

slide58
overestimation

underestimation

[m2/s2]

[m2/s2]

Wind S-SW-W

Linear Regression

autumn

upper mountainous

macro-areas

remarks1
Remarks
  • The lack of improvement can be ascribed to the lack of a strong relationship between forecast and observed data.
  • It is necessary to generate correction functions which are weather-type specific. The training sample should be divided into sub-samples which have similar characteristic with respect to the meteorological situation. Hence, a model error which is systematic with respect to the meteorological situation could be identified and reduced by a specific correction function.
next developments
Next developments
  • improve the analogue search, by using upper air fields (geopotential and specific humidity) at different levels and daytimes and testing the size of the domain used for the analogue search
  • apply LR and CDF on a limited sample of analogues
  • verify results by the coupling with hydrologic models
  • extend the calibration over other areas, if observed data will be made available (data over Switzerland are available)
  • reduce the size of the reforecast dataset (in order to use more recent hourly data to calibrate precipitation forecasts accumulated over 12 or 6 h and enable more detailed hydrological applications)
problems encountered
Problems encountered
  • The implementation of the back-up suite has just started (with delay, not critical for the moment)
  • Difficulty in objectively evaluating COSMO-SREPS since the ensemble is often incomplete; problems in verifying precipitation in the summer season
  • Calibration:
    • The performance of the calibration methodology is dependent on the precipitation threshold and on the considered area
    • Difficulty in “catching the bias” of precipitation over Emilia-Romagna, dependent on weather type
    • Are good data over other areas available?
decisions needed
Decisions needed
  • In order to calibrate the ensemble over the whole domain, very good (dense and covering a log period, i.e. years) observations should be made available by other regions/countries within the Consortium. And in principle also outside!
lessons learned
Lessons learned
  • The development of a methodology always introduces new questions, not foreseen, which need to be answered within the project => increase of the amount of work needed (e.g. assess the effect of parameter perturbations in a robust manner, calibration)
  • Some re-shuffling of the timing of the tasks has been applied, but without influencing the project development, since the tasks are independent
ad