410 likes | 563 Views
COSMO-LEPS: present status and plans Andrea Montani, C. Marsigli, T. Paccagnella ARPA-SIMC HydroMeteoClimate Regional Service of Emilia-Romagna, Bologna, Italy. COSMO General meeting Sibiu, 2-5 September 2013. Outline. Present status of COSMO-LEPS: about the operational verification,
E N D
COSMO-LEPS:present status and plansAndrea Montani,C. Marsigli, T. PaccagnellaARPA-SIMCHydroMeteoClimate Regional Service of Emilia-Romagna, Bologna, Italy COSMO General meeting Sibiu, 2-5 September 2013 A.Montani; The COSMO-LEPS system.
Outline • Present status of COSMO-LEPS: • about the operational verification, • about the calibrated precipitation, • about the convection schemes, • about the clustering technique, • about the future plans. A.Montani; The COSMO-LEPS system.
COSMO-LEPS suite @ ECMWF: present status 16 Representative Members driving the 16 COSMO-model integrations (weighted according to the cluster populations) using either Tiedtke or Kain-Fristch convection scheme (members 1-8 T, members 9-16 KF) + perturbations in turbulence scheme and in physical parameterisations 3 levels 500 700 850 hPa 4 variables Z U V Q d+3 d+4 d d+5 d+1 d+2 d-1 Cluster Analysis and RM identification Cluster Analysis and RM identification older EPS 00 2 time steps younger EPS 12 European area clustering period Complete Linkage • suite runs twice a day (00 and 12UTC) as a “time-critical application” managed by ARPA-SIMC; • Δx ~ 7 km; 40 ML; fc+132h; • COSM0 v4.26 since Jan 2013; • computer time (50 million BUs for 2013) provided by the ECMWF member states in COSMO. COSMO-LEPS Integration Domain COSMO-LEPS clustering area A.Montani; The COSMO-LEPS system.
16 January 2013: COSMO upgrade: 4.21 4.26; • int2lm upgrade: 1.18 1.20. • 17 January 2013: operational dissemination implemented for ARPA-Veneto. • 22 January 2013, technical changes at ECMWF: • change of ECMWF super-computer and of the user running the suite: itm zcl; • introduction of a new “dissemination stream” for COSMO-LEPS: “ad-hoc” initial and boundary conditions do not have to be retrieved any more, but are prepared on a dedicated file system; product dissemination starts about 40 minutes earlier than before (at 9UTC and 21UTC). • 7 May 2013: enriched test dissemination implemented for HNMS. • 13 May 2013: in the framework of GEOWOW research project, COSMO-LEPS was the first system to populate TIGGE-LAM archive at ECMWF (high-priority parameters in grib2 format). • 25 June 2013: tests with Fieldextra 11.1.0 started. Main changes during the COSMO year A.Montani; The COSMO-LEPS system.
Outline • Present status of COSMO-LEPS: • about the operational verification, A.Montani; The COSMO-LEPS system.
SYNOP on the GTS Time-series verification of COSMO-LEPS Main features: variable: 12h cumulated precip (18-06, 06-18 UTC); period : from Dec 2002 to Jul 2013; region: 43-50N, 2-18E (MAP D-PHASE area); method: nearest grid point; no-weighted fcst; obs: synop reports (about 470 stations/day); fcst ranges: 6-18h, 18-30h, …, 102-114h, 114-126h; thresholds: 1, 5, 10, 15,25, 50mm/12h; system: COSMO-LEPS; scores: ROC area, BSS, RPSS, Outliers, … both monthly and seasonal scores were computed A.Montani; The COSMO-LEPS system.
Brier Skill Score: time series and seasonal scores • BSS is written as 1-BS/BSref. Sample climate is the reference system. Useful forecast systems if BSS > 0. • BS measures the mean squared difference between forecast and observation in probability space. • Performance of the system assessed as time series (6-month running mean) and for the last 4 springs (MAM). • Time series: high month-to-month variability, but a positive trend can be noticed. • Good scores in 2012 and 2013; BSS positive for all thresholds since April 2009; fewer and fewer problems with high thresholds. • Seasonal scores for a fixed event (“12h precip > 10mm”):need to take into account the different statistics for each season (last MAM was the wettest). • Best performance for the last spring, with BSS positive for all forecast ranges. A.Montani; The COSMO-LEPS system.
Outline • Present status of COSMO-LEPS: • about operational verification (time-series scores show improvements), • about the calibrated precipitation; A.Montani; The COSMO-LEPS system.
about calibrated precipitation • For each COSMO-LEPS member, calibrated precipitation is operationally generated over Germany, Switzerland and Emilia-Romagna; the calibration technique is based on CDF-based corrections, making use of COSMO-LEPS reforecast. • For MAM2013, inter-comparison between raw and calibrated 24h TP forecast. Main features: variable: 24h cumulated precip (06-06 UTC); period : DJF 2012-13 and MAM 2013; region: Germany, Switzerland, Emilia-Romagna; method: nearest grid point; no-weighted fcst; obs: synop reports (about 300 stations/day); fcst ranges: 18-42h, 42-66h, 66-90h, 90-114h; thresholds: 1, 5, 10, 15,25, 50mm/12h; system: opecleps and Calibcleps; scores: ROC area, BSS, RPSS, Outliers, RelDiag, … A.Montani; The COSMO-LEPS system.
opecleps vs Calibcleps fc 42-66h; 10mm/24h A.Montani; The COSMO-LEPS system.
Outline • Present status of COSMO-LEPS: • about operational verification (time-series scores show improvements), • about calibration (positive impact, especially over Emilia-Romagna); • about convection schemes; • about the clustering technique; • about the future plans. A.Montani; The COSMO-LEPS system.
Adapt COSMO-LEPS suite to ECWMF forthcoming upgrades: increase of vertical resolution in ECMWF-EPS: 62 91; change of Member-State server: ecaccess ecgb; change of super-computer: IBM Cray. Keep an eye (possibly, two) to the performance of ECMWF EPS. Carry on study about the clustering methodology. about the future plans • Analysis of the performance of COSMO-HYBEPS (COSMO-LEPS + 2-3 COSMO runs nested on IFS/GME/GFS): tests ongoing. • Increase of COSMO-LEPS vertical resolution (40 50ML): tests start in October. • Use of high-resolution ECMWF-EPS boundaries (LAMEPS_BC project): tests start by the end of 2013. • Support verification with Versus. A.Montani; The COSMO-LEPS system.
Thank you for the attention ! A.Montani; The COSMO-LEPS system.
Extra slides on configuration • European Conference on Applications of Meteorology / EMS annual meeting • 09 – 13 September 2013, Reading (UK) • Session NWP4 (on 13 September): Probabilistic and ensemble forecasting at short and and medium-range • http://www.ems2013.net/home.html A.Montani; The COSMO-LEPS system.
Dim 2 Possible evolution scenarios Cluster members chosen as representative members (RMs) Initial conditions Dim 1 LAM scenario Dim 2 LAM scenario LAM integrations driven by RMs LAM scenario Dim 1 Initial conditions COSMO-LEPS methodology ensemble size reduction A.Montani; The COSMO-LEPS system.
COSMO-HYBrid Ensemble Prediction System From the results of CONSENS PP, come to a synthesis with the different ensemble systems / strategies, considering scientific, implementation, solidity aspects. Generate 20-member hybrid ensemble (COSMO-HYBEPS) , where: 16 members comes from COSMO-LEPS, 1 member is nested on IFS (uses Tiedtke scheme), 1 member is nested on IFS (uses Kain-Fritsch scheme), 1 member is nested on GME, 1 member is nested on GFS. already existingtaken from CONSENS. All members have Δx ~ 7 km; 40 ML; fc+132h; Study performance of different members’ combinations with the same ensemble size. “20-members esuite” implemented on 7/9/2012; will be run up to the end of the year A.Montani; The COSMO-LEPS system.
COSMO-LEPS (developed at ARPA-SIM) What is it? It is a Limited-area Ensemble Prediction System (LEPS), based on COSMO-model and implemented within COSMO (COnsortium for Small-scale MOdelling, which includes Germany, Greece, Italy, Poland, Romania, Switzerland). Why? It was developed to combine the advantages of global-model ensembles with the high-resolution details gained by the LAMs, so as to identify the possible occurrence of severe and localised weather events (heavy rainfall, strong winds, temperature anomalies, snowfall, …) generation of COSMO-LEPS to improve the Late-Short (48hr) to Early-Medium (132hr) range forecast of severe weather events. A.Montani; The COSMO-LEPS system.
Operational set-up Additional products: • 1 deterministic run (ICs and 3-hourly BCs from the high-resolution deterministic ECMWF forecast) to “join” deterministic and probabilistic approaches: start at 12UTC; t = 132h; • 1 hindcast (or proxy) run (ICs and 3-hourly BCs from ECMWF analyses) to “downscale” ECMWF information: start at 00UTC; t = 36h. Core products: 16 perturbed COSMO-model runs (ICs and 3-hourly BCs from 16 EPS members) to generate, “via weights”, probabilistic output: start at 12UTC; t = 132h; A.Montani; The COSMO-LEPS system.
Types of perturbations • As for types and values, the results from CSPERT experimentation were followed (* denotes default values for COSMO v4.26 ): • convection_scheme: Tiedtke* (members 1-8), Kain-Fritsch (members 9-16), • tur_len (either 150, or 500*, or 1000), • pat_len (either 500*, or 2000), • crsmin (either 50, or 150*, or 200), • rat_sea (either 1, or 20*, or 40), • rlam_heat (either 0.1, or 1*, or 5), • mu_rain : either 0.5* (with rain_n0_factor =0.1) or 0 (with rain_n0_factor =1.0), • cloud_num (either 5x10^8* or 5x10^7). A.Montani; The COSMO-LEPS system.
convection scheme: T=Tiedtke KF=Kain-Fritsch; • tur_len: maximal turbulent length scale (default 500m); this parameter is used mainly in the calculation of the characteristic length scale for vertical mixing and thus into the calculation of the vertical transport momentum coefficient; • pat_len: length scale of thermal surface patterns (default 500m); this parameter is mainly used in the calculation of the large-scale part of the equation addressing the heat flux parameterisation; horizontal length; • rlam_heat: scaling factor of the laminar layer depth (default 1); it defines the layer with non-turbulent characteristics (molecular diffusion effects only); • rat_sea: ratio of laminar scaling factors for heat over sea (default 20); • crsmin: minimal stomata resistance (default 150); • Cloud_num: Cloud droplet number concentration; • Mu_rain: Exponent of the raindrop size distribution; • ( gscp: Switch on/off of the graupel scheme).
Main results • Time-series verification • ECMWF EPS changed substantially in the last years (more and more weight to EDA-based perturbations) and it is hard to disentangle improvements related to COSMO-LEPS upgrades from those due to better boundaries; nevertheless: • high values of BSS and ROC area for the probabilistic prediction of 12-h precipitation for autumn 2011; • poor performance in the first months of 2012, then recovery. Need to investigate what happened. Case-study verification Consistent signal for different forecast ranges of a high-impact weather event for the snowfalls of February 2012. A.Montani; The COSMO-LEPS system.
Extra slides on verification A.Montani; The COSMO-LEPS system.
Time series of ROC area (6-month running mean) • Area under the curve in the HIT rate vs FAR diagram; the higher, the better … • Valuable forecast systems have ROC area values > 0.6. • Highest scores in the 2nd part of 2011 and, for the highest threshold, in 2013. Positive trend through the years can be noticed. • Drier seasons during 2011 and 2012 with few heavy-precipitation events: limited significance of the results for the 15mm threshold. • Limited loss of predictability with increasing forecast range (not shown). A.Montani; The COSMO-LEPS system.
Seasonal scores of ROC and BSS: last 4 springs • Fixed event (“12h precip > 10mm”): consider the performance of the system for increasing forecast ranges. • Valuable forecast systems have ROC area values > 0.6 and BSS > 0. • Need to take into account the different statistics for each season (MAM 2011 was the driest). • Best performance for the spring 2011 and 2013, but less marked diurnal cycle in 2013. • Spring 2013: BSS is positive for all forecast ranges • Similar results for the other thresholds (not shown). A.Montani; The COSMO-LEPS system.
Outliers: time series + ………seas scores (DJF)? • How many times the analysis is out of the forecast interval spanned by the ensemble members. • … the lower the better … • Performance of the system assessed as time series and for the last 4 winters. • Evident seasonal cycle (more outliers in winter). • Overall reduction of outliers in the years up to 2007; then, again in 2009 and 2010, but later. • Need to take into account the different statistics for each season. • In the short range, best results for winter 2010-2011. • For longer ranges, the performance of the system is “stable”. • Outliers before 10% from day 3 onwards. A.Montani; The COSMO-LEPS system.
Seasonal scores of BSS: ……last 4 winters • Fixed event (“12h precip > 10mm”): consider the performance of the system for increasing forecast ranges. • Fixed forecast range (fc 30-42h): consider the performance of the system for increasing thresholds. • Need to take into account the different statistics for each season (last DJF was the driest). • Fixed event: best performance for the last two winters (ECMWF EPS had a record performance for winter 2009-2010): BSS positive for all forecast ranges. • Fixed forecast range: similar results as before. • Similar results for longer forecast ranges and for higher thresholds. A.Montani; The COSMO-LEPS system.
Ranked Probability Skill Score: time series + …….. seasonal scores (MAM) • A sort of BSS “cumulated” over all thresholds. RPSS is written as 1-RPS/RPSref. Sample climate is the reference system. RPS is the extension of the Brier Score to the multi-event situation. • Useful forecast systems for RPSS > 0. • Performance of the system assessed as time series and for the last 4 springs (MAM). • the increase of the COSMO-LEPS skill is detectable for 3 out of 4 forecast ranges along the years, BUT • low skill in the first months of 2012 (the problem comes from MAM), then recovery. • Best results for MAM 2011; quick decrease of RPSS with fcst range for MAM 2012. A.Montani; The COSMO-LEPS system.
Bias and rmse of T2M Ensemble Mean • Consider bias (the closer to zero, the better) and rmse (the lower the better). • Bias closer to zero (0.5 °C of decrease) and lower rmse for the 7-km suite. • The improvement is not “massive”, but detectable for all forecast ranges, especially for day-time verification. • The signal is stable (similar scores for 1-month or 3-month verification). • Need to correct T2M forecasts with height to assess the impact more clearly. A.Montani; The COSMO-LEPS system.
Overestimation of Td2m and soil moisture (1) • Verification period: MAM07 and MAM08. • Obs: synop reports (about 470 stations x day). • Region: 43-50N, 2-18E (MAP D-PHASE area). • Larger bias and larger rmse in MAM08 rather than in MAM07 for COSMO-LEPS deterministic run (in 2007, no multi-layer soil model). A.Montani; The COSMO-LEPS system.
Extra slides on LAMEPS-BC A.Montani; The COSMO-LEPS system.
Test data for LAMEPS Boundary Conditions A.Montani; The COSMO-LEPS system.
Outline • Introduction: • migration to the 7-km system. COSMO-LEPS 10 km (old) COSMO-LEPS 7 km (new) A.Montani; The COSMO-LEPS system.
COSMO-LEPS 16-MEMBER EPS 51-MEMBER EPS MAM06 Average values (boxes 0.5 x 0.5) tp > 1mm/24h tp > 5mm/24h • As regards AVERAGE precipitation above these two threshols, the 3 systems have similar performance. A.Montani; The COSMO-LEPS system.
opecleps vs Calibcleps fc 42-66h; 10mm/24h fc 42-66h; 1mm/24h A.Montani; The COSMO-LEPS system.
Outline • Present status of COSMO-LEPS: • about operational verification (time-series scores show improvements), • about calibration (positive impact, especially over Emilia-Romagna!); • about convection schemes, • members 1-8 use Tiedtke convection scheme (8TD), • members 9-16 use Kain-Fritsch (8KF). • MAM 2013: compare cleps16, 8TD, 8KF. A.Montani; The COSMO-LEPS system.
about the convection scheme ___ cleps16 ___ 8TD ___ 8KF BSS, tp > 10mm BSS, tp > 1mm • As expected, best performance by the full ensemble (cleps16). • Tiedtke-members better than Kain-Fritsch members, but NOT for all scores. ROC, tp > 1mm ROC, tp > 10mm A.Montani; The COSMO-LEPS system.
about the clustering technique • AIM: provide limited-area ensembles (either convection-parameterised or convection-permitting) with the best set of boundary conditions. • Study different types of clustering analyses (over the same area, grid): • ope: use two 12-hourly-lagged EPS, steps=96/120 (108/132) for the younger (older) EPS, variables=Z/U/V/Q, levels=500/700/850 hPa; • rnd0: use the younger EPS and always select members 0 to 15 (0 denotes the control member) • rnd1 : like rnd0, but select members 1 to 16. • ... • Analyse properties (e.g. spread, skill) of the 16-member global ensembles for several upper-air variables • Outcome: modifications to the number of clusters / number of EPS considered / clustering intervals. A.Montani; The COSMO-LEPS system.
Test modifications of clustering methodology • Consider distances between ECMWF EPS members according to “COSMO-LEPS metric” (Z, U, V, Q in the mid-lower troposphere over the clustering domain). • Look at distances between pairs of ECMWF EPS members; to what extent these distances grow with forecast range, using “COSMO-LEPS metric”? • Study a number of seasons. • Outcome: modifications to the number of clusters / number of EPS considered / clustering intervals. A.Montani; The COSMO-LEPS system.
Extra slides on COSMO-S14-EPS A.Montani; The COSMO-LEPS system.
Important ingredients (from 1st and 2nd FROST meetings) • Provide reasonable “numbers”. addressed • Develop experience with probabilities. ? • Feedback on the top-priority products. being addressed • Snow analysis. ? • Soil-field initialisation. addressed • High-res obs to assess the quality of the system. being addressed • Computer time. addressed • Timeliness in product delivery. addressed • ...... anything to add/remove? A.Montani; The COSMO-LEPS system.
FROST-2014 vs SOCHMEL • Introduction to FROST-2014: • What is it? • What has to do with COSMO? • COSMO ensemble activities within FROST-2014: • introduction to SOCHMEL (the SOCHi-targeted Mesoscale EnsembLe system) • methodology; • phases of development; • planned activity. • Final remarks. A.Montani; The COSMO-LEPS system.