Recent developments and perspectives in meteorological data assimilation jean.pailleux@meteo.fr

1. Recent developments and perspectives in meteorological data assimilationJean Pailleux (Météo-France /CNRM / Toulouse)jean.pailleux@meteo.frPresentation based mainly on experience and work of the NWP teams in Toulouse (Meteo-France) and Reading (ECMWF). Special thanks to the assimilation people of these two teamsNWP = Numerical Weather Prediction

2. Recent developments and perspectives in meteorological data assimilationjean.pailleux@meteo.fr • 1. Estimation theory and data assimilation problems. • 2. The algorithmic evolution of assimilation in NWP. • 3. More and more observation variety for met models. • 4. From NWP to ocean, climate, chemistry assimilation • 5. Probalistic approach of model forecasting through initial state uncertainties (analysis errors). • 6. Some conclusions on the specific aspects of the atmosphere.

3. DATA ASSIMILATION (OBJECTIVE ANALYSIS) • Determination of initial conditions for a model, or…. • Estimation from all the available pieces of information (mainly the observations) of the atmospheric state which is necessary to run a NWP model.

4. THE OBJECTIVE ANALYSIS SCHEME OF PRINCIPLE Y XaA = ? We know the observed data: X1o, X2o, X3o,…. Xb = « background » = « guess » = 6h forecast εX = X – Xb εXaA = ΣWi(Xio-Xib) X2o X1o X3o A O X

5. THE 4D DATA ASSIMILATION PROBLEM X(t) = model variables vector t2 t1 t Find a MODEL TRAJECTORY whose distance to all the available informations on (t1,t2) is MINIMUM

6. where is the vector of atmospheric (model) variables, a « background » for the analysis, the vector of observations, the observation operator, including the model integration, the background error covariance matrix, the observation error covariance matrix. Resolution of the optimisation problem Minimize the cost function

7. Solution in the linear case With the gain matrix And Analysis error Covariance • This is the Optimal least-squares estimator minimum variance for the analysis error • Or BLUE= Best Linear Unbiased Estimator • If all errors are Gaussian, then it is also the maximum likelihood estimate (Lorenc, 1986)

8. These data assimilation equations can be applied…. • …To any physical system (S) described by a state vector X whose time evolution can be described by a model M: • X(t) = M(t,t1) X(t1) • … which is observed in the different components of vector y = HX (R = obs error cov matrix; H = obs operator); • … with Xb being the prior estimate of the model state vector (or background) and B its error cov matrix. • Treated in a sequential way, carrying forward in time X and the error cov matrix A of X, the algorithm is called Kalman filter.

9. THE OBJECTIVE ANALYSIS SCHEME OF PRINCIPLE Y XaA = ? We know the observed data: X1o, X2o, X3o,…. Xb = « background » = « guess » = 6h forecast εX = X – Xb εXaA = ΣWi(Xio-Xib) X2o X1o X3o A O X

10. « WEIGHT AVERAGE » or « SUCCESSIVE CORRECTION » ALGORITHMS Cressman (1959) • Consist in correcting empirically each grid point through a weight average of corrections at observation points in the vicinity • An influence radius a is chosen arbitrarily for each grid point • The weights Wi are computed through a mathematical function which is a priori prescribed (and decreasing with respect to the distance ri between obs. point and grid point point to estimate). Ex: Cressman function: (a2 – ri2)/(a2 + ri2)

11. THE OBJECTIVE ANALYSIS SCHEME OF PRINCIPLE Y XaA = ? We know the observed data: X1o, X2o, X3o,…. Xb = « background » = « guess » = 6h forecast εX = X – Xb εXaA = ΣWi(Xio-Xib) X2o X1o X3o A O X

12. OPTIMUM INTERPOLATION Gandin (1963) • The weights Wi are coming out from the resolution of a linear system of equations based on the covariance matrix of the departures « Obs-Background » at the obs. points (subset of the matrix noted (H.B.Ht + R) before) • The space statistical structure of background errors and observation errors has to be modelled (and evaluated a priori): space autocorrelation functions; structure functions. • When several observation types are introduced in the estimation, the scheme is called « multivariate optimum interpolation »

13. THE OBJECTIVE ANALYSIS SCHEME OF PRINCIPLE Y XaA = ? We know the observed data: X1o, X2o, X3o,…. Xb = « background » = « guess » = 6h forecast εX = X – Xb εXaA = ΣWi(Xio-Xib) X2o X1o X3o A O X

14. PRINCIPLE OF A VARIATIONAL ANALYSIS GLOBAL MINIMISATION OF a « COST FUNCTION »: J(X) = Jo(X) + Jb(X) (dist. to obs.) (dist. to backg.) = (HX-y)tR-1(HX-y) + (X-Xb)tB-1(X-Xb) R et B: error covariance matrices

15. EQUIVALENCE between variational analysis and optimum interpolation • According to the estimation theory there is a mathematical equivalence between the variational formulation (3D) and the optimum interpolation producing analyses at a given time t… • …If some conditions are met!… such as « using the same observationsfor both algorithms »

16. TYPICAL SCHEME FOR MINIMISING J(X) • 0) Initialise X (starting point of the minimisation) • 1) Calculate J(X) and its gradient with respect to X • 2) Xold --- Xnew ( 1 itération of the minimisation scheme) • 3) Go to « 1) » until convergence

17. DIRECT AND ADJOINT OBSERVATION OPERATORS ymod X (model var.) H2 Hn …………. H1 ….. Jo Chain of direct operators y Hn* Jo H1* H2* …………. ymod X Chain of adjoint operators

18. 3D-VAR vs OPTIMUM INTERPOLATION • A 3D Variational analysis (3D-VAR) guarantees a better space consistency of analysed meteorological fields • Improved possibilities for observation use • 3D-VAR is an algorithmic step towards 4D-VAR (4D Variational assimilation)

19. Principle of 4D-VAR assimilation obs Jo Previous forecast analysis Jo xb obs Corrected forecast Jb Jo xa obs 9h 12h 15h Assimilation window

20. DIRECT, TANGENT LINEAR AND ADJOINT MODELS X(ti) -------------- M ----------------- X(tf) δX(ti) ---------------M’----------------- δX(tf) X(ti) ------------M* ----------------- X(tf)

21. DESCRIPTION OF THE OPERATIONAL 4D-VAR AT METEO-FRANCE (Toulouse) SINCE JUIN 2000(Version March 2005) • 6h (max) minimisation window • INCREMENTAL technique: increments evaluated through 2 minimisations at T107 and T149 (c=1) about a T358/c2.4 main model trajectory • Non separable structure functions • OBS USED: conventional, satellite winds, ATOVS radiances,scatterometer data • Weak contraint based on digital filters (also used to smooth the trajectory after the last minimisation)

22. PRINCIPLE OF « INCREMENTAL » 4D-VAR • For computing cost considerations, the incremental variational scheme consists in performing the minimisation (then the assimilation) in a sub-space of the X space. • The minimisation is often performed in the spectral space. The incremental technique then consists in truncating the short waves, then analysing only the large scales.

23. « Incremental » formulation Minimize the cost function where is the vector of « increment » to the background, is the linearized and simplified observation operator. can have a lower resolution than • (Courtier, Thépaut, Hollingsworth, 1994)

24. Recent developments and perspectives in meteorological data assimilationjean.pailleux@meteo.fr • 1. Estimation theory and data assimilation problems. • 2. The algorithmic evolution of assimilation in NWP. • 3. More and more observation variety for met models. • 4. From NWP to ocean, climate, chemistry assimilation • 5. Probalistic approach of model forecasting through initial state uncertainties (analysis errors). • 6. Some conclusions on the specific aspects of the atmosphere.

25. RADIOSONDE OBSERVATIONS

26. AIRCRAFT OBSERVATIONS

27. SATELLITE SOUNDER OBSERVATIONS

28. RETRIEVAL OF A VERTICAL TEMPERATURE PROFILE Z ou p Z ou p INVERSION 1 Radiative transfer 2 T Radiances(weight functions) Ri = Bi[T(p0)].τi(p0) – ƒpBi[T(p)].dτi(p)

29. HIRS channel weighting functions AIRS channel weighting functions

30. GRAS GOME AVHRR ASCAT AMSU-A2 HIRS IASI AMSU-A1 MHS

31. = the path of the ray perigee through the atmosphere Radio occultation geometry

32. Global distribution of simulated GRAS observations over 24 h

33. Recent developments and perspectives in meteorological data assimilationjean.pailleux@meteo.fr • 1. Estimation theory and data assimilation problems. • 2. The algorithmic evolution of assimilation in NWP. • 3. More and more observation variety for met models. • 4. From NWP to ocean, climate, chemistry assimilation • 5. Probalistic approach of model forecasting through initial state uncertainties (analysis errors). • 6. Some conclusions on the specific aspects of the atmosphere.

34. Data assimilation and atmospheric chemistry • More and more atmospheric chemical species (O3, Nox,…) are observed and assimilated into chemical models. • Some operational NWP models (e.g. ECMWF) are treating ozone and assimilating ozone data.

35. Total column ozone – Ozone hole 30 Sept. 1990 No O3 assim. With O3 assim TOMS

36. Data assimilation and ocean modelling • Data assimilation is operational in several oceanographic centres • Observations: SST (Sea Surface Temperature), SSH (Sea Surface Height –altimeter), temperature and salinity profiles,etc…

37. Data assimilation and climate modelling • Coupled atmospheric and ocean systems used for simulating the evolution of the climate • Climate studies relies more enad more on « re-analyses » performed with simplified versions of NWP data assimilation systems (e;g;: http://www.ecmwf.int/research/era/ to study ERA40)

38. Recent developments and perspectives in meteorological data assimilationjean.pailleux@meteo.fr • 1. Estimation theory and data assimilation problems. • 2. The algorithmic evolution of assimilation in NWP. • 3. More and more observation variety for met models. • 4. From NWP to ocean, climate, chemistry assimilation • 5. Probalistic approach of model forecasting through initial state uncertainties (analysis errors). • 6. Some conclusions on the specific aspects of the atmosphere.

39. Sources of uncertainty • Analysis errors due to • Imperfections of the assimilation • Lack of observations • Model errors

40. How can we take into account the initial uncertainty • Analysis perturbations to sample the more unstable modes • Based on the computation of the more unstable modes • Perturbations consistent with the analysis error estimate • Méthod : singular vectors, • Sampling of the model uncertainty • Perturbation of some physical parameters… • Etc…

41. ECMWF/EPS: 10-day forecast over Novosibirsk from Wednesday 16 March 00UTC Cloudiness Precipitation Wind speed 2m Temperature

42. CONCLUSION 1: SPECIFIC ASPECTS OF THE ATMOSPHERIC DATA ASSIMILATION • Atmosphere = « Large size Lab », difficult to observe, where each meteorological situation is unique. • Open system: precise modelling of the atmosphere leads to modelling of ocean, land surface, vegetation, hydrology etc…. • NWP has to be quick and real time: trade-off between speed and reliability of the computations • Large range of scales to treat.

43. CONCLUSION 2: DATA ASSIMILATION AND INVERSE PROBLEMS • Data assimilation is an inverse problem with a very high dimension • It has to be approached in a very pragmatic way: the optimal choice of a data assimilation algorithm depends on the application treated and the computer available

44. RESERVE For questions Not to be shown

45. Analysis Forecast Equivalence with Kalman filter Optimal sequential assimilation: loop over observation times ti

46. Equivalence with Kalman filter At the end of the assimilation window, same Optimal analysis obtained by 4D-Var and Kalman filter (equivalent to the Kalman smoother over the whole assimilation window) Comparison of the model trajectory to observations performed at « the appropriate time » Full use of the dynamics over the window to update implicit forecast error matrix

47. Properties of Variational methods Can be extended to non-Gaussian errors: Var-QC (Lorenc, Andersson) Can use a wide range of observations including those with a complex link to atmospheric variables (eg radiances, Andersson et al, 1994) Efficient use of asynoptic data (Järvinen, Andersson)

48. Data assimilation • Problems to consider in an operational analysis : • nature and quality of the different observations • Quality control • Taking into account the information coming from the past • Preserve the atmosphere balance and control the waves which are not relevant for modeling

49. ADM-AEOLUS (space lidar)

50. Satellite impact satellite: 20 years of ECMWF scores (NH and SH) Document CEPMMT