JCSDA seminar 20/10/2014 Satellite Data Assimilation at ECMWF

1. JCSDA seminar 20/10/2014 Satellite Data Assimilation at ECMWF Stephen English with special thanks to: Tony McNally, Niels Bormann, Alan Geer, Marco Matricardi, Sean Healy, Cristina Lupu, Marta Janisková, Michael Rennie, Massimo Bonavita, Lars Isaksen, Mike Fisher, Richard Engelen, Peter Bauer, David Richardson, Thomas Laiden and Erland Källén.

2. What to expect in this talk • General introduction to ECMWF • Satellite data assimilation overview at ECMWF • Data assimilation strategy at ECMWF • Recent highlights in satellite DA at ECMWF • Current impact – recent ECMWF OSEs

3. General introduction

4. E. Källén Evolution of ECMWF forecast skill ~8-10km ~16km ~39km ~210km ~63km

5. E. Källén Forecast error growth 500hPa height, NH 1990 2000 2010 2.5 day gain RMS error (m) 2 day gain 1.5 day gain Forecast lead time (days)

6. T. Haiden Skill gain relative to ERA-Interim (day 5) Skill relative to ERA-I at day 5. Verification against analysis for 500 hPageopotential (Z500), 850 hPa temperature (T850), mean sea level pressure (MSLP) and 2 m temperature (T2M_AN), using RMSE as a metric; against SYNOP for 2 m temperature (T2M), 10 m wind speed (V10), and total cloud cover (TCC), using error standard deviation.

7. P. Bauer System updates • Recent implementations: • November 2010 (36r4): New cloud scheme, SEKF soil moisture analysis, SKEB • June 2011 (37r2): AMSU-A obs. error, EDA variances in 4D-Var • November 2011 (37r3): Rev. cloud scheme, aircraft b/c, NEXRAD assimilation • May 2012 (38r1): New Jb, EDA-filtering, clouds/convection • June 2013 (38r2): HRES/4DVAR/EDA L91 → L137 • November 2013 (40r1): ENS L62 → L91, ocean day-0, 25M EDA, GWD/diffusion • Note: Satellite introductions often between cycles e.g. first use of China’s FY3B satellite in September 2014. • Future implementations • December 2014 (40r3): 4DVAR 3xT255, 2x weekly 11M reforecasts, new surface fields and trace gas climatologies, clouds & convection improvement (e.g. supercooled water), lake model (FLAKE), “all-sky” MHS, 2D RO operator • Autumn 2015 (41r2): Horizontal resolution → 8 or 10km

8. ECWMF Satellite Data Assimilation

9. Around 80 satellite instruments are processed in the operational IFS. Sounders (microwave and infrared) are the most critical observations for accurate NWP and must be properly maintained. 5.5 more instruments per year

10. New technologies in space: Cloud radar and lidar M Janiskova S Di Michele New technology can fill gaps in the Global Observing System. Many years of sustained effort are needed to develop the capability to fully understand and make use of highly innovative new observations in a data assimilation system. Observed (O), Background (B) and Analysis (A) for spaceborne radar (Cloudsat) and lidar (Calipso) data. ECMWF short range forecast captures 2D cloud structures seen in the observations. O B A Lidar Radar

11. A. Geer Improved assimilation: “all-sky” microwave sounding All sky assimilation of microwave imagers improves wind and humidity fields. SSMIS and MHS humidity sounders are being moved to all-sky, and infrared humidity observations may follow. Modelling cloud effects extract more benefit (blue colours) from MHS than trying to screen out cloud-affected data. Clear-sky MHS impact All-sky MHS impact 4% 0% -2% 2% -4% T+72 RMS difference normalised by RMS of control

12. A. Geer Water vapour in the presence of cloud - 183±1 Time of observation Start Assimilation window UTH increment (200-500 hPa mean RH) Humidity reduction at observation time generated by changes in wind (and other dynamical variables) 1000km away, 9h earlier! Zonal wind increment at 400 hPa

13. ECMWF Data Assimilation

14. Ensemble of data assimilation • 25 members of 2 inner-loop 4D-Var’s at T95/159 L137, T399 outer lops • Perturbations from observations, SST, SPPT; noise filtering, scaling

15. DA future strategy at ECMWF • Hybrid EDA & 4D-Var • Long window 4D-var with weak constraint. • 24h weak constraint beats 24h strong constraint. • More work needed on weak constraint to beat current operational 12h strong constraint. • Scalability of 4D-var is an issue – but we anticipate solvable, at least for the time being (time-parallel saddlepoint formulation). • Theoretically the longer the window the better, but devil is in the detail (formulating model error!).

16. M. Bonavita EDA is a very skilful system, but it is expensive. How well do cheaper alternatives to error cycling compare? An EnKF-based error cycling system without perturbed observations could be a cheaper alternative Other options being investigated • One among many options: If we have EnKF and 4D-var we can blend the Kalman gain – we call this “hybrid-gain” xa=xb +(αKEnKF + (1- α)K4DVar)(y-Hx)

17. Recent satellite DA highlights

18. Recent highlights and progress in Satellite DA Externally funded (12 people) • Evaluation of Chinese FY3A/B/C (2014) • Skin temperature analysis (2014, 40R2) • RTTOV-11 (2014, 40R2+3) • Improved MW emissivities(2014, CY40R3) • Improved use of AMVs (2014, CY40R3) • 2D RO operator (2014, CY40R3) • MW observation error model (2015) • Improved aerosol detection for IASI, CrIS and AIRS (2015) • Improved MW imager assimilation e.g. cold air outbreaks (2015) • PC assimilation demonstrated for cloud-free scenes (2016+) • New ideas to exploit correlated error, rather than just allow for ir (2016+) ECMWF funded (6 people) • All sky MHS assimilation (2014, CY40R3) • Metop + S-NPP instruments (2014) • IVER, ODB-IFS, cycle testing (2014, CY40R3) • High profile case studies and OSEs e.gHaiyan, Sandy (2014) • Improved understanding of observation impact diagnostics (2014) • Observation error correlations for hyperspectral instruments (2015) • All sky IR assimilation (2016)

19. S. Healy Improvement of the 2D approach in 40R3 ray path Interpolate 2D information to the ray path Tangent height of the ray-path determined by the impact parameter provided with the observation, . surface The outer loop uses 31 profiles to describe the 1200 km “occultation plane”. 7 profiles used for inner loop.

20. S. Healy Impact of tangent point drift (37R2) and the 40R3 2D operator

21. M. Rennie Aeolus wind product status • Launch mid 2016? • 2014: L2B processing facility: prototype → operational • Summer 2014: ESA ground Segment testing started

22. N. Bormann Updated R for IASI and AIRS (new σO + correlations vs operational σO without correlations) Better background fit to other observations over the tropics, especially for humidity-sensitive observations: Humidity Temperature

23. N. Bormann Updated R for IASI and AIRS (2) S.Hem. N.Hem. Normalised RMSE differences over 6 months: Aug-Oct 2013; Jan-Mar 2014, (vs own analysis) New worse New better Forecast day Forecast day Tropics New worse New better Forecast day Forecast day Forecast day

24. Fastem – development history Full model Integration into RTM Fast model Dielectric properties As full model Surface roughness Fast fit to full model RTTOV, CRTM etc. As full model Directional anisotropy As full model Whitecapping V1 V2 V3 V4 V5 V6 Fastem-1: English and Hewison 1998: Created for 20-90 AMSU-A. Fastem-2: Deblonde and English 2002: Extend to MW imagers. Fastem-3: English 2007: Extend to polarimetric imagers. Fastem-4: Liu et al. 2011: Extend to 1-20 GHz and 90-200 GHz. Fastem-5: Liu et al. 2012: Fixed some weaknesses identified in Fastem-4 by users. Fastem-6: Kazumori and English 2014: Fixed anisotropy model in Fastem-5

25. M. Kazumori (JMA) Fastem-6: more accurate ocean emissivity for RTTOV-11/CY40R3 + 2 QJ papers June 20 to October 3 = Fastem-5 = Fastem-6 Use of the Ocean Surface Wind Direction Signal in Microwave Radiance Assimilation Masahiro Kazumori and Stephen J. English Asymmetric features of oceanic microwave brightness temperature in high surface wind speed condition Masahiro Kazumori, Akira Shibata, and Stephen J. English

26. Recent OSEs

27. Whilst the impact of a single instrument may appear small, when we test multiple instruments together, like all the Metop-B instruments, the benefit is clear. S Healy A. McNally -2% to +2% -2% to +2% Metop-B IASI+AMSU-A +MHS+GRAS+ASCAT Metop-B IASI only Long term gains in forecast skill can arise from many small changes. We should not always expect to get measurable medium range positive impact from adding new observations.

28. T. McNally OSE 500z (NH-24h)

29. T. McNally OSE 500z (SH-24h)

30. T. McNally 72hr Tropics 200hPa VW

31. T. McNally OSE 500z (NH-24h) v FSO (NH-24h)

32. C. Lupu Do results of OSE and FSO disagree ? RMSE(IASI) minus RMSE(NO-IASI) RMSE(IASI*) minus RMSE(NO-IASI)

33. Summary and thoughts • Forecast skill continues to improve rapidly. • By 2016 90+ satellite instruments processed operationally – a peak? • Most satellite DA is achieved through collaboration (e.g. NWPSAF, CMA-ECMWF partnership for FY3, Horizon-2020, Copernicus…). • New instruments: Radar, lidar, L-band, Limb sounders all demonstrating impact. Who will pay for an operational service? • Handling cloud and rain affected satellite data will become mainstream – potential being realised, this is the way forward. • System is robust to losing ANY observation type, even the most important. But note in situ data remains important. • ECMWF still thinks 4D-var has a bright future, but we are evaluating other approaches.

35. R. Engelen Atmospheric Composition • IFS code has been extended with full chemistry, aerosols, and greenhouse gases -> “C-IFS” • Strong reliance on ageing satellite systems: ENVISAT (failed), EOS AURA (OMI & MLS), MOPITT, MODIS • Metop, Sentinel-5p, -4 and -5 therefore very important for sustaining and improving current capabilities • Frequently retrievals are assimilated Global mean CO analysis profiles Without AK Use of Averaging Kernel in data assimilation removes sensitivity to prior information used in retrievals With AK

36. P de Rosnay, J Muñoz Sabater, C Albergel Land Surface NH 1000 hPa Z Marine, land surface and atmospheric composition also need high quality satellite observations though lack of mature tools means retrievals are often assimilated: however we now know better how to assimilate retrievals. Snow cover preprocessing and 24 to 4 km resolution improvement SMOS brightness temperature assimilation: soil moisture increments Root zone soil moisture analysis from EKF assimilation of ASCAT

37. Mike Fisher 24h weak vs 24h strong 24h weak-constraint (red) significantly better than 24h strong-constraint in NH. Small improvement also in SH. Verification is against own analyses.

38. M. Fisher 24h weak vs 12h strong 24h weak-constraint (red) remains worse than 12h strong-constraint in NH. Similar scores in SH (although some degradation at short range). Verification is against own analyses.

39. M. Bonavita T399 EnKF (100 member) T95/T159/T399 4DVar with static B T399 Hybrid Gain EnKF (100 member) Z500 hPa AC - NHem Z500 hPa AC - SHem

40. S. Healy 2D impact at short-range (winter experiment) SH NH Clear improvement in GPS-RO statistics of ~ 5 % in troposphere. More importantly, the 2D framework enables us to investigate further improvements include more physics.

41. M. Rennie Winds assimilated in an ECMWF cycle • Very uneven distribution • AMV coverage good in tropics, but obs errors large • Stratosphere poorly sampled log10(number obs per area)

42. M. Matricardi Assimilation of PC scores derived from 305 IASI channels Verification against radiosondes: Temperature in the Tropics Forecast rms errors for the 850 hPa relative humidity in the Tropics RAD PC_CORR RAD PC_CORR

43. Hurricane Sandy

44. T. McNally Hurricane Sandy

45. T. McNally Hurricane Sandy

46. T. McNally Analysis differences that led to failed (NO –LEO SAT) forecast Control minus NO-LEO SAT MSLP 2012102500z (after 5 days of data denial)