Overall Goals SQWG

1. OverviewaboutnewprocessorbaselineH. Bovensmann, G. Lichtenberg, R. Snel, C. Lerot, T Fehr (on behalf of SQWG)ENVIVAL-LIFE Meeting, 6.12.2010, Bremen

2. Overall Goals SQWG • The main goal for the SQWG is establishing L1 and L2 high data quality for the operational data products of the full SCIAMACHY data set, which allows substantial contributions to long term atmospheric data sets in the areas of: • the development of the O3 layer • global air pollution • Global Greenhouse gas observations • Solar-Terrestrial interactions • In addition, the data SCIAMACHY data set shall allow to investigate new scientific methods and new products.

3. SCIAMACHY QWG • SCIAMACHY Quality Working Group • Inter-Agency Activity: ESA-DLR-NSO • Maintenance and development of the ESA operational SCIAMACHY processors and tools • Level 1b / Level 2 • SCICAL • SciaL1C • Established 2006 • Extension 2011-2012 currently prepared • SQWG Statement of Work currently finalised (2011/12) • SCIAMACHY ESL contract: Proposal received from DLR (2010-2012) • Coordinator: Heinrich Bovensmann

4. SQWG Team IUP: S. Noel, K. Bramstedt, P. Liebing, M. Buchwitz, C. von Savigny, A. Richter, A. Rozanov, F. Wittrock, M. Vountas, A. Kokhanovsky, K. U. Eichmann, M. Weber, J.P. Burrows, H. Bovensmann SRON: R. van Hees, P. van der Meer, J.M. (Matthijs) Krijger, R. Snel, I. Aben BIRA: C. Lerot, M. van Roozendael KNMI: L.G. (Gijsbert) Tilstra, P. Stammes DLR IMF: G. Lichtenberg, A. Doicu, S. Gimeno Garcia, F. Schreier, M. Meringer, D. Sherbakov, S. Slijkhuis, B. Aberle, K. Kretschel, S. Hrechanyy, M. Gottwald ESA: T. Fehr SERCO: A. Dehn, G. Brizzi

5. Operational SCIA Processors activated Feb. 2010 • Level 1 IPF 7.04 compliant with ENVISAT 2010+ mission extension • No product format changes • No algorithm changes wrt IPF 7.03 • CFI 5.8.1 implemented • Level 2 Off-Line Processor SGP 5.01 • Product format substantially changed. • Additional trace gases: • Nadir columns of SO2, BrO, H2O, OClO and CO and BrO Limb profiles • Limb Cloud indicators for cloud presence and type • Improved Absorbing Aerosol Index. • Level 2 “Fast-Delivery” processing with SGP 5.01 (<24h) at D-PAC based on Level 1b NRT and predicted ADFs

6. Level 1 v7

7. Level 0-1 processor (IPF L1 v7) • Stray light matrix channel 2 introduced • Writing of limb mesosphere states to limb MDS introduced • On-board error of writing scan encoder values corrected in processor

8. SciaL1c • Version 2.1 delivered October 2008 • New: • M-factor handling • Bug fixes • ASCII output • Build environment improved

9. Degradation Correction: M-Factor’s • L1b Version 7 m-factors: • NRT: • Quasi-operational forward-processing V7 running since Feb 2010 at IUP. • Variable limits in m-factor quality checks implemented by ESA • M-factor plots are included in the bi-monthly report • Reprocessing: • L1b monitoring data 08/2002 – 01/2010 completely provided by DLR. • M-factors for V7 re-processing are complete and delivered to ESA. • Documentation, m-factor data base, plots & applicator s/w are also available via m-factor website: http://www.iup.uni-bremen.de/sciamachy/mfactors • M-factors for L1b version 6.0x (08/2002 – 01/2010), still available too.

10. Level 2 v5

11. Level 1b-2 Version 5 – Product Overview(Blue: New, Green: Improved) Nadir UV/VIS: O3, NO2, BrO(BIRA, SDOAS/GDP4) SO2 (volcanic & anthropogenic), OClO(IUP, algorithm using SDOAS/GDP4 formalism), H2O(IUP, AMC DOAS) Nadir SWIR CO(IMF, BIRRA) Nadir Clouds & AAI: Cloud Top Height & Optical Thickness (IUP, SACURA) Nadir Cloud Fractions (IMF, OCRA) Absorbing Aerosol Index (KNMI) Limb: O3, NO2, BrOProfiles (IMF: operational version, IUP: retrieval settings) Cloud flagging(IUP, SCODA)

12. Nadir Products

13. Nadir O3 total columns SCIAMACHY OL 3.01- reference (%)‏ • Algorithm Heritage: SDOAS (BIRA) • Basic algorithm settings as L2 v3.01 • Retrieval settings: • Fit window: 325 – 335 nm • Daily calibrated ESM irradiance spectra (D0). • Radiances fully calibrated. • M-factors. • Cloud parameters from OCRA/SACURA. • Expected quality / known issues: • Small temporal trend (up to 0.5%/year), minimised by applying m-factors at least in the tropics. Impact of m-factors on total O3 (%) See also C. Lerot et al., Atmos. Meas. Tech., 2, 87-98, 2009

14. O3: SCIAMACHY trend m-factor impact • Trend with respect to GOME1: • systematic trend of -4%/decade for nearly all latitudes w.r.t. GOME • This trend does not dependent on algorithm (see Lerot et al., 2009, Loyola et al., 2009) • m-factor corrections improves situation in the tropics (see also Bramstedt et al., 2009) • But: m-factors do not completely remove the observed trend in SCIAMACHY total ozone • L1: -all calibrations on • -solar reference (ESM diff.) • L1 w./m-factor: • -all calibrations on, but no etalon M. Weber

15. Nadir NO2 total columns • Algorithm Heritage: SDOAS (BIRA) • Basic algorithm settings as L2 v3.01 • Retrieval settings: • Fit window: 426.5 – 452.5 nm • Daily not calibrated ASM irradiance spectra (A0). • Radiances extracted without radiometric calibration (no m-factors)  Higher fit quality and equivalent NO2 columns • Cloud parameters from OCRA/SACURA • Expected quality / known issues: • To be validated …

16. Nadir BrO total columns • New implementation • Algorithm Heritage: SDOAS (BIRA) • Slant column retrieval as SGP/SDOAS • AMF calculation uses new BrO profile stratospheric climatology developed by BIRA used (Theys et al. ACP 2009)‏ • Retrieval settings: • Fit window: 336 – 351 nm • Daily not calibrated ASM irradiance spectra (A0) • Radiances extracted without radiometric calibration (no m-factors) • Cloud parameters from OCRA/SACURA • Expectedquality / knownissues: • small difference in VCDs of about 2x1012molec cm-2 on averagebetween the reference algorithm and operational processor • Stratospheric AMF not representative for trop. enhanced BrO

17. Nadir OClO slant columns Note: OClO only present in chlorine activated polar vortex at SZA > 90° • New implementation • Algorithm Heritage: • Slant column retrieval as SGP/SDOAS • Settings from IUP • Retrieval settings: • Fit window: 365 – 389 nm • Daily not calibrated ASM irradiance spectra (A0) • Radiances extracted without radiometric calibration (no m-factors) • SZA cut-off = 95° • Expectedquality / knownissues: • Small but relatively constant low bias of 3x1013molec cm-2 • polynomial degree will be adapted to minimise bias

18. Nadir SO2 total column • New implementation • Algorithm Heritage: • Slant column retrieval as SGP/SDOAS • Retrieval and AMF settings from IUP • Retrieval settings: • Fit window: 315 – 327 nm • Daily not calibrated ASM irradiance spectra (A0) • Radiances extracted without radiometric calibration (no m-factors) • SCD post-processing step: reference sector subtraction • daily subtraction of reference sector SCD (Pacific) to compensate for SZA and ozone sensitivity of retrieval etc. • SCD is converted to VCD using simulated AMF’s for • “anthropogenic pollution” • “volcanic eruption” scenarios

19. SO2: Volcanic Columns • Globally, there are several artefacts: • Strongly negative values at high latitudes • Sea – land contrast • Very large values in winter high latitudes • Some scattered high values • Artefacts in IUP Bremen product have similar pattern but much smaller values => Product under revision SAA SAA

20. Nadir H2O total column • New implementation • Algorithm Heritage: • Air Mass Corrected (AMC)-DOAS (Noël et al., 2007) • Retrieval settings: • Fit window: 688 – 700 nm • Daily not calibrated ASM irradiance spectra (A0) • Radiances calibration: memora effect, leakage, wavelength (no m-factors) • Key product entries: • Vertical column density (VCD), error in VCD, air mass factor (AMF) correction • Expected quality / known issues: • Average bias -0.2 g/cm2 w.r.t. ECMWF • Precision estimate (w.r.t. ECMWF) • 3 - 5% for columns >0.6 g/cm2 • small columns: up to 10% • Absolute precision always better than 0.2 g/cm2

21. Nadir CO column • New implementation • Algorithm Heritage: • BIRRA (direct non-lin. least square fit to radiance, Schreier et al. 2009) • Retrieval settings: • Fit window: 2324 – 2335 nm (channel 8) • Expected quality / known issues: • highly sensitive to DBPM • Over sea signal too low, numerical effects can play a role • Most likely use case: Means over many orbits • Mean values will depend on filtering of data • Current version: not to be used • Algorithm and settings under revision

22. Limb Products

23. Limb Changes • The followingchangesweremadetotheLimbretrievals • New ProductLimbcloudflags • New ProductBrO Profile • LimbRetrievalsrecognisethecloudheightfromcloudflagproduct, i.e. theretrievalstartsatthefirstcloudfreeheightfoundabovetheminimumheight • Note: smalltrend in pointingknowledgeover time mayaffecttrends in all limbproducts – undercontinuesinvestigation

24. V4 V3.01 Limb O3 Profile • Optimised Retrieval • Algorithm Heritage: • Optimal Estimation (IMF, Doicu et al.) • Optimisation of retrieval settings IUP • Retrieval settings: • Spectral range 520 – 590 nm • Fine altitude grid (33 layers instead 24) • 4th order polynomial • Picard iteration • Limb radiance ratio w.r.t. 46 km TH • LOWTRAN aerosol model (backgroundconditions) used in RTM • Retrieval starts above clouds using cloud parameters from SCODA • Expectedquality / knownissues: • Bias as obseved in v3.01 O3 should be much smaller • Clouds have impact on O3 around tropopause • Validation ongoing

25. Limb O3 improvements vs. LIDAR L2 v3.01 L2 v4.4 L2 v5 Comparison: Anne Gijsel, RIVM via ESA VALID / EQUAL project

26. Limb NO2 Profile • Optimised Retrieval • Algorithm Heritage: • Optimal Estimation (Doicu et al. • Optimisation of retrieval settings IUP • Retrieval settings: • Optimised settings • Fine altitude grid (33 layers instead 24) • Picard Iteration • Extended spectral range 420 – 470 nm • Reduced regularization parameter • Limb radiance ratio w.r.t. 43 km TH • Cloud parameters from SCODA • Retrieval starts above clouds • Expected quality / known issues: • To be validated

27. Limb BrOProfile • New Retrieval • Algorithm Heritage: • Optimal Estimation (Doicu et al. • Optimisation of retrieval settings IUP • Retrieval settings: • Spectral range 337 – 357 nm • Limb radiance ratio w.r.t. 35 km TH • Cloud parameters from SCODA • Retrieval starts above clouds • Expected quality / known issues: • deviations on average up to 40% were found above with a large standard deviation • For not too large SZA‘s and in certain latitude bands on average difference < 20% • Some „data gaps“ – under investigation

28. Limb Cloud Flagging • New Product • Algorithm Heritage: • SCODA using colour index ratios to identify strong scattering layers in limb (IUP, Eichmann et al.) • New Limb MDS introduced: • FLAG: 0 = nocloud 1 = patiallycloudy 2 = fullycloudy 3 = baddata • MAXVAL: max. colorindexratio, • MAXHEIGHT: tangentheight (in km) where MAXVAL isfound • MAXHEIGHTIDX: indexof MAXHEIGHT • eachforthree different cloudtypes • WCL: normal waterclouds, • ICL: iceclouds • PSC: polar stratosphericclouds. • Expectedquality / knownissues: • To be validated

29. Summary/Outlook

30. From Level 2 V 3.01 to V 5.01

31. From Level 2 V 3.01 to V 5.01

32. Preliminary Validation • SCIAVALIG Meeting 07/08 September 2010, BIRA • In general good agreement of the SCIAMACHY products and available validation data • Ozone Limb profiles are clearly improved • New products are considered of good or sufficient good quality • SO2 slant and anthropogenic columns show product deficits. • CO column retrievals need an optimisation • SCIAMACHY Level 2 reprocessing interrupted • Improved baseline delivery expected February 2011 • OClO, CO and SO2 improvements • ESA/SQWG decision to postpone reprocessing re-start until availability of corrected baseline • SCIAMACHY delta-Validation Report made available:http://envisat.esa.int/earth/www/object/index.cfm?fobjectid=7279

33. Level 2 Version 5.0x (1. Quarter 2011) • Nadir OClO: bug-fix in degree of polynomial will reduce bias in SC • Nadir SO2: bug-fix in reference sector data base routine • Nadir CO: bug-fix in • Limb O3: impact of aerosol a-priori under investigation • Limb BrO gaps under investigation

34. Level 1 version IPF version 8 (2011/12) Work in progress ... • Changes (Verified/ To Be Verified) • Memory Effect Limb improved • Non-Linearity PMD scaling noise check introduced • Hot Pixel Check Limb improved • Polarisation for occultation switched off • SAA check for etalon switched off • Selection of individual dark states • Stray light matrix channels 3-8 • Key-data and scan angle dependent degradation

35. Next Baseline: Level 1b-2 Version 6 (2011/12) New in version 6 (Verified) • Porting to 64bit + current versions of libraries and compilers • AAIA improvements (use Ozone columns from retrieval) • Nadir Snow/Ice separation • VCD HCHO • VCD CHOCHO • Trop. NO2 (limb-nadir) • Improved CO • XCH4 or XCO2 (tbc) • NLC detection Limb • Ozone Profile extension to lower Mesosphere

36. Outlook: ESA‘s „atmospheric“ missions ESA Atmospheric Mission 2nd half 2013  mid-2015 (to be confirmed)  12/2013 ? 2017 ?   06-07/2011 ! 2018 ?  12/2014  2018  2020  Figurecourtesy T. Fehr, ESA 04/2013  Weneedtolobbyfor a missionextensionbeyond 2014 !

37. Coming soon: The SCIAMACHY „Biography“

38. The End