Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems

1. Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems Yanqiu Zhu Contributions from: John Derber, Andrew Collard, Dick Dee, Russ Treadon, George Gayno, Jim Jung, Emily Liu, Min-Jeong

2. Part I: The enhanced radiance bias correction scheme • Part II: New emissivity predictor to handle large land-sea differences with the new CRTM (will be presented at the 6th WMO Symposium on Data Assimilation) • Part III: Radiance bias correction for all-sky radiance bias correction

3. Part I: Enhanced radiance bias correction scheme Yanqiu Zhu John Derber, Andrew Collard, Dick Dee, Russ Treadon, George Gayno, Jim Jung • The original radiance bias correction scheme • The motives to enhance the original scheme – the goals for the enhanced scheme • Methodology for the enhanced scheme • Experiment setup and results • Conclusions

4. Why radiance bias correction – to generate unbiased analysis • the error associated with the instrument and calibration • the error of the radiative transfer model • Inconsistencies among the assumptions of data usage and model as well as algorithms Derber et al. (1991), Harris and Kelly (2001), Dee (2004), etc

5. Original radiance bias correction scheme Air-mass dependent in GSI Separate scan-angle dependent A two-step procedure where is radiative transfer model is predictor term with predictor coeff. is pre-specified parameter The control variables and are estimated by minimizing (Derber et al., 1991, Derber and Wu, 1998)

6. What we expect to achieve with the enhanced radiance bias correction scheme • Combine the scan angle and air-mass biascomponentsinside the GSI variational framework. • Eliminate the duplications and potential risk of opposite drifting between the two components • Remove the pre-specified parameter for each predictor, so avoid the trial-and-error procedure for any new bias predictors • Apply a new pre-conditioning to the predictor coefficients taking into account of observation contribution, speeding up the convergence

7. What we expect to achieve with the enhanced radiance bias correction scheme (cont.) Radiance bias correction is based on the data passing the quality control Quality control is performed on the bias corrected data Reasonable radiance bias correction estimate need to be manually derived prior to the use of any new radiance data or during the re-analysis process with the original scheme

8. We can use any new radiance data with initial radiance bias correction set to be zero What we expect to achieve with the enhanced radiance bias correction scheme (cont.) • Automatically detect any new/missing/recovery of radiance data and initialize new radiance data; • Quickly capture any changes in the data and the system. • A new approach of specifying the background error variances for the predictor coefficients • An new initialization procedure for any new radiance data • Add capability of bias correction for passive channels

9. Original scheme: a two-step procedure Enhanced scheme: a one-step procedure • is computed outside GSI, a running average of OMF • The predictors of air-mass are computed inside the GSI • Variational angle bias correction, is computed inside the GSI along with other predictor terms • satang file no longer needed • Special attention to the initialization of bias coefficients: using quality-controlled data from previous run; or mode of all data

10. Change of the Preconditioning Current preconditioner Modified preconditioner • Hessian w.r.t. predictor coeff. of the cost function Modified block-diagonal preconditioning is set to be the inverse of (Dee 2004) For simplicity, only diagonal elements of are considered at each analysis cycle

11. Background variance for predictor coeff. Currently scheme Enhanced scheme More & high quality data Smaller variance • Set to be the estimate of analysis error variance of the coeff. from previous analysis cycle • New data: is initialized to be 10000.0 • Missing data: is twice as much as that of the previous analysis cycle Automatically adjust variances of coeff.

12. At the end of analysis,minimizing the functional where or If adp_anglebc=.t. Bias correction for passive channels (passive_bc=.true.) • Make preparation for assimilating passive channels data • Aid quality control of other channels

13. Experiment configuration • High Resolution: T574L64 3DVAR and T254L64 EnKF with 80 ensemble members • 0.25 for static B and 0.75 for ensemble • Two-way hybrid coupling • EnKF relies on the GSI to perform the radiance bias correction CTL: prda141b, current radiance bias correction RBC : prda141br, enhanced radiance bias correction Experiment period: 00Z July 5, 2012 to 00Z Aug. 16, 2012 Spin-up period: 00ZJuly 5, 2012 to 18Z July 13, 2012

14. Convergence comparison 00Z July 15, 2012

15. Variance vs. Observation number A big drop of obs number prompts a jump of variance AIRS CH22

16. Global offset and total bias terms: AMSUA-NOAA18

17. OMF with and without bias correction RBC CTL Without BC With BC AMSUA-N18, CH4 00Z July 10, 2012

18. Mean Temperature Analysis & Difference 700hpa

19. HGT 500hpa NH Anomaly Correlation

20. HGT 500hpa SH Anomaly Correlation

21. RMSE Wind Vector 200hpa Tropics

22. 24h & 48h Forecasts Fits to Obs

23. Conclusions • One-step bias correction procedure • Improved convergence rate • Made it easier to add new bias predictors • Added the capability to automatically detect any new/missing/recovery of radiance data • New radiance data can be used without providing prior bias correction estimate • Neutral forecast skills in NH and Tropics, better in SH • The scheme was also tested on NDAS/NAM • Currently is used in the wind energy project (Jacob’s talk at 6th WMO Symposium on Data Assimilation)

24. Part III: Radiance bias correction with all-sky radiance assimilation Yanqiu Zhu, Emily Liu, Min-Jeong Kim, Andrew Collard, John Derber Two other talks on all-sky radiance assimilation at NCEP: Emily Liu’s and Min-Jeong Kim’s talks

25. Bias correction for Operational clear radiance • Angle bias correction • Air-mass bias correction with five predictor terms: global offset, a zenith angle term, cloud liquid water, lapse rate, and the square of the lapse rate. All-sky radiance assimilation • Control variable: total cw/normalized cw/normalized RH • State variables: Hydrometeors ql, qi, (qr, qs, qg, qh) • The cloud liquid water predictor is removed • Observation error based on averaged CLW • Quality control: cloud filtering is removed • Linearized moist physics

26. Issues of radiance bias correction with all-sky radiance assimilation • Different error characteristics between clear-sky and cloudy radiance • Different cloud information between radiance observation and first guess • O:clear vs. F:clear • O:clear vs. F:cloudy  eliminate cloud • O:cloudy vs. F:clear  add cloud • O:cloudy vs. F:cloudy Ideally, Gaussian distributed (O:clr,F:clr) & (O:cld,F:cld), with one hump ((O:clr,F:cld), (O:cld,F:clr)) on each side

27. Issues of radiance bias correction with all-sky radiance assimilation (cont.) OmF before BC OmF after BC Over bias corrected (O:cld,F:cld) and (O:clr,F:clr) Lots of (O:cld, F:cld), too fewer other categories

28. Issues of radiance bias correction with all-sky radiance assimilation (cont.) OmF before BC OmF after BC METOPA AMSUA Ch. 15

29. Issues of radiance bias correction with all-sky radiance assimilation (cont.) (O:cld, F:cld) OmF before BC OmF after BC METOPA AMSUA CH. 15

30. Inclusion of the (O:clr,F:cld) and (O:cld,F:clr) in bias correction • Radiance data with large OmF but small obs error • What may have hurt the radiance bias correction • Experiments are underway, and results will be presented later at the GSI meeting Experiment strategies: • Bias correction coeff. are retrieved using only (O:clr,F:clr) and (O:cld,F:cld); Fixed previous coeff. are used to calculate the bias corrected OmF for (O:clr,F:cld) and (O:cld,F:clr). • Variational quality control