Progress on accelerating radiative transfer calculations using PCRTM Xu Liu

1. Progress on accelerating radiative transfer calculations using PCRTM Xu Liu Z. Jin, H. Li, W. Wu, B. Wielicki, M. G. Mlynczak, D. F. Young, S. Kato, F. Rose, R. Baize, A. Larar, and D. K. Zhou…. NASA Langley Research Center P. Yang Texas A & M University Bryan Baum and W. L. Smith University of Wisconsin

2. Presentation outline • Motivations • How does PCRTM works? • Status on thermal infrared PCRTM • Code structure and functionality • Speed benchmark • Status on the solar PCRTM • Ocean training • Land training • Summary and Conclusions

3. Motivations Hyperspectral data has thousand of channels Traditional sounders have dozens of channels Typically large number of spectral channels discarded in variational retrieval due to computational speed limitations Current NWP centers only use 4-10% of data Traditional radiative transfer models may be too slow for large data volume Time consuming to simulate instantaneous CLARREO spectra for OSSE study Different GCM models Long term record Different sceneries Need to explore spectral correlations in the hyperspectral data PCA has been used to reduce instrument noise and to compress spectra Only the first ~50-100 leading eigenvectors are used for optimal fingerprinting Leading EOFs captures all essential information of thousands of channels PCRTM - Principal Component-based Radiative Transfer Model Uses PCA to compress information content and speed up the RT model calculations

4. How does PCRTM work? • Principal Component-based Radiative Transfer Model (PCRTM) • Model PC scores (Y) instead of channel radiances (R) • PC scores (super channels) are linearly related to channel radiances • The relationship is derived from the properties of eigenvectors and instrument line shape functions: • PCRTM parameterization is physical-based fast model • Channel-to-channel spectral correlations are captured by eigenvectors • Reduce dimensionality of original spectrum by a factor of 10-90 • Radiative transfer done monochromatically at minimal essential frequencies • Very accurate relative to line-by-line (LBL) RT model ( < 0.05K in IR or <~0.2% in solar) • 3-4 orders of magnitude faster than LBL RT models • A factor of 2-900 times faster than channel-based RT models • Provide radiative kernel needed for retrievals and climate studies

5. Flow chart for the PCRTM model Flow diagram of the PCRTM forward model • Regular Monochromatic RT performed • This function can be performed by LBL, MODTRAN… • Can use lookup table and fast parameterization if needed • RT needs to be done at minimum number of frequencies • Orders of magnitude smaller than LBL and MODTRAN Input PCRTM Parameters (OD, PCs, cloud Parameters, etc.) Data Generate predictors by Performing mono RT calculations Next Profile Calculate PC scores and Jacobian EOF Transformation No Channel Radiance? • O2 A-band (12000 mono RT calculations using LBL) • Only 7 Mono frequencies are essential when modeling at the OCO and SCIAMACHY spectral resolution Yes

6. Typical accuracy of the forward model ( < 0.03 K relative to LBL) • Bias error relative to LBL is typically less than 0.002 K • The PDF of errors at different frequencies are Gaussian distribution • RMS error less than 0.03K • Large ensemble of spectra used in the training • Independent validations perform well

7. Status on thermal Infrared PCRTM • Thermal IR PCRTM code has been completely rewritten • Can handel different sensors without re-compile the code • Currently 10 sensor IDs (see tables on next slide) • Modular design • Memory dynamically allocated • Can use subset of channels • Can use multiple instrument at the same time • Have options to calculate: • PC scores • PC scores + PC Jacobian • PC scores + channel radiances • PC scores + PC and channel Jacobians • Includes: • 15 variable trace gases • Can add multiple scattering clouds at any atmospheric layers • Very fast speed • See table on next page • Available upon request • The Inversion algorithm using PCRTM model has been developed and tested • Applied to real hyperspectral data • AIRS, IASI, CrIS, NAST-I ….

8. PCRTM computational speed benchmark • A benchmark PCRTM computational speed has been performed • Intel(R) Quad core Q9550 CPU, 2.83GHz, Linux system • No parallelization performed

9. PCRTM has been successfully used on real hyperspectral data (CrIS, IASI, AIRS, NAST-I) • PCRTM has been successfully used to perform forward radiative transfer modeling and geophysical parameter retrievals of the following on orbit instruments: • Metop-A IASI • NASA Aqua AIRS • Suomi NPP CrIS • NAST-I (NASA airborne instrument)

10. Example of CO retrieval sensitivity study and global CO retrieved from real IASI data • Notice that the feature near 2020-2250 cm-1 are removed when CO profile is explicitly retrieved in the inversion algorithm

11. Status on the solar PCRTM • The training is almost complete • Ocean with water and ice clouds • Ocean with multiple cloud layers • Land with water and ice clouds • Still need to train under multiple cloud layers over land • Have trained PCRTM under several spectral resolutions • 1 cm-1 spectral resolution from 300-2500 nm • 4 nm spectral resolution from 300-2500 • 0.045 nm spectral resolution for oxygen A band • 0.2 nm spectral resolution for oxygen A band • More works needed • Add more perturbations to the land BRDF • Explore with stratified surface conditions (snow, ice, vegetated land etc ..)

12. Training PCRTM in Solar Spectral Region • MODTRAN used as the RT model for training • Both ocean and land surface included • Both ice and water clouds included • Cloud layers varies from 0-3 • 1-17 Ks are used for gas absorptions and the code is modified to output the radiances at each k-node and the weighted-average • 12-16 stream used in DISORT calculations • Spectral range: 300-2500 nm with Δν=1cm-1 • 259029 monochromatic RT needed per spectrum • The goal is to reduce the points to 300-1000 • First train PCRTM at 1 cm-1 resolution • 23531 spectral channels • 771 mono RT needed (ocean example) • A factor of 340 less RT calculation relative to MODTRAN • Train PCRTM at CLARREO RS resolution • 4 nm spectral spacing (546 spectral channels) • 282-300 mono RT needed • A factor of ~900 faster relative to MODTRAN The following parameters are randomly varied in the MODTRAN runs

13. 1 cm-1 spectral resolution PCRTM (ocean example) Example of TOA radiance spectra and RMS errors on depend and independent data set ( < 0.002 mW/cm2/sr/cm-1)

14. CLARREO resolution PCRTM (Ocean) • Convolve 1 cm-1 spectral resolution MODTRAN spectra to CLARREO resolution • 300 nm to 2500 nm (covers the RS spectral region 320 – 2300 nm) • 546 channels • Only 145 EOFs • 282 predictors • 918 fold speed-up relative to MODTRAN

15. The fitting accuracy has been validated using independent data sets (ocean)

16. Relative Errors are small unless radiance are are less than 0.001 mW/(cm.sr.cm-1) • Relative errors are typically smaller than 0.5% at most wavelengths • Relative errors are larger at ~1380 nm and ~1850 nm • Due to strong water absorption • TOA radiances at these wavelength are close to zero for some cases • large solar zenith angles, large water column, low surface BRDF ….. • The absolute errors are comparable to other wavelengths • The absolute errors are smaller than instrument noise • Remove radiances close to zero will decrease the relative error dramatically

17. A detailed look at the relative error near 1874 nm • The large relative errors are due to small radiance • Left plot • Not important as compared to instrument noise • Removing small radiances decrease relative error dramatically • See plot on the right • As we set different threshold to remove small radiances, the relative errors drop dramatically

18. CLARREO resolution PCRTM (Land) • 200 EOFs • 300 mono RT needed • 862 fold speed-up relative to MODTRAN

19. MODTRAN land surface modeling (3) • Using the RossThickLiSparseReciprocal BRDF model

20. Land Surface BRDF treatment

21. Relative errors are small except when the radiances are less than 0.001 mW/(cm.sr.cm-1)

22. Summary and Conclusions • A progress report has been given to the PCRTM development • PCRTM is a useful tool specific for hyperspectral data with thousands of channels • PCRTM is very fast • PCRTM is very accurate relative to the training RT model • PCRTM IR code has been re-written to provide more uniform support for all hyperspectral sensors • Model has been developed for AIRS, NAST,IASI, CLARREO, and CrIS • Used for both forward model simulation and data inversions • Good feedbacks from other science team members • Work on extending PCRTM to UV-VIS-near IR spectral region is going well • Future plan • More forward model training in the solar spectral region • Couple PCRTM with MODTRAN to provide a faster solar RT calculation • Implement faster cloudy radiative transfer calculations • Validate model using real data • Interact with STM team members and science community to improve the model