Progress in Joint OSSE Simulation of observation for calibration Preparation for DWL OSSE

1. Progress in Joint OSSE Simulation of observation for calibration Preparation for DWL OSSE Michiko Masutani NOAA/NWS/NCEP/EMC JCSDA , Wyle IS

2. Main contributors in 2010 Michiko Masutani[1,2,#], Jack S. Woollen[1,+], Zaizhong Ma[2,$], Lars Peter Riishojgaard, [2,$,5], Dave Emmitt[6], Sid Wood[6], Steve Greco[6], Tong Zhu[3,@], Haibing Sun[3,%], Nikki Prive[4,@], Yuanfu Xie[4],  Ellen Salmon[5] , Bill McHale[5], Thomas J. Kleespies[3], Guan Ping Lou[1+], FangLin Yang[1,+] Ronald Vogel[3,+], Diane. Stokes[1], Kate. L. Howard[1,+], Fuzhong Weng[3], Yong Han[3] [1]NOAA/National Centers for Environmental Prediction (NCEP) [2]Joint Center for Satellite and Data Assimilation (JCSDA) [3]NOAA/ NESDIS/STAR,[4]NOAA/Earth System Research Laboratory (ESRL) [5] NASA/GSFC ,[6]Simpson Weather Associates # Wyle Information Systems, McLean, VA, ,+IM Sciance Group)IMSG), MD $Goddard Earth Science and Technology Center, University of Maryland, Baltimore, MD, @Cooperative Institute for Research in the Atmosphere (CIRA)/CSU, CO %Perot System Government Services, Virginia OSSE:Observing Systems Simulation Experiments http://www.emc.ncep.noaa.gov/research/JointOSSEs/ Acknowledgement The nature runs for Joint OSSEs were produced by Dr. Erik Andersson of ECMWF. We appreciate GMAO to provide initial satellite data for calibration at ESRL. GMAO also provided code to add random error to simulated data.

3. Observing Systems Simulation Experiments (OSSE) A Nature Run (NR, proxy true atmosphere) is produced from a free forecast run using the highest resolution operational model which is significantly different from NWP model used in DAS. Data Assimilation Systems (DAS) Process initial condition from observational data OSSE Evaluate observing systems and data assimilation systems using simulation experiments.

4. The Purpose of Data AssimilationAndrew S. Joneshttp://www.emc.ncep.noaa.gov/research/JointOSSEs/references/Jones_Data_Assimilation_Ed_Forum.ppt • Why do data assimilation? • I want better model initial conditions for better model forecasts • I want better calibration and validation (cal/val) • I want better acquisitionguidance • I want better scientificunderstandingof • Model errors (and their probability distributions) • Data errors (and their probability distributions) • Combined Model/Data correlations • DA methodologies (minimization, computational optimizations, representation methods, various method approximations) • Physical process interactions (i.e., sensitivities and feedbacks)Leads toward better future models

5. The Data Assimilation CommunityAndrew S. Joneshttp://www.emc.ncep.noaa.gov/research/JointOSSEs/references/Jones_Data_Assimilation_Ed_Forum.ppt • What skills are needed by each involved group? • NWP Data Assimilation Experts (DA system methodology) • NWP Modelers (Model + Physics + DA system) • Application and Observation Specialists (Instrument capabilities) • Physical Scientists (Instrument + Physics + DA system) • Radiative Transfer Specialists (Instrument config. specifications) • Applied Mathematicians (Control theory methodology) • Computer Scientists (DA system + OPS time requirements) • Science Program Management (Everything + $$ + Good People) • Forecasters (Everything + OPS time reqs. + Easy/fast access) • Users and Customers (Could be a wide variety of responses)e.g., NWS / Army / USAF / Navy / NASA / NSF / DOE / ECMWF

6. Full OSSEs • A Nature Run (NR, proxy true atmosphere) is produced from a long free forecast run using the highest resolution operational model which is significantly different from NWP model used in DAS. • Calibrations will be performed to provide quantitative data impact assessment. • Without calibration quantitative evaluation of data impact is not possible. • Data impact on analysis and forecast will be evaluated. • A Full OSSE can provide detailed quantitative evaluations of the configuration of observing systems. • A Full OSSE can use an existing operational system and help the development of an operational system

7. International Joint OSSE capability • Full OSSEs are expensive • Sharing one Nature Run and simulated observation save the cost • Share diverse resources • OSSE-based decisions have international stakeholders • Decisions on major space systems have important scientific, technical, financial and political ramifications • Community ownership and oversight of OSSE capability is important for maintaining credibility • Independent but related data assimilation systems allow us to test robustness of answers

8. Joint OSSE Data Nature Run Spectral resolution : T511 13 month long. Starting May1st,2005 Vertical levels: L91, 3 hourly dump Daily SST and ICE: provided by NCEP Andersson, Erik and Michiko Masutani 2010: Collaboration on Observing System Simulation Experiments (Joint OSSE), ECMWF News Letter No. 123, Spring 2010, 14-16. Copies are available to designated users for research purpose& users known to ECMWF User list is maintained by Michiko Masutani (NOAA/NCEP) contact:michiko.masutani@noaa.gov Complete Nature run data set is posted from NASA/NCCS portal http://portal.nccs.nasa.gov/osse/index.pl Password protected. Accounts are arranged by Ellen Salmon (Ellen.M.Salmon@NASA.gov)

9. Shared Joint OSSE data NASA/NCCS provided dis space for Joint OSSE data shirring There is a entry created for Joint OSSE http://portal.nccs.nasa.gov/josse/index.pl Make entry to each data set and generating institute, and contact person. People use these data must contact generating institutes.

10. Data posted by NCEP and NESDIS • Four months of simulated radiance data by NCEP and NESDIS • with and without cloud • bufr and ascii format • template data and DBL91 which contain NR profile at foot print • (ASCII file and DBL91 are provided to help development and verification) • Simulated SBUV Ozone Retrievals by Jack Woollen(NCEP) (four month) • Simulated conventional data by Jack Woollen (13 month) NCEP will post simulated observation as progress. First we post from NCEP ftp site (rzdm) to be evaluated. After the data is evaluated it will be transferred to NASA NCCS portal. We appreciate any help in evaluation of the simulated observations. Note. Radiance data are also being simulated by Environment of Canada using RTTOV. GMAO is working on calibrated radiance data

11. [Plans ] Complete 13month of radiance observation has been simulated and being evaluated. Ascii data will not be posted for new data but software to conert bufr to ascii data will be provided. Prepbufr for real data will be posted after sensitive data removed radiance data for 2009-2010 satellite. [consideration] Sampling strategies for further radiance data RTM used for simulation. RTTOV is available in reserch version of CRTM Observational error to be added Correlated observational error by Tom Kleespies Non Gaussian error distribution Ozone may need to be completed for 13 month.

12. Data posted from Joint OSSE Home page http://www.emc.ncep.noaa.gov/research/JointOSSEs/ Preliminary simulation which need to be verified Description of data set posted from NASA/NCCS/portal Software used to simulation Simulated TC vital Michiko Masutani (NCEP/EMC Guan Ping Lou (NCEP/EMC) Radiance Simulation codes Software used to simulate radiance data posted from NCCS portal CRTM used for simulation. CRTM1.2.2 (Different from the version posted from JCSDA website) Presentation for verification by Tong Zhu and Haibing Sun Link to libraries requres Sample data set Software to simulate IASI, Metop data has been tested and sample is posted [Simulation of TC vital] TC vital was simulated using software originally written by Tim Marchock and currently developed by Guan Ping Lou of NCEP. TC-vital for 13 month The software are posted 15OAS-P199

13. OSSE Calibration ● In order to conduct calibration all major existing observation have to be simulated. ●The calibration includes adjusting observational error. ●If the difference is explained, we will be able to interpret the OSSE results as to real data impact. ●The results from calibration experiments provide guidelines for interpreting OSSE results on data impact in the real world. ●Without calibration, quantitative evaluation data impact using OSSE could mislead the meteorological community. In this OSSE, calibration was performed and presented. 15OAS-P199

14. Calibration at NCEP and ESRL • Calibration at NOAA/ESRL with NCEP • Using simulated data by GMAO and additional data from NCEP. • Focused on July-August 2005. • GSI version May 2007. • Some comparison between NCEP and GSO radiance data with and without observational error. NCEP radiance give about 0.5-1% higher AC in day 3. GMAO Obs error reduce AC for about 1-2% in day 3. Real Simulated Calibration at NCEP FY10 NCEP GDAS system Obsevational error is not added for simulated data Data denial experiment for RAOB wind showed simulated wind without observational error has about 1.5 more impact compared to real data.

15. Preparation toward OSSE for GWOS DWL  DWL data will be assimilated as Line Of Sight (LOS) wind component. Two LOS observation like U and V must produce very similar impact as one vector wind U and V. i.e. Two LOS observation at close location by GWOS will produce a vector wind like impact.  Clean up code and remove some mishandling of the data improve the performance at polar region.  Up grade code to handle various type of DWL data  Upgrade code are submitted as NCEP subversion to be reviewed for implementation 15OAS-P199

16.  Variational quality control improve the performance in polar region. (The figure shows a zonally averaged improvement V on day three. Cint=0.05 and maximum is about 0.5m/s in both poles.) 0.5  Sensitivity to various constant to observational error is noted. Current assignment seems to be too small Tuning factors Gross error, representativeness error Measurement error included the data Observational error assigned to data Response function in GSI

17. Note on Data impact depend on forecast skill metric Evolution of GFS Forecast Skill S. Lord and Fanglin Yang (NCEP/EMC) 500 hPa Anomaly Correlation Does It Make a Difference toHow Forecasters Use Product? NH NCEP Percent Poor Forecasts SH Percent good Forecasts • “But these are just ‘width of the line’ improvements…” • Are these differences in scores important to users? • Can more sensitive measures of forecast performance be developed? NCEP Yang, Fanglin 2011: 24th Conference on Weather and Forecasting [Available online at http://ams.confex.com/ams/91Annual/webprogram/Paper182605.html] 15OAS-P199

18. Zonally and time averaged improvement by DWL compared to CTL Reduction of the distance from the Nature run U V Analysis Averaged from forecast date July 7 to August 15. The impact start from upper level tropics and spread to midlatitude Some negative value in summer hemisphere is noted. However case study showed positive impact of DWL. 1 day fcst 2 day fcst 3 day fcst 4 day fcst 5 day fcst SH NH Countour interval is doubled for U 15OAS-P199

19. Max distance from Nature run in 72 hr fcst Mean sea level pressure Control With DWL MSLP of 72 hour fcst. Coutour is NR Fcst time 00Z August 3rd verified at 00Z August 6 NH AC drop off on 00z August 3rd forecast in control but not with DWL Location of storm agree with Nature run when DWL added

20. Time series of 500hPa geopotential height AC Zaizhong Ma et al (LWG Miami, February 2011) NH SH Day 1 Candidates for additional study Day 3 00Z August 3rd Day 5 02/08/2011 Working Group on Space-based Lidar Winds, Miami 20

21. Summary The Joint OSSE Nature runs have been evaluated, posted and made available to research community NCEP data assimilation system is ready for simulation of DWL in different characteristics. Four month of simulated radiance (used in 2005), and ozone data at NCEP-NESDIS are posted from NASA/NCCS portal. Radiance data for complete 13 month period is being evaluated, The complete 13month long simulated conventional observation were been posted from NASA /NCCS portal. TC vital was simulated and posted from NCEP rzdm to be evaluated. Global OSSEs using Joint OSSE Nature run are conducted at NCEP-JCSDA-NESDIS-SWA (DWL) NASA/GMAO (ADM) NOAA/ESRL (UAS) Environment of Canada (PCW) MRI, Japan starting 15OAS-P199

22. Remark on Joint OSSEs Using Full OSSE, various experiments can be performed and various verification metrics can be tested to evaluate data impact from future instruments and data distributions. Theoretical predictions have to be confirmed by full OSSEs. The results are often unexpected. OSSE results also require theoretical back ups. Evaluation Data Assimilation System though OSSE is drawing many interest Many interest from Regional OSSEs Lack of appreciation of amount of work discourage the collaboration 15OAS-P199