Evaluation of a Variable Time-step Algorithm in CMAQ

1. Evaluation of a Variable Time-step Algorithm in CMAQ Talat Odman and Yongtao Hu School of Civil & Environmental Engineering Georgia Institute of Technology 3rd Annual CMAS Models-3 Conference October 19, 2004 Georgia Institute of Technology

2. Objective Short Term: • To cut CMAQ runtime in half • Without investment in new machinery • Without machine/compiler dependent programming tricks • But through a major algorithmic change • Do this with minor impact on model results and no impact on conclusions derived from CMAQ • e.g., effectiveness of a control strategy Long Term: • To unleash the power of Adaptive Grid Modeling Georgia Institute of Technology

3. Time Stepping in CMAQ • At present, CMAQ uses a global “synchronization time step” (DT) • Different processes interact once every DT • There is also an “advection time step” (Dt) that must satisfy the following condition: • Not only because explicit advection schemes require it (CFL condition) • But to avoid “skipping” grid cells and emissions in those cells • The user defines upper and lower bounds for DT • CMAQ determinesDT which is the smallest integer multiple of Dt (ideally, DT = Dt) Georgia Institute of Technology

4. Opportunity for Speedup • At present, the maximum wind speed ( umax) determines Dt and DT • As of Version 4.3 the user can select a model layer over which wind speeds are not considered in determination of DT. This way, strong winds aloft do not dictate unnecessarily short time steps in lower layers. • However, high wind speeds in one part of the lower layers (e.g., over water) can still dictate unnecessarily small time steps in other parts of the domain. • This can be circumvented by a “variable time-step algorithm” Georgia Institute of Technology

5. Analysis of Wind Fields • The wind fields were analyzed over the 36- and 12-km VISTAS grid domains during the July-99 and Jan-02 episodes. • The winds are highly non-uniform over any horizontal slice. • More variation over the 36-km grid than the 12-km • More variation in July-99 episode than Jan-02 • Largest wind speeds are observed over water. Georgia Institute of Technology

6. Horizontal Variation in Wind Speed: Surface Layer Georgia Institute of Technology

7. Wind Speed Distribution Georgia Institute of Technology

8. Georgia Institute of Technology

9. VARTSTEP: the VARiable Time-STEP algorithm implemented in CMAQ • Current implementation is two-dimensional (2-D) with some extension for 3-D • Every vertical column is assigned its own local synchronization time step • which satisfies the CFL condition with the maximum wind speed for that column (from layer 1 to ADVLAYR) • which is an integer multiple of the global synchronization time step and an integer divisor of 1 hour (e.g., if the global time step is 5 minutes, the local time step can be 5, 10, 15, 20, 30, or 60 minutes) Georgia Institute of Technology

10. VARTSTEP (Continued) • The clock is advanced by the global time step • Process routines are called (and concentrations are updated) only when it is time to synchronize, and for the duration of the local time step • Horizontal transport is tricky • A cell which is not ready for synchronization can still receive fluxes from its neighbors with shorter local time steps. • Therefore, horizontal fluxes (advective and diffusive) are stored in “reservoirs” to be “flushed” when it’s time to synchronize • This increases memory requirements (hey! nothing is free!) Georgia Institute of Technology

11. Speedup and Change in Results • Results presented here are from the simulation of the January 1-4, 2002 period over the 12-km VISTAS grid • 35% shorter run time was achieved. • Time spent in aerosol module is reduced by 2/3 • Time spent in chemistry module is reduced by 1/4 • Time spent in vertical diffusion is reduced by more than 1/2 • Ways of speeding up horizontal transport were not explored • Spatial distributions (PAVE plots) are from the peak hours • Comparisons of daily averages are at Class-1 areas Georgia Institute of Technology

12. Difference in NO3- CMAQ VARTSTEP-CMAQ Georgia Institute of Technology

13. Class-1 Areas Georgia Institute of Technology

14. Georgia Institute of Technology

15. Georgia Institute of Technology

16. Georgia Institute of Technology

17. Georgia Institute of Technology

18. Georgia Institute of Technology

19. Georgia Institute of Technology

20. Georgia Institute of Technology

21. Georgia Institute of Technology

22. Some Reasons for the Differences • Meteorological variables used in CMAQ are at the middle of the global time step while those used in VARTSTEP are not necessarily at the middle of the local time step • Emission rates are constant during each hour (step function) in VARTSTEP while they vary linearly in CMAQ • In CMAQ, RADM cloud module operates on the concentrations at the middle of the hour. In VARTSTEP, the same module operates on concentrations at the end of the hour • A mass conservation error was discovered in CMAQ and corrected in VARTSTEP • This will be presented later by Dr. Yongtao Hu Georgia Institute of Technology

23. Conclusion • Using the VARTSTEP algorithm, CMAQ run time is reduced by 35% • With some additional work, it can be halved • More memory is required • An FGRID array that has the same size as CGRID is needed to hold horizontal fluxes. • Change in PM concentrations are small • Impact on model performance is not evaluated yet • Impact on conclusions drawn from the model is not evaluated yet • Both of these impacts are expected to be negligible Georgia Institute of Technology

24. Future Work • Investigate how CMAQ results are affected by the relation between characteristic times of different processes and the synchronization time step? • Currently the synchronization time step is set equal to the advection time step • With VARTSTEP much smaller time steps can be used • Incorporate (dynamic) adaptive grid modeling into CMAQ • The global time step dictated by the smallest grid size was too short (order of seconds) resulting in unacceptable run times for practical use. • With VARTSTEP algorithm this is no longer a limitation Georgia Institute of Technology

25. Snapshot of a Continuously Adapting Grid • From “Meteorological Research Needs for Improved Air Quality Forecasting, ” Dabberdt et al., BAMS vol. 85, no.4, pp. 563-586, April 2004. Georgia Institute of Technology

26. Acknowledgement • Visibility Improvement State and Tribal Association of the Southeast (VISTAS) • Georgia Department of Natural Resources (DNR) • Fall line Air Quality Study (FAQS) principal investigators Dr. Michael Chang and Dr. Ted Russell Georgia Institute of Technology

27. Difference in O3 CMAQ VARTSTEP-CMAQ Georgia Institute of Technology

28. Georgia Institute of Technology