A Demonstration of the National Air Quality Forecast Capability

1. A Demonstration of the National Air Quality Forecast Capability To Provide Local Forecast Guidance for the Air Quality Index Brian Eder1, Daiwen Kang2, S. T. Rao1, Rohit Mathur1, Shaocai Yu2, Tanya Otte1, Ken Schere1, Richard Wayland3 , Scott Jackson3, Paula Davidson 4, Jeff McQueen 4 and George Bridgers5 1 National Exposure Research Laboratory, U.S. Environmental Protection Agency, RTP, NC 2 Science and Technology Corporation, Hampton, VA 3 Office of Air Quality Planning and Standards, U.S. Environmental Protection Agency, RTP, NC 4 National Weather Service, National Oceanic and Atmospheric Administration, Silver Spring, MD 5 Division of Air Quality, NC Department of Environmental and Natural Resources, Raleigh, NC

2. Motivation Demonstrate the efficacy of the NAQFC in providing AQI forecast guidance to local forecasters, thereby encouraging its use as a forecast tool.

3. Motivation Demonstrate the efficacy of the NAQFC in providing AQI forecast guidance to local forecasters, thereby encouraging its use as a forecast tool. Outline ►Brief NAQFC Background ● Evaluation of 2007 and 2008 ►Evaluation Adaptation ●Convert [ O3 ] to AQI ●Change Paired “Grid cell – monitor” to MSA wide evaluation ►Demonstration for the Charlotte MSA ●Compare NAQFC performance with local forecast ► Results for other MSAs across the Nation

4. National Air Quality Forecast Capability Uses the North American Mesoscale (NAM) meteorological model run of the Weather and Research Forecasting, Nonhydrostatic Mesoscale Model, (WRF-NMM) configured with Community Multiscale Air Quality (CMAQ) Model. ►Emissions are processed using SMOKE based on the 2001 NEI (w/ updates) ►Version 4.2 Carbon Bond chemical mechanism. ►22 layers extending from the surface to 100 hPa. ►Initial condition chemical fields are specified using the previous forecast cycle. The target forecast period is local midnight through local midnight. An additional eight hours are required beyond midnight to calculate peak 8-h average O3 conc., as a result: a 48-h NAM–CMAQ forecast is needed (based on a 12 UTC initialization) to obtain the desired 24-h forecast period.

5. CONUS Domain http://www.nws.noaa.gov/aq/

6. Recent NAQFC Evaluations for 2007*, 2008 • The purpose of these evaluations is twofold: • to identify problem areas with either the meteorological or CMAQ models, thereby providing critical information to the model developers; • to support, in part, the NAQFC’s transition from experimental status to operational status during each major upgrade and expansion. • The evaluations use: • ►Hourly O3 concentrations from nearly • 1100 monitors from the AIRNOW network. ►Discrete statistical metrics (i.e. MB, NMB, RMSE, NME) associated with forecasts of maximum 8-hour O3 concentrations. • ►Observations that are matched/compared directly to grid cells (i.e. no interpolation nor smoothing is used) • * Results from the 2007 NAQFC evaluation will soon be published in Atmospheric Environment.

7. Evaluation Discrete Forecast / Evaluation Observed vs. Forecast Max. 8 hr. Concentration Example of strict grid-cell to monitor matching CONUS Forecasts for the Summer (J, J, A)

8. Evaluation Adaptation

9. Evaluation Adaptation Conversion of O3 concentrations (ppb) to AQI* In order to demonstrate the NAQFC’s ability to provide guidance to local and State forecasters, we must talk “your language”, i.e. the AQI, not ppb. Where: Ip = the index for pollutant p (O3 in this case) Cp = the rounded concentration of pollutant p BPHi = the breakpoint that is ≥ Cp BPLo = the breakpoint that is ≤ Cp IHi = the AQI value corresponding toBPHi ILo = the AQI value corresponding toBPLo * The AQI’s for this demonstration are calculated based on the old standard. Results using the new AQI will be presented at the end of the presentation.

10. Example NAQFC Output – [O3] ppb

11. “Adapted” NAQFC Output – AQI

12. Evaluation Adaptation In addition to changing the model output in this demonstration, we also need to “rethink” the way in which we evaluate the performance of the NAQFC, in order to be more consistent with the local forecaster, who: generally forecast the maximum AQI value that they expect to occur anywhere within their area of responsibility (i.e. MSA); and then: verify this forecast with the maximum monitored value within that area. ►As an example of this adaptation, we will provide an in-depth evaluation for the Charlotte MSA, for one day (August 15, 2007). ● The Charlotte MSA is comprised of 8 counties, 7 in NC, 1 in SC. ● There are 8 AQS monitors in those counties, 7 in NC, 1 in SC. ● The MSA is represented by 103, 12-km grid cells by the NAQFC.

13. Evaluation Adaptation

14. Evaluation Adaptation This modified, somewhat more relaxed evaluation approach results in “improved” statistics when compared to the more rigid observation vs. grid cell approach. ●We will demonstrate this approach using both: [O3]AQI

15. Current Evaluation Approach The statistics below are based on using all8 monitors in the Charlotte MSA with the monitors matcheddirectly with their grid cell. Period: 1 May – 30 Sept. 2007 n = 1207 MB = 2.1 ppb; NMB = 3.3% RMSE = 12.0 ppb; NME = 16.8% r = 0.63

16. Modified Evaluation Approach - Step 1 The statistics below are based on the max. of the 8 monitors and the max. of 8 grid cells in the Charlotte MSA, where monitors are not matched with their grid cell. Period: 1 May – 30 Sept. 2007 n = 153 MB = -0.8 ppb; NMB -1.1% RMSE = 10.5 ppb; NME = 11.8% r = 0.73

17. Modified Evaluation Approach - Step 2 The statistics below are based on the maximum of the 8 monitors and maximum of all 103 model grid cells in the Charlotte MSA, where the monitors are not matched with their grid cell. Period: 1 May – 30 Sept. 2007 n = 153 MB = 3.1 ppb; NMB = 4.4% RMSE = 11.0 ppb; NME = 13.1% r = 0.74

18. Modified Evaluation Approach - Step 3 We can take the last approach and convert the concentrations to AQI values. Period: 1 May – 30 Sept. 2007 n = 153 MB = 6.7; NMB = 9.3% RMSE = 22.0; NME = 25.1% r = 0.74

19. NAQFC Performance compared with Human Forecast Summer 2007

20. NAQFC Performance compared with Human Forecast* Summer 2007 * Provided by NC Department of Environmental and Natural Resources

21. NAQFC Performance compared with Human Forecast Daily

22. NAQFC Categorical Performance vs.Human Forecast • Category Hit Rate: where i is the AQI index (1, 2, 3, 4, 5) category or the color scheme (green, yellow, orange, red, purple), and is the forecast instances in the i th category and is the number of observed instances in the i th category. • Exceedance Hit Rate: • where N fo is the number of both observed and forecast • exceedances (AQI ≥ 3), Nois the number of observed, but not forecast exceedances. • Exceedance False Alarm Rate: • where Nf is the number of forecast but not observed exceedances (AQI ≥3), Nfois the number of both observed and forecast exceedances.

23. NAQFC Categorical Performance vs.Human Forecast Charlotte MSA - Summer 2007 Category Hit Rate Exceedance Hit Rate Exceedance False Alarm Rate

24. NAQFC Categorical Performance vs.Human Forecast In addition to Charlotte, the performance of the NAQFC was compared to human forecast from four other MSA’s • Atlanta (Georgia Department of Natural Resources) • Dallas (Texas Commission on Environmental Quality) • Houston (Texas Commission on Environmental Quality) • Washington DC (Metropolitan Washington Council of Governments)

25. NAQFC Categorical Performance vs.Human Forecast Category Hit Rate

26. NAQFC Categorical Performance vs.Human Forecast Exceedance Hit Rate Exceedance False Alarm Rate Human NAQFC Because the NAQFC is positively biased, it tends to capture a higher percentage of exceedance hit rates, but this also results in a higher percentage of false alarm rates.

27. New AQI Standard On March 12, 2008, EPA strengthened the NAAQS for ground-level ozone, and changed Air Quality Index (AQI) breakpoints to reflect the new primary standard. The new 100 level of the ozone AQI is 75 ppb, 8-hour average, which is the level of the revised primary standard.

28. NAQFC Categorical Performance Results for new AQI Standard Exceedance Hit Rate Exceedance False Alarm Rate Old AQI New AQI Because the NAQFC tends to better forecasts O3 values in the middle of the distribution, its performance improves with the tighter standard based on 75 ppb

29. Summary The purpose of this work was to demonstrate the efficacy of the NAQFC in providing AQI forecast guidance to local forecasters, thereby encouraging its use as a forecast tool. The model often performed quite well at 5 MSAs across the nation. ●This was especially true for the higher AQI values. ●This performance will improve further when providing guidance associated with the new AQI values. This research was recently submitted to the Bulletin of the AMS.

30. Summary When compared to the evolution of numerical weather prediction’s incorporation into weather forecasting, the success of this program has been realized in a remarkably short time frame. It is important to remember that the NAQFC is intended to provide guidance to, but certainly not replace the local air quality forecaster. Just as with weather forecasting, the reliability of an AQI forecast made at the local level will almost always be greater than one made at the National level.

31. Thank you Acknowledgements: Rob Gilliam, David Wong, Hsin-mu Lin, Daniel Tong, Jeff Young, Annemarie Carlton, Jerry Herwehe. Georgia Department of Natural Resources Texas Commission on Environmental Quality Metropolitan Washington Council of Governments Disclaimer: The research presented here was performed under the Memorandum of Understanding between the U.S. Environmental Protection Agency (EPA) and the U.S. Department of Commerce's National Oceanic and Atmospheric Administration (NOAA) and under agreement number DW13921548. This work constitutes a contribution to the NOAA Air Quality Program. Although it has been reviewed by EPA and NOAA and approved for publication, it does not necessarily reflect their policies or views.