CMAS Conference, October 16 – 18, 2006

1. Exploring Approaches to Integrate Observations and CMAQ Simulations for Improved Air Quality Forecasts C. Hogrefe1,2, W. Hao1, K. Civerolo1, J.-Y. Ku1, and G. Sistla1 1New York State Department of Environmental Conservation 2Atmospheric Sciences Research Center, State University of New York at Albany CMAS Conference, October 16 – 18, 2006 The work presented here was performed by the New York State Department of Environmental Conservation with partial support from the U.S. EPA under cooperative agreement CR83228001. The views expressed in this paper do not necessarily reflect the views or policies of the New York State Department of Environmental Conservation or those of the U.S. EPA.

2. Introduction • Overview of NYSDEC/EPA/NOAA air quality pilot study and past model performance • Description of potential approaches to improve air quality predictions • Results • Summary and Outlook

3. Overview of Pilot Study • Establish partnership between NYSDEC, NOAA and EPA in the area of numerical air-quality prediction • Complements the NOAA/EPA national air quality forecasting activities • Project period: January 2005 – December 2007 • Apply and evaluate a state-of-science photochemical modeling system on an ongoing basis with special emphasis on PM2.5 predictions over New York State • Assess the potential usefulness of grid-based photochemical models to provide O3 and PM2.5 forecasts across New York State • Archive daily concentrations fields for potential use in air quality / health studies

4. Simulation Setup • Meteorology/Emissions: Based on NCEP/NWS48-hr ETA (prior to June 2006) or WRF (since June 2006) forecasts initialized at 12:00 UTC and 2002/2004/2005 emission inventory processed with PREMAQ • Photochemical Modeling: CMAQ (version 4.5.1) • Horizontal resolution: 12 km • Vertical resolution: 22 layers, lowest layer ~40 m • Simulation Periods: • July – September, 2004 • January – March, 2005 • June 2005 – present

5. Model Performance over NYS: The Not-So-Good … Fractional Bias (FB) as defined in Morris et al. (2005) for 24-hr average total PM2.5 predictions at all FRM monitors located in New York State calculated for 2004 - 2005 Observed and Predicted 24-hr Average Speciated PM2.5 at Eight STN Monitors in NYS (Upstate Rural, Upstate Urban, NYC Metro), July – September 2004 Hogrefe et al., 2006, JAM, in press

6. … and the promising Correlation Coefficients for Time Series of Daily Maximum 8-hr O3 Comparison of Predicted Air Quality Index Tendencies for CMAQ (left), Routine Expert-Based Forecasts (center), and Trend Persistence Correlation Coefficients for Time Series of Daily Average PM2.5 Hogrefe et al., 2006, JAM, in press

7. CMAQ forecasts often have significant biases, especially for PM2.5 • On the other hand, CMAQ simulations show skill in capturing temporal trends in air quality --> Explore approaches to integrate observations and CMAQ predictions for improved air quality forecasts

8. Potential Approaches for Combining CMAQ Simulations and Observations for Air Quality Forecasts Simple Bias-Correction Prediction – “Adjustment 1” “Bias-Adjustments” Binned Bias-Correction Prediction – “Adjustment 2” Simple CMAQ-Tendency Prediction – “Adjustment 3” Variability-Adjusted CMAQ-Tendency Prediction – “Adjustment 4” “Tendency-Adjustments” Slope-Adjusted CMAQ-Tendency Prediction – “Adjustment 5”

9. Time Period and Domain of Analysis • June – September, 2005 • Focus on monitors in New York State • Note: Methods 1-2 and 4-5 rely on incorporating observations not just for the current day but for an extended time period: • In a routine forecast setting, this extended time period could be the past week, month, or season • In this study, we utilized the fixed time period from June 1 – September 30 also used for evaluating these methods, i.e. for any given day, both past and future observations were included • Future analysis will consider the impact of the choice of the “calibration” or “learning” period over which the adjustment parameters in methods 1-2 and 4-5 are calculated on the performance of these approaches

10. Methods of Comparison • Focus on daily maximum 8-hr O3 and daily average PM2.5 • Comparison of observed and simulated pollutant concentration distributions • RMSE (total, systematic, and unsystematic) • Categorical metrics as defined in Kang et al. (2005) for thresholds of 84 ppb (O3) and 40 ug/m3 (PM2.5) • False Alarm Ratio (FAR) • Probability of Detection (POD) • Critical Success Index (CSI)

11. Unsystematic RMSE is determined by the distance between the datapoints and the linear regression best-fit line Total RMSE is determined by the distance of the data points from the 1:1 line Systematic RMSE is determined by the distance between the the linear regression best-fit line and the 1:1 line Illustration of Total, Systematic, and Unsystematic RMSE

12. Distributions of Daily Max. 8-hr O3 (left) and 24-hr Av. PM2.5 (right) from Observations, CMAQ Predictions, and Adjustment Methods 1 – 5 • Methods 3-5 yield closer agreement with observed distributions than either the unadjusted CMAQ simulations or methods 1-2

13. Total, Systematic, and Unsystematic RMSE of Daily Max. 8-hr O3 • Methods 1 and 2 (the “bias-adjustment” methods) significantly reduce total RMSE, mostly by reducing the systematic RMSE • Methods 3 -5 (the “tendency-adjustment” methods) generally show little improvement in terms of overall RMSE. While they strongly reduce the systematic RMSE, they increase the unsystematic RMSE

14. Total, Systematic, and Unsystematic RMSE of Daily Average PM2.5 • For PM2.5, the “binned-bias-adjustment” yields the largest reduction of total RMSE • Similar to daily maximum 8-hr ozone, the “tendency” adjustment approaches reduce the systematic RMSE but tend to increase the unsystematic RMSE

15. False Alarm Ratio (FAR), Probability of Detection (POD), and Critical Success Index (CSI) For Daily Average PM2.5 Above a Threshold of 40 ug/m3 • Consistent with the narrower density functions shown before, the “bias-adjustment” methods reduce both FAR and POD, while the “tendency-adjustment” methods increase both FAR and POD • As a result, the overall improvement in CSI over the original CMAQ simulations is relatively small for all methods

16. Percentage of Stations at Which a Given Adjustment Method Performed Best for a Given Metric and Pollutant • The “bias-correction” approaches 1 and especially 2 work best for reducing the total RMSE at most sites for both O3 and PM2.5 • The “tendency-correction” approaches often work best for improving the CSI, especially for O3

17. Summary and Outlook • Motivated by past CMAQ forecast evaluation results, we tested five potential approaches for providing improved air quality forecasts based on both observations and CMAQ simulations. • While the “bias-correction” approaches 1 or 2 work best for reducing the total RMSE at most sites, the approaches that combine today’s observations with unadjusted or adjusted CMAQ-predicted temporal changes often work best for improving the CSI, especially for O3 • Moreover, the best adjustment method to improve the CSI, which measures the quality of categorical forecasts, needs to be chosen on a pollutant-by-pollutant and station-by-station basis • Other studies explored more advanced techniques. For example, Delle Monache et al. (2006) and Kang et al. (2006) describe the application of a Kalman filter to generate improved air quality forecasts and report good success. • Additional methods might aim at including spatial correlation structures into the model adjustment algorithm rather than relying solely on temporal structures at individual monitors. Such analyses will be performed in the future.