Presentation Outline

1. The Development of PM2.5 Forecasting Tools for Ten Citiesin the MARAMA Region2005 National Air Quality ConferenceSharon Douglas, ICF Consulting/SAI andBill Gillespie, MARAMAFebruary 15, 2005San Francisco, CA

2. Presentation Outline • Origin of the PM2.5 forecasting project • Technical approach • Data used in the CART analysis • CART results • CART-based forecasting tool • Evaluation • Limitations, lessons learned, and future work

3. Origin of the PM2.5 Forecasting Project • MARAMA members had strong ozone forecasting programs, but little or no experience forecasting PM • MARAMA states were observing relatively high PM2.5 values in some areas • There was increasing interest in fine particulate pollution because of its adverse human health impacts • EPA initiated year-round air quality forecasting

4. Origin of the PM2.5 Forecasting Project Counties designated nonattainment for PM2.5 2001-2003 Design Values Nonattainment County Partial Nonattainment County

5. Technical Approach • The Goal: • Forecast tomorrow’s Air Quality Index (AQI) Level (AQI color category), and • Estimate tomorrow’s 24-hour average PM2.5 concentration for ten cities in the MARAMA Region

6. Technical Approach Nine original forecast areas in the MARAMA Region. An additional forecasting tool is being developed for Allegheny County, PA

7. Technical Approach • General approach for each forecast area: • Collect and quality assure historical meteorological and PM2.5 monitoring data for the forecast area • Analyze these data using Classification and Regression Tree (CART) software, using 24-hour average PM2.5 concentration as the classification variable • Develop a forecast algorithm from the CART results • Create a forecasting tool (computer software) based on the algorithm

8. Technical Approach • An important requirement for tool development was: …consider only variables forecasters would have data for on the day the forecast is prepared • Input parameters included: • Yesterday’s 24-hour average PM2.5 concentration (often from several locations) • Forecasted surface and upper air meteorological data from the Eta, GFS or NGM models for the day being forecast

9. Data Used in the CART Analysis • Examples of meteorological surface variables (for the day being forecast) • Daily maximum surface temperature  • Daily minimum surface temperature  • Average relative humidity based on temperatures and dew point temperatures • Number of 6-hour periods with rainfall greater than 0.1 inches • Average wind direction • Average wind speed

10. Data Used in the CART Analysis • Examples of upper air variables • 850 mb temperature   • Stability parameters: 950, 900, & 850 “Delta T” – temperature difference between 950, 900, & 850 mb and the surface • Wind speed at 850 mb and other heights • Wind direction at 850 mb and other heights • Cloud variable based on relative humidity

11. CART Results CART tree for Philadelphia, PA

12. Forecasting Tools Three data input buttons. Each button changes color after data have been entered Allows a user to automatically input surface and upper air data from a text file.

13. Forecasting Tools

14. Forecasting Tools

15. Forecasting Tools Forecast results. Screen changes color to match AQI color Probability of other AQI colors in the same forecasted bin Uncertainty information. Other bins the tool would select if met. variables are adjusted by small or large amounts

16. Evaluation Procedures • Real-time evaluation • February/March 2004 • CART tools applied by state/local forecasters on a daily or nearly daily basis using forecasted meteorological parameters and observed air quality data • Results compared to • continuous measurements • FRM measurements (several months later)

17. Evaluation Procedures • Historical evaluation • June/July/August 2003 • “Forecasts” prepared using actual/historical input data • Results compared to FRM measurements

18. Evaluation Procedures • Evaluation statistics include • Percent correct • False alarm rate (for USG days) • Probability of detection (for USG days) • Critical success index • Bias • Contingency matrix

19. Evaluation Procedures • Evaluation considered • Observed category match (strict) • Neighboring category match if observation is within 10 percent of the value defining the categories (fuzzy)

20. Evaluation Procedures Evaluation tool (developed by MDE)

21. Evaluation Results: Feb/Mar 2004 Continuous Data

22. Evaluation Results: Feb/Mar 2004 FRM Data

23. Evaluation Results: Jun/Jul/Aug 2003 Prob. of Detection Range: 50-100% Bias for USG Days: 1 – 5 gm-3

24. Evaluation: Key Findings • Real-time evaluation inconclusive due to lack of USG days • Use of FRM rather than continuous data increases accuracy for Richmond & Wilmington and decreases accuracy for Charlotte & Baltimore • Borderline days, if not carefully considered, can account for a 10 percent reduction in forecast accuracy

25. Evaluation: Key Findings • Historical evaluation indicates that, under ideal conditions, the CART tools are able to correctly predict approx. 60 to 70 percent of the days (70 to 85 percent if fuzzy evaluation is used) • Somewhat worse performance for Bristol & Roanoke (which have only 3-in-1 data) indicates that the more limited data sets are incomplete in representing the types of conditions that occurred in Summer 2003 • Relatively high FAR, reflects CART’s tendency to classify days into higher PM categories (and perhaps difficulty representing transition from high to lower PM concentrations)

26. Limitations of the Current Tools • Tools are based on 3-years of data with few USG days, and thus the full range of meteorological/PM2.5 conditions that may occur in the future may not be represented • Tools were developed using FRM PM2.5 measurements while forecasters must rely on continuous measurements • Uncertainties or errors in the forecasted meteorological parameters will affect the accuracy of the PM forecasts

27. Assessment & Areas for Future Work • CART-based PM2.5 forecasting tools are a useful addition to the forecaster’s toolkit • Possible updates and improvements include: • Incorporate additional years of data into the CART analysis • Add multi-day forecast capabilities • Areas for further research include: • Use current-day PM2.5 information in preparing the forecasts • Evaluate and compare the use of the different meteorological forecast products • Use speciated (STN) data to examine species distributions among the CART bins and classification categories

28. Contact Information • Bill Gillespie, Mid-Atlantic Regional Air Management Association, (410) 467-0170 or bgillespie@marama.org • Sharon Douglas, ICF Consulting/SAI, (415) 507-7108 or sdouglas@icfconsulting.com • The report, “The Development of PM2.5 Forecasting Tools for Selected Cities in the MARAMA Region” is available at: http://www.marama.org/Projects/