Meaningful Fire Service Research: A Role for Metro Chiefs

1. Meaningful Fire Service Research:A Role for Metro Chiefs

2. PURPOSE: To demonstrate the usefulness of existing Metro Department data in research examining fire service operational efficiency & effectiveness

3. BACKGROUND Based on discussions during the National Fire Service Data Summit in Tempe, AZ in January 2011, many participants believed there is not an ability to effectively utilize existing fire service data to answer important questions regarding operational efficiency and effectiveness. In August 2005, article published in Fire Chief summarized findings from 17 FDs, pulling from approximately 1 million CAD records, over 43,000 structure fire responses to explore response time components. � This presentation will briefly explore the capability to utilize existing data (CAD & NFIRS) to analyze basic relationships between inputs (i.e. response times) to outcomes (i.e. dollar loss & extension beyond the room of origin).

4. CONTEXT: Why Metro Departments? 51.9% of all FDs are fully volunteer with only a single station Fully career or mostly career agencies with 3 or > stations = 641 agencies (2.7% of all FDs) Metro Departments = 98 agencies (� of 1%), yet protect 24.4% of the population (Source � 2005 Fire Act Grant applications. N=19,441)

5. PROPOSAL Demonstrate the ability to combine currently available CAD data and existing NFIRS Examine basic research questions regarding fire department operations Data taken from Deccan�s CAD Analyst software and merged with NFIRS data Examine dollar loss in structure fires and/or extent of fire beyond the room of origin and correlate this to response times

6. Methodology

7. METHODOLOGY All 2009 NFIRS structure fire reports obtained from USFA Select Metro Chiefs agreed to participate � each utilize Deccan�s CAD Analyst Analysis explored relationships between dollar loss & extension beyond room of origin to response times Department A (N = 464 incidents) Department B (N = 146 incidents) We could readily get data for 2009 NFIRS. The response time data available from NFIRS is limited. As a result, we sought response time data from the CAD Analyst tool we support for many Metro Fire departments. CAD Analyst enabled us to calculate a variety of response time criteria that we will soon see. Since we have been supporting clients for many years, we have many years of data including for 2009. However the analysis was restricted to 2009 because of what we had for NFRIS. NFIRS has many outcome measures but we focused on dollar loss and room of origin. We analyzed impact of response times on dollar loss and extension beyond room of origin. Five Metro fire departments permitted us to analyze their CAD data. However some of their NFIRS data were incomplete and so this analysis was done using two fire departments, one (Department A) with 464 structure fire incidents in 2009 and another (Department B) with 146 incidents. Each department was analyzed one at a time, i.e. their data were not combined. In this manner we could eliminate noise created by departments using different ways of calculating response times and estimating fire loss. Also we avoided the noise of different fire fighting skills and strategies between one department and another. At this point, we were unsure of what we would find. For starters fire loss estimation can be subjective. Second, all said and done our analysis were limited to two departments. What was interesting was despite all this we got insights that were encouraging of being able to extract meaningful relationships between response times and outcome measures. We could readily get data for 2009 NFIRS. The response time data available from NFIRS is limited. As a result, we sought response time data from the CAD Analyst tool we support for many Metro Fire departments. CAD Analyst enabled us to calculate a variety of response time criteria that we will soon see. Since we have been supporting clients for many years, we have many years of data including for 2009. However the analysis was restricted to 2009 because of what we had for NFRIS. NFIRS has many outcome measures but we focused on dollar loss and room of origin. We analyzed impact of response times on dollar loss and extension beyond room of origin. Five Metro fire departments permitted us to analyze their CAD data. However some of their NFIRS data were incomplete and so this analysis was done using two fire departments, one (Department A) with 464 structure fire incidents in 2009 and another (Department B) with 146 incidents. Each department was analyzed one at a time, i.e. their data were not combined. In this manner we could eliminate noise created by departments using different ways of calculating response times and estimating fire loss. Also we avoided the noise of different fire fighting skills and strategies between one department and another. At this point, we were unsure of what we would find. For starters fire loss estimation can be subjective. Second, all said and done our analysis were limited to two departments. What was interesting was despite all this we got insights that were encouraging of being able to extract meaningful relationships between response times and outcome measures.

8. Relationship Possibilities Fire Loss/Contained Within Room Of Origin Vs First Engine First Truck Initial Attack Force (IAF) Effective Response Force (ERF) First Engine + ERF IAF + ERF With one year data, sample size for First Engine only With the CAD Analyst tool we had potential to explore the impact of different mix of response times versus outcome measures. Simply based on fire fighting experience, we all know that it�s not just the first engine or first truck but response force plays a role in mitigating fires. We all suspect perhaps it�s a combination of initial attack and effective response force could play a role. As you can see, we have listed some of the possibilities we could explore in our analysis. (Read out some of them) However many of these times, in particular, Effective Response Force times, are limited to a few incidents in a year and so with the 2009 data we had we could not explore them. Instead we focused on First Engine times. With multiple years so data we are hopeful that the combination times could also be analyzed. With the CAD Analyst tool we had potential to explore the impact of different mix of response times versus outcome measures. Simply based on fire fighting experience, we all know that it�s not just the first engine or first truck but response force plays a role in mitigating fires. We all suspect perhaps it�s a combination of initial attack and effective response force could play a role. As you can see, we have listed some of the possibilities we could explore in our analysis. (Read out some of them) However many of these times, in particular, Effective Response Force times, are limited to a few incidents in a year and so with the 2009 data we had we could not explore them. Instead we focused on First Engine times. With multiple years so data we are hopeful that the combination times could also be analyzed.

9. FINDINGS Department A Considered 464 incidents: First engine dispatch to on scene time > 0 Property loss > $0 Ignored �outlier� incidents for which property loss > $120,000 We are 99.5% confident that average property losses were higher when response times were greater than 4 minutes 57 seconds (297 seconds) When we first looked at the data it was all over the place. There were high and low fire losses across the spectrum of response times. We then focused on some of the very high loss incidents and found that there were some even with very low response times. We then posited that these incidents may those for which no response time could have an impact. Perhaps the significant time from ignition to notification had elapsed. In this case, perhaps we could consider these as lost cause, outlier incidents. So we explored what if we ignored incidents for which the property loss was above a high value. For this client, a reasonable high value was $120,000. Alternately, we could easily use a data based cutoff high value, say, past two standard deviations. We do this cut we do this no matter the response times and so we believe we were not be biased towards either lower or higher times. We also cutoff incidents for which we did not have first engine times and for which the property loss was zero, the idea being perhaps those incidents data were suspect. We ended up with 464 incidents for 2009. Our first question was "is there any reasonable pattern of relationship between fire losses with response times. For this, we took a mid point response time and asked if as a group the incidents that have response times lower than the midpoint has lower fire los compared with the group of incidents with response times higher than the midpoint. We used a basic statistics test, called the standard T test to compare the two groups and found that with high confidence we could say the fire losses were indeed lower in the group with smaller response times. This gave us some hope that perhaps the data may indeed give us something. When we first looked at the data it was all over the place. There were high and low fire losses across the spectrum of response times. We then focused on some of the very high loss incidents and found that there were some even with very low response times. We then posited that these incidents may those for which no response time could have an impact. Perhaps the significant time from ignition to notification had elapsed. In this case, perhaps we could consider these as lost cause, outlier incidents. So we explored what if we ignored incidents for which the property loss was above a high value. For this client, a reasonable high value was $120,000. Alternately, we could easily use a data based cutoff high value, say, past two standard deviations. We do this cut we do this no matter the response times and so we believe we were not be biased towards either lower or higher times. We also cutoff incidents for which we did not have first engine times and for which the property loss was zero, the idea being perhaps those incidents data were suspect. We ended up with 464 incidents for 2009. Our first question was "is there any reasonable pattern of relationship between fire losses with response times. For this, we took a mid point response time and asked if as a group the incidents that have response times lower than the midpoint has lower fire los compared with the group of incidents with response times higher than the midpoint. We used a basic statistics test, called the standard T test to compare the two groups and found that with high confidence we could say the fire losses were indeed lower in the group with smaller response times. This gave us some hope that perhaps the data may indeed give us something.

10. Here the X axis is response time in seconds. The Y axis is property loss. Here the outlier fire loss incidents have been cutoff. For a divider of 4:57, the blue points represent the lower response time group and the red points the higher response time group. Again, at first glance the groups look similar. But note the higher number of blue points that are on the floor, i.e., those with very low fire loss. There are a lot more of these in the blue group compared to the red group. This is what enabled us to say with over 99.5% confidence that the blue points as a group has lower fire loss. We then explored with different divider times and compared the average fire loss of the lower group compared with the upper group. Here the X axis is response time in seconds. The Y axis is property loss. Here the outlier fire loss incidents have been cutoff. For a divider of 4:57, the blue points represent the lower response time group and the red points the higher response time group. Again, at first glance the groups look similar. But note the higher number of blue points that are on the floor, i.e., those with very low fire loss. There are a lot more of these in the blue group compared to the red group. This is what enabled us to say with over 99.5% confidence that the blue points as a group has lower fire loss. We then explored with different divider times and compared the average fire loss of the lower group compared with the upper group.

11. This graph compares for each divider time the average fire loss of the blue group, i.e. those with lower response times versus the average loss of the red group, i.e., those with high response times. As you can see the divider times span 180 second or 3 minutes to 600 seconds or 10 minutes. The red line consistently was higher than the blue lines. Yet another confirmation that higher times mean higher fire loss. This graph compares for each divider time the average fire loss of the blue group, i.e. those with lower response times versus the average loss of the red group, i.e., those with high response times. As you can see the divider times span 180 second or 3 minutes to 600 seconds or 10 minutes. The red line consistently was higher than the blue lines. Yet another confirmation that higher times mean higher fire loss.

12. FINDINGS We got results similar to with Department A. As with previous we explored what if we ignored incidents for which the property loss was above a high value. For this client, a reasonable high value was $100,000. As with previous dept, alternately, we could easily use a data based cutoff high value, say, past two standard deviations. Again, we do this cut we do this no matter the response times and so we believe we were not be biased towards either lower or higher times. We ended up with 146 incidents for 2009. Again, we took a midpoint response time and asked if as a group the incidents that have response times lower than the midpoint has lower fire los compared with the group of incidents with response times higher than the midpoint. Again we found that with high confidence we could say the fire losses were indeed lower in the group with smaller response times. We got results similar to with Department A. As with previous we explored what if we ignored incidents for which the property loss was above a high value. For this client, a reasonable high value was $100,000. As with previous dept, alternately, we could easily use a data based cutoff high value, say, past two standard deviations. Again, we do this cut we do this no matter the response times and so we believe we were not be biased towards either lower or higher times. We ended up with 146 incidents for 2009. Again, we took a midpoint response time and asked if as a group the incidents that have response times lower than the midpoint has lower fire los compared with the group of incidents with response times higher than the midpoint. Again we found that with high confidence we could say the fire losses were indeed lower in the group with smaller response times.

13. Again, here the X axis is response time in seconds. The Y axis is property loss. Here for a divider of 7:24, the blue points represent the lower response time group and the red points the higher response time group. Again, at first glance the groups look similar. Again, note the higher number of blue points that are on the floor, i.e., those with very low fire loss. There are a lot more of these in the blue group compared to the red group. This is what enabled us to say with over 95.7% confidence that the blue points as a group has lower fire loss. Again, we then explored with different divider times and compared the average fire loss of the lower group compared with the upper group. Again, here the X axis is response time in seconds. The Y axis is property loss. Here for a divider of 7:24, the blue points represent the lower response time group and the red points the higher response time group. Again, at first glance the groups look similar. Again, note the higher number of blue points that are on the floor, i.e., those with very low fire loss. There are a lot more of these in the blue group compared to the red group. This is what enabled us to say with over 95.7% confidence that the blue points as a group has lower fire loss. Again, we then explored with different divider times and compared the average fire loss of the lower group compared with the upper group.

14. Again, this graph compares for each divider time the average fire loss of the blue group, i.e. those with lower response times versus the average loss of the red group, i.e., those with high response times. As you can see the divider times span 420 second or 7 minutes to 540 seconds or 9 minutes. The red line consistently was higher than the blue lines. Here with higher times the red line takes off suggesting fire loss gets much higher with higher times. Again, this graph compares for each divider time the average fire loss of the blue group, i.e. those with lower response times versus the average loss of the red group, i.e., those with high response times. As you can see the divider times span 420 second or 7 minutes to 540 seconds or 9 minutes. The red line consistently was higher than the blue lines. Here with higher times the red line takes off suggesting fire loss gets much higher with higher times.

15. FINDINGS Department B With 89.9% confidence, it is more likely for a fire to travel beyond the room of origin with longer response times Here we compared extension beyond room of origin. The idea here is this measure is less sensitive to fire loss estimation challenges. Again we used the divider approach and chose a midpoint of 401 seconds or 6:41. Looking at the second column there were a total of 55+43 or 98 incidents with response times lower than midpoint. Clearly more often the fire is contained within room of origin that not. However for response times higher than midpoint, of the total of 20 + 28 or 48 incidents fewer incidents had fire contained within room of origin than not. When we compare the two groups, the incidents with response times less than midpoint versus incidents with times greater than midpoint, we are almost 89 confident that with lower times, the fire is more likely to be contained within room of origin. Here we compared extension beyond room of origin. The idea here is this measure is less sensitive to fire loss estimation challenges. Again we used the divider approach and chose a midpoint of 401 seconds or 6:41. Looking at the second column there were a total of 55+43 or 98 incidents with response times lower than midpoint. Clearly more often the fire is contained within room of origin that not. However for response times higher than midpoint, of the total of 20 + 28 or 48 incidents fewer incidents had fire contained within room of origin than not. When we compare the two groups, the incidents with response times less than midpoint versus incidents with times greater than midpoint, we are almost 89 confident that with lower times, the fire is more likely to be contained within room of origin.

16. RESULTS For all agencies, no matter the response times, there were high dollar losses � �outliers� For longer response times, average property losses were significantly higher than those with quicker response times For longer response times, it was more likely for a fire to extend beyond the room of origin

17. CONCLUSIONS Combining NFIRS & CAD data can be effectively utilized to answer important questions regarding operational efficiency & effectiveness Aggregating data from multiple departments may be limited by extraneous variables It is possible, as shown here, to estimate how a 30 second improvement in response time can save X dollars Metro Departments are unique as they have large numbers of incidents per agency

18. ROLE FOR METRO CHIEFS ? Should Metro Departments be utilized to answer questions related to firefighter safety and deployment? If so, how could this be accomplished?

19. RESEARCH OPTIONS Create working group to define research questions for Metro Chiefs and seek funding/researchers to undertake the studies - Publish findings under Metro Chiefs brand Collaborate with other researchers by offering cooperation / data (Fire Fighter Safety & Deployment Study; NFPA; others?)

20. QUESTIONS ?