Implementation of the National Incident Management System in New Jersey – A Mixed Methods Study

1. Implementation of the National Incident Management System in New Jersey –A Mixed Methods Study Len Elisha Clark, DPA CEM June 10, 2010

2. Purpose of the Study and Research Question • Purpose • Rigorous research of local emergency management topics is missing • Coordination of response activities is noted as a flaw in the popular media • Seek to examine relevant public administration theories in a “real-world” environment

3. Purpose of the Study and Research Question • Research Question • What theories of implementation inform the assessment of the implementation of the National Incident Management System by municipalities in New Jersey?

4. Academic and TheoreticalLiterature Found to be Particularly Helpful • “The Garbage Can Model” – Cohen, March and Olsen • “The Price of Federalism” – Peterson • “Implementation and Public Policy” - Mazmanian and Sabatier • “Implementation Theory and Practice” - Goggin, O’Toole, Lester & Bowman

5. Methodology • Case study using mixed methods • Sampling Frame • Population of NJ municipalities (n=566) • Unit of Analysis • Class – NJ municipalities

6. Methodology • Purposive Sampling • NIMSCAST and after-action reports • Not all municipalities submitted

7. Methodology • Data Analysis • Quantitative – Regression and General Linear Model • NIMSCAST – indexed data • Compliant, Mostly compliant, In progress, Non-compliant • Funding dollars – per capita • Qualitative • Content Analysis • After-action reports • NIMSCAST narrative • Grant funding reports

8. Results • Hypothesis 1 – • Those municipalities which indentified fewer local units responsible for the implementation of NIMS will have greater success in its implementation.

9. Results • Regression • N=374 • Mean Index value=106, s.d.=24 (d.v.) • Mean # of units = 5, s.d. = 3 (i.v.) • R Square = 0.001 • Significance = 0.604 > 0.05 • Cannot reject null hypothesis • Content analysis • N=22 • No agreement on agencies involved • OHSP reports “impossible to ascertain each response agencies [sic] level of readiness…” (2008: 49)

10. Results • Hypothesis 2 – • Municipalities within the New York Metropolitan Statistical Area (MSA) are more likely to implement NIMS than within the Philadelphia MSA or those outside of either MSA.

11. Results • Analysis • N=376 • 75% rated as “compliant” or “mostly compliant” • Outside either MSA – highest percentage “non-compliant” and “complaint” • New York MSA – highest percentage “in progress” and “mostly compliant” • Significance 0.190 > 0.05 – Chi-Square test • Cannot reject null hypothesis

12. Results • Hypothesis 3 – • Municipalities that received federal Emergency Management Performance Grants (EMPG) have implemented more components of NIMS than those which received no amount of funds. • EMPG funds evaluated on a “per-capita” basis with a presumptive baseline value of $1.00 per capita for all municipalities

13. Results • Regression • N=376 • Mean Index value=106, s.d.=24 (d.v.) • Mean # of EMPG values = $1.06, s.d. = $0.19 (i.v.) • R Square = 0.009 • Significance = 0.073 > 0.05 • Cannot reject null hypothesis • Content analysis • 8.4% of statewide award passed through to municipalities

14. Results • Hypothesis 4 – • Municipalities in counties which received more Presidential Disaster Declarations from the years 1979 through 2008 tend to implement more components of NIMS than those which received fewer declarations.

15. Results • Regression • N=376 • Mean Index value=106, s.d.=24 (d.v.) • Mean # of units = 5, s.d. = 1 (i.v.) • R Square = 0.014 • Significance = 0.02 < 0.05 • Can reject null hypothesis • Y=116 (constant) -2.0x (with a negative decrease)

16. Results • Hypothesis 5 – • Using a General Linear Model to determine, how, if any way, these factors (# of agencies involved [fixed variable], location within an MSA [covariate], disaster declaration experience [covariate]) combine t shape the implementation of NIMS

17. Results • Model • n=374 • Philadelphia MSA – highest mean Index value=108, s.d.=24 • Non-MSA – lowest = 103, s.d = 32 • R Square = 0.031 • Significance for declarations = 0.013 < 0.05 • Cannot reject this covariate • The balance of the factors remained statistically insignificant • NY + 3 points, the balance -4 in comparison to the each other

18. Results • Hypothesis 6 – • Municipalities which submitted “compliant” NIMSCAST reports will identify fewer negative after-action report outcomes in the ETEAM NIMS-related categories than those municipalities which received other compliance ratings.

19. Results • Regression • n=74 • Mean Index value=110, s.d.=20 (i.v.) • Mean # of shortfalls = 1, s.d. = 1 (d.v.) • R Square = 0.012 • Significance = 0.346 > 0.05 • Cannot reject null hypothesis • Content Analysis • Shortfalls (23) – Equipment (3), Facilities (5), Training (5), Personnel (4), Planning (6)

20. Results • Paradigm of Maximum Variation • MSA location and Index score key factors • NY MSA – Rockaway Township (4) and Harrison (132) • Philadelphia – Springfield (14) and Pemberton Township (133) • Non-MSA – Franklin (1) and Commercial Township (132)

21. Results • The number/type of involved agencies was neither standardized nor significant • Location within the State was not a factor • EMPG funding did not appear to be a factor • Disaster declaration experience appears to hold as an influencing factor, but counter-intuitive • In combination, the previous factors remain the same • Full-scale exercises or actual occurrences were not conducted and shortfalls not captured

22. Discussion • NJ municipalities may not be “compliant” with the Federal intent, but progress is being made. • Theoretical Significance • Majority of the theoretical assertions were not supported by the results. • Can see a positive tie into the Garbage Can Model’s idea of “Organized Anarchies.”

23. Discussion • Methodological Significance • Mixed methods capitalized upon strengths of both quantitative and qualitative methods • NIMSCAST and ETEAM data proved easily adaptable to this use, although not designed as a statistical dataset.

24. Discussion • Methodological Significance • Recommendations • Conduct a longitudinal study (2005 to present) • Include regional or national studies • Examine from a “systems” aspect • Investigate the use of Patton’s “Complexity Theory” for non-linear examinations • Studies using “mixed methods” is a viable method

25. Discussion • For the Field Of Action • Government should continue to leverage “Strong Statute” and “Wilsonian” Perspectives • Federal government should refine definition of compliance and its metrics • Consider a multiple tiered approach with additional funding • Move from “compliance” and develop a “systems” approach to build capacity – planning, training, exercises, etc.

26. Discussion • Action-Research • Examine the results in greater detail • Why did 25% not score well in NIMSCAST? • Incorporate units into this process • Identify factors which impede implementation • Some known, others unknown • Why the absence of municipal submissions • Identify clear, concise measures in the absence of Federal direction

27. Discussion • Develop actionable items – policy and/or process redesign and an implementation plan • Collect and evaluate data • Reports • Exercises • Actual emergencies • Needed due to constant reworking of requirements by Federal government

28. Parting Thoughts • NIMS implementation is based on sound public administration theory. • Local agencies have a clear understanding of their importance in public safety • Statistical tests challenge existing theory • New Jersey is making positive strides along the compliance continuum