280 likes | 412 Views
Implementation of the National Incident Management System in New Jersey – A Mixed Methods Study . Len Elisha Clark, DPA CEM June 10, 2010 . Purpose of the Study and Research Question. Purpose Rigorous research of local emergency management topics is missing
E N D
Implementation of the National Incident Management System in New Jersey –A Mixed Methods Study Len Elisha Clark, DPA CEM June 10, 2010
Purpose of the Study and Research Question • Purpose • Rigorous research of local emergency management topics is missing • Coordination of response activities is noted as a flaw in the popular media • Seek to examine relevant public administration theories in a “real-world” environment
Purpose of the Study and Research Question • Research Question • What theories of implementation inform the assessment of the implementation of the National Incident Management System by municipalities in New Jersey?
Academic and TheoreticalLiterature Found to be Particularly Helpful • “The Garbage Can Model” – Cohen, March and Olsen • “The Price of Federalism” – Peterson • “Implementation and Public Policy” - Mazmanian and Sabatier • “Implementation Theory and Practice” - Goggin, O’Toole, Lester & Bowman
Methodology • Case study using mixed methods • Sampling Frame • Population of NJ municipalities (n=566) • Unit of Analysis • Class – NJ municipalities
Methodology • Purposive Sampling • NIMSCAST and after-action reports • Not all municipalities submitted
Methodology • Data Analysis • Quantitative – Regression and General Linear Model • NIMSCAST – indexed data • Compliant, Mostly compliant, In progress, Non-compliant • Funding dollars – per capita • Qualitative • Content Analysis • After-action reports • NIMSCAST narrative • Grant funding reports
Results • Hypothesis 1 – • Those municipalities which indentified fewer local units responsible for the implementation of NIMS will have greater success in its implementation.
Results • Regression • N=374 • Mean Index value=106, s.d.=24 (d.v.) • Mean # of units = 5, s.d. = 3 (i.v.) • R Square = 0.001 • Significance = 0.604 > 0.05 • Cannot reject null hypothesis • Content analysis • N=22 • No agreement on agencies involved • OHSP reports “impossible to ascertain each response agencies [sic] level of readiness…” (2008: 49)
Results • Hypothesis 2 – • Municipalities within the New York Metropolitan Statistical Area (MSA) are more likely to implement NIMS than within the Philadelphia MSA or those outside of either MSA.
Results • Analysis • N=376 • 75% rated as “compliant” or “mostly compliant” • Outside either MSA – highest percentage “non-compliant” and “complaint” • New York MSA – highest percentage “in progress” and “mostly compliant” • Significance 0.190 > 0.05 – Chi-Square test • Cannot reject null hypothesis
Results • Hypothesis 3 – • Municipalities that received federal Emergency Management Performance Grants (EMPG) have implemented more components of NIMS than those which received no amount of funds. • EMPG funds evaluated on a “per-capita” basis with a presumptive baseline value of $1.00 per capita for all municipalities
Results • Regression • N=376 • Mean Index value=106, s.d.=24 (d.v.) • Mean # of EMPG values = $1.06, s.d. = $0.19 (i.v.) • R Square = 0.009 • Significance = 0.073 > 0.05 • Cannot reject null hypothesis • Content analysis • 8.4% of statewide award passed through to municipalities
Results • Hypothesis 4 – • Municipalities in counties which received more Presidential Disaster Declarations from the years 1979 through 2008 tend to implement more components of NIMS than those which received fewer declarations.
Results • Regression • N=376 • Mean Index value=106, s.d.=24 (d.v.) • Mean # of units = 5, s.d. = 1 (i.v.) • R Square = 0.014 • Significance = 0.02 < 0.05 • Can reject null hypothesis • Y=116 (constant) -2.0x (with a negative decrease)
Results • Hypothesis 5 – • Using a General Linear Model to determine, how, if any way, these factors (# of agencies involved [fixed variable], location within an MSA [covariate], disaster declaration experience [covariate]) combine t shape the implementation of NIMS
Results • Model • n=374 • Philadelphia MSA – highest mean Index value=108, s.d.=24 • Non-MSA – lowest = 103, s.d = 32 • R Square = 0.031 • Significance for declarations = 0.013 < 0.05 • Cannot reject this covariate • The balance of the factors remained statistically insignificant • NY + 3 points, the balance -4 in comparison to the each other
Results • Hypothesis 6 – • Municipalities which submitted “compliant” NIMSCAST reports will identify fewer negative after-action report outcomes in the ETEAM NIMS-related categories than those municipalities which received other compliance ratings.
Results • Regression • n=74 • Mean Index value=110, s.d.=20 (i.v.) • Mean # of shortfalls = 1, s.d. = 1 (d.v.) • R Square = 0.012 • Significance = 0.346 > 0.05 • Cannot reject null hypothesis • Content Analysis • Shortfalls (23) – Equipment (3), Facilities (5), Training (5), Personnel (4), Planning (6)
Results • Paradigm of Maximum Variation • MSA location and Index score key factors • NY MSA – Rockaway Township (4) and Harrison (132) • Philadelphia – Springfield (14) and Pemberton Township (133) • Non-MSA – Franklin (1) and Commercial Township (132)
Results • The number/type of involved agencies was neither standardized nor significant • Location within the State was not a factor • EMPG funding did not appear to be a factor • Disaster declaration experience appears to hold as an influencing factor, but counter-intuitive • In combination, the previous factors remain the same • Full-scale exercises or actual occurrences were not conducted and shortfalls not captured
Discussion • NJ municipalities may not be “compliant” with the Federal intent, but progress is being made. • Theoretical Significance • Majority of the theoretical assertions were not supported by the results. • Can see a positive tie into the Garbage Can Model’s idea of “Organized Anarchies.”
Discussion • Methodological Significance • Mixed methods capitalized upon strengths of both quantitative and qualitative methods • NIMSCAST and ETEAM data proved easily adaptable to this use, although not designed as a statistical dataset.
Discussion • Methodological Significance • Recommendations • Conduct a longitudinal study (2005 to present) • Include regional or national studies • Examine from a “systems” aspect • Investigate the use of Patton’s “Complexity Theory” for non-linear examinations • Studies using “mixed methods” is a viable method
Discussion • For the Field Of Action • Government should continue to leverage “Strong Statute” and “Wilsonian” Perspectives • Federal government should refine definition of compliance and its metrics • Consider a multiple tiered approach with additional funding • Move from “compliance” and develop a “systems” approach to build capacity – planning, training, exercises, etc.
Discussion • Action-Research • Examine the results in greater detail • Why did 25% not score well in NIMSCAST? • Incorporate units into this process • Identify factors which impede implementation • Some known, others unknown • Why the absence of municipal submissions • Identify clear, concise measures in the absence of Federal direction
Discussion • Develop actionable items – policy and/or process redesign and an implementation plan • Collect and evaluate data • Reports • Exercises • Actual emergencies • Needed due to constant reworking of requirements by Federal government
Parting Thoughts • NIMS implementation is based on sound public administration theory. • Local agencies have a clear understanding of their importance in public safety • Statistical tests challenge existing theory • New Jersey is making positive strides along the compliance continuum