Conquering the Unknowns: Addressing Undetermined and Missing Origin and Cause Entries in Fire Incident Reporting

1. Conquering the Unknowns:Addressing Undetermined and MissingOrigin and Cause Entries in Fire Incident Reporting Project Update NASFM Annual Conference Indianapolis, IN August 7, 2013 Karen Deppa Shannon Frattaroli Ed Comeau

2. Project Overview • General recognition that our nation’s fire data is hampered by high percentage of missing responses or “undetermined” responses in the causal factors sections of National Fire Incident Reporting System (NFIRS) reports • Causal factors include ignition source, heat source, factors contributing to ignition, area of origin, equipment contributing to ignition

3. Project Overview • NFPA Home Structure Fires Report (Ahrens, April 2013) • For non-confined home structure fires, cause of ignition known in: • 70% of the fires • 43% of the civilian deaths • 68% of the civilian injuries • 57% of the direct property damage • Factor contributing to ignition coded as none, undetermined or left blank in: • 47% of the non-confined home structure fires • 66% of the associated deaths • 45% of the associated injuries • 55% of the associated direct property damage

4. Project Overview • Anecdotes and hypotheses about why this is so – but real need for more and better data • Grant to NASFM Foundation from FEMA’s Assistance to Firefighters Grants – Fire Prevention & Safety Program, FY 2011 • Project will run through October 2013, final report due January 2014

5. Project Overview • Goals • Obtain better understanding of reasons behind missing data, or excessive use of “undetermined” or “none” in causal factors section of fire incident data reports • Report on findings, emphasizing how departments can overcome barriers to more effective fire incident data reporting • Identify gaps in available resources to educate/train fire department personnel on complete and accurate recording and reporting of fire incident data

6. Project Overview • Deliverables • Appoint Advisory Committee • US Fire Administration • International Association of Arson Investigators • NFPA’s Fire Data and Research Analysis Division • NFIRS State Program Managers • ME, NJ, NH, CA, TN, SC • Collect fire department policies, guidelines, communications on fire incident data collection • Conduct in-depth interviews with personnel who input fire incident data

7. Project Overview • Deliverables • Administer online survey to confirm issues from in-depth interviews • Conduct in-depth interviews with data specialists in non-fire professions • Compile toolkit of existing resources and guidance for first responders to improve origin and cause reporting • Identify gaps in available information and plan to create/update resources • Report and disseminate findings and recommendations

8. Findings from Research to Date • Collect policies, guidelines, communications by fire departments on fire incident data collection • General request to all State Fire Marshals • Responses from departments in CA, IA, NJ, TX • Most departments have no formal policies or communications about fire incident data collection

9. Findings from Research to Date • Features of the best one shared with us (4-page document from Pennsauken, NJ): • Describes why data collection is important • States who has responsibility to input the data • Explains basics of report writing • Specifies when reports must be completed • Emphasizes quality control and accuracy • Provides references for additional information

10. Findings from Research to Date • Conduct in-depth interviews with fire department personnel who input fire incident data • 20 fire departments across 8 states selected with assistance from State Program Managers • CA, IA, ID, ME, NH, TX, TN, SC • Career, Combination, Volunteer • High and Low Data Quality – rankings provided by USFA

11. Findings from Research to Date • In-Depth Interview Questions • Reporting practices • Training and education about reporting • How firefighters view reporting • Whether they get feedback about their reporting • How missing or undetermined fire data are addressed

12. Findings from Research to Date • Major findings related to “undetermined” or missing causal factors • NFIRS data forms and codes are too confusing, not “user friendly” • Fire personnel don’t understand the “point” of such detailed reporting • Frustrations with software • Data not used by department • Data reporting seen as “necessary evil” • Lack of awareness of training options, no formal training received in most cases

13. Findings from Research to Date • Major findings related to “undetermined” causal factors (continued) • Data reports not modified when new information, such as fire investigation report, is obtained • “Litigation Cloud” - Fear about being “hung out to dry” in court if cause is specified • NFPA 921 Guide inaccurately viewed as requiring 100% certainty in cause determination • Perception is reality

14. Findings from Research to Date • Factors common to departments with high-quality data • Data quality a priority from chief down • Formal quality control review, usually by one person • Data “champion” makes it his/her business to understand the NFIRS codes, seek training, ask questions, help others in department be more accurate • Reports are updated when new information surfaces – especially fire investigations • Data is used extensively within department

15. Findings from Research to Date • Other observations • “Completeness” of forms (satisfying the computer) does not necessarily equal accuracy or data quality • Lack of time to devote to reporting can lead to shortcuts • Desire for more and better training, “models” of what good reports look like • Desire for simpler, more user-friendly system • Desire for “permission” to list “most probable cause” or otherwise not give impression of 100% certainty • State Data Program Managers can play a crucial role

16. Findings from Research to Date • Big Observation: “Closing the loop” of updating reports after investigations may be single most significant action to reducing “undetermineds” in fire incident data (especially for serious fires)

17. Next Phase of Research • Online survey to confirm issues from in-depth interviews • Examples of questions • How we plan to field the survey • How State Fire Marshals can help

18. Examples of Questions • What incentives or legal requirements exist for fire incident reporting in your state or jurisdiction? • Does your department use fire incident data for staffing, resource allocation, or other purposes? • If so, how? • Does anyone in the department review fire incident reports for quality control before they are submitted? • Can a fire incident report be updated once it is initially filled out?

19. Examples of Questions • On a scale of 1-5, with 1 being “Not at all confident” and 5 being “Very confident,” what is your level of confidence about what the code choices mean in the fire incident reporting system? • Are fire incident reports produced by your department updated if more or better information becomes available about the incident? • If incident reports are not updated after additional information is obtained, why is that so?

20. Examples of Questions • Choose the phrase that best describes how you feel about fire incident reporting • It’s a good use of my time • It’s a necessary evil; I would rather be doing other things • I don’t see the value in the time spent on it • Choose the phrase that best describes how most of your colleagues in the department feel about fire incident reporting • [Same choices as above]

21. Examples of Questions • Within your department’s fire incident reports, why would “undetermined” be entered in one or more of the “causal factors” sections? • Fire incident reports are not updated after investigation is complete • People making entries are discouraged from entering specific fire causes unless absolutely sure • Concern about fire department findings contradicting insurance company findings • No investigation was done • Cannot determine cause or origin after investigations • Don’t know

22. Examples of Questions • If personnel making fire incident report entries are discouraged from entering specific fire causes, why do you think this is the case? • The incident report did not warrant an investigation, so the cause was left undetermined • The personnel entering the data are not qualified fire investigators • A determination on causal factors can be used against the department and/or its personnel in court proceedings • If we can’t be 100% sure of causal factors, we put down “undetermined”

23. How We Plan to Field the Survey • Send survey link to all departments via State Fire Marshals • Survey link will be active from approx. August 19 to September 6 • Responses will be tracked by state • Press release to fire service trades encouraging responses

24. How State Fire Marshals Can Help • Send survey link to sample of departments in your state per instructions we provide • We will communicate directly with you about this starting next week • Ask departments to respond promptly • Send reminders to departments as requested by us • Question: If we ask respondents to provide name of department only for tracking purposes and to target reminders, will that discourage them from being candid with their answers?

25. Thank You! • Questions? • For more information: • Karen Deppa, Project Manager • NASFM Fire Research & Education Foundation • kdeppa@firemarshals.org