William King , Ph.D. Sam Houston State University

1. OPENING THE BLACK BOX OF NIBIN: A DESCRIPTIVE PROCESS AND OUTCOME EVALUATION OF THE USE OF NIBIN AND ITS EFFECTS ON CRIMINAL INVESTIGATIONSNIJ, Research for the Real World, July 15, 2014. William King, Ph.D.Sam Houston State University William Wells, Ph.D.Sam Houston State University Charles Katz, Ph.D. Arizona State University Edward Maguire, Ph.D. American University James Frank, Ph.D. University of Cincinnati

2. Overview of this presentation • Ballistic imaging. • ATF’s NIBIN program. • The research methodology for this project. • Research findings: • The nature of NIBIN inputs • The nature of NIBIN hits • Policy recommendations: NIBIN’s tactical and strategic value.

3. The fine print • Research supported by Grant #2010-DN-BX-0001 awarded by the National Institute of Justice, OJP, U.S. Department of Justice. • Points of view expressed in this presentation are those of the authors and do not necessarily represent the official position or policies of the U.S. Department of Justice.

4. Tool marks on cartridge cases • Comparison of extractor marks.1 • Comparison of breech face tool marks.1 • Comparison of firing pin impressions. 1 1. Thompson, R.M. 2010. Firearm Identification in the Forensic Science Laboratory. Nat’l District Attorneys Association. Alexandria, VA. (images from pps. 19, 21, 26).

5. Ballistics imaging • A computer converts images of firearms evidence into a digital signature.1 • This signature can be stored in a database and compared to other signatures. • The computer returns a list of probable matches (correlations). • A firearms examiner or technician reviews the possible matches. • The examiner confirms the possible match by visually comparing both pieces of evidence with a comparison microscope. • Matches are designated as hits. • Ballistics imaging has greatly improved the productivity of firearms units.2 1. National Research Council, National Academy of Sciences. (2008). Ballistic imaging. Washington, DC: National Academies Press. Pps. 26, 93-94, 101. 2. Braga, A.A. & Pierce, G.L. (2004). Linking crime guns: The impact of ballistics imaging technology on the productivity of the Boston Police Department’s ballistics unit. Journal of Forensic Sciences, 49, 1-6.

6. A brief overview of NIBIN • National Integrated Ballistics Information Network (NIBIN). • NIBIN is a computerized national network, populated by images of criminal evidence from firearms. • NIBIN is managed and overseen by ATF. • Local, state, and regional LE agencies/labs serve as NIBIN sites. • There were 236 NIBIN sites in 2005. • Currently 150 NIBIN sites. • Since program inception in 1999, NIBIN has produced more than 50,000hits (as of May 2014). • A “hit” links two criminal cases via a common firearm. • NIBIN has been studied and reviewed before (DOJ, OIG 2005;1 NRC, NAS 20082). U.S. Department of Justice, Office of the Inspector General, Audit Division. (2005). The Bureau of Alcohol, Tobacco, Firearms and Explosives’ National Integrated Ballistic Information Network Program. Washington, DC: U.S. Department of Justice. 2. National Research Council, National Academy of Sciences. (2008). Ballistic imaging. Washington, DC: National Academies Press.

7. Research methods and data sources A. Site visits with 10 NIBIN partner sites conducted during 2012 and 2013. Austin, Bowling Green (OH), Houston, Indianapolis/Marion Cty., Kansas City, New Orleans, Onondaga Cty. (NY), Phoenix, Santa Ana, Stockton. B. Survey data from 112 NIBIN sites (or former NIBIN sites), administered January – July 2012. C. ATF/NIBIN hit data from 19 NIBIN sites on 8,231 hits. D. Aggregate ATF/NIBIN hit and input data, by month, by site, for all NIBIN sites (between June 2006 and July 2012). E. Interviews with police investigators in 9 agencies with cases who had received a NIBIN hit (n=65 cases).

8. Overview of the findings • I. Process indicators (inputs and hits) for NIBIN sites. • II. A new performance metric: Elapsed time (between crime and hit identification). • III. Are NIBIN hit reports helpful to criminal investigations? • IV. NIBIN as a source of strategic intelligence.

9. I. Process indicators (inputs and hits) for NIBIN sites • Process indicators: • Inputs: Cartridge cases (brass) and bullets input in NIBIN. • Hits: Outputs, (brass and bullets). • These process indicators were the performance metrics for NIBIN sites prior to 2013. • We found tremendous variation across NIBIN sites as of July 2012. • The following analysis includes sites that were dropped from NIBIN during 2011.

10. The nature of NIBIN inputs1 • Most inputs are brass (88.19%), not bullets. • On average each site had 4,719 brass inputs and 463 bullet inputs. • Most inputs are test fires (80.9% of bullet inputs and 72.9% of brass inputs). • Tremendous variation across sites: • 3 sites had less than 100 brass inputs • 25% of sites had less than 2,317 brass inputs • 25% of sites had less than 99 bullet inputs • 22% of sites had more than 10,000 brass inputs 1. As of July 2012. n=223 NIBIN sites, including sites deactivated in 2011.

11. The nature of NIBIN hits1 • On average (median) sites produced 0 bullet hits and 76 brass hits. • 67.3% of sites produced 0 bullet hits. • The most productive bullet site produced 61 bullet hits. • 13% of sites produced 0 brass hits. • 25% of sites produced 9 or fewer brass hits. • 50% of sites produced 76 or fewer brass hits. • 4.5% of sites produced 1,000 or more brass hits. 1. As of July 2012. n=223 NIBIN sites, including sites deactivated in 2011.

12. Number of brass hits by number of NIBIN sites as of July 2012

13. Number of bullet hits by number of NIBIN sites

14. II. Elapsed time as a performance metric for NIBIN sites • Producing hits is like baking bread: both are good when fresh. • Investigators need information about hits quickly. • Old hits will not help investigations. • Elapsed times are output indicators of performance. • What are the elapsed times for labs for: • Evidence submission to lab and NIBIN input? • Crime and identification of a NIBIN hit?

15. Elapsed times reported by NIBIN firearms sections, submission to entry1 Days 1. Survey responses from 93 NIBIN firearms sections. Median days = 25.

16. Elapsed time: Crime and identification of a NIBIN hit • Elapsed time: the days between the most recent crime in a hit dyad and when the hit was confirmed. • Data from 19 NIBIN sites from (approx.) 2007 to 2012 for 7,141 hits • Overall: • A median of 101 days between crime and hit confirmation. • Sites are different. • Some sites are slow. • Some sites are quick.

17. Median elapsed days between crime and hit confirmation for 18 sites

18. Median elapsed days between crime and hit confirmation for 19 sites

19. III. NIBIN hits and criminal investigations • What role does a NIBIN hit play in a criminal investigation? • Does it help ID a suspect? • Does it help investigators arrest a suspect? • Does it help at trial or sentencing? • Our methods: • We selected 104 criminal cases with a NIBIN hit at 9 of our 10 visit sites. • Most cases (83.1 percent) were homicides. • Most cases occurred between 2009 and 2012. • Face-to-face interviews with homicide investigators in 7 police agencies, and gathered information on cases in 2 other agencies. • Gathered data on 65 of the 104 criminal cases.

20. Results of investigator interviews from 65 cases • Suspect was identified before the NIBIN hit • Yes: 50 percent of cases (n=30) • Suspect was arrested before the NIBIN hit • Yes: 34.5 percent of cases (n=20) • Suspect was charged before the NIBIN hit • Yes: 18.2 percent of cases (n=10)

21. Results of investigator interviews from 65 cases • Did the NIBIN hit identify a suspect? • Yes: 9.7 percent of cases (n=6) • Did the NIBIN hit lead to an arrest? • Yes: 1.6 percent of cases (n=1) • Did the NIBIN hit help obtain charges or a plea bargain? • Yes: 4.9 percent of cases (n=3) • Did the NIBIN hit help with sentencing? • Yes: 1.7 percent of cases (n=1)

22. NIBIN and criminal investigations • “…the contributions of NIBIN hit reports to investigations are modest” (p. 74) • “NIBIN hit reports were rarely used to identify an unknown suspect, make an arrest, or contribute to charging, plea bargaining, or sentencing” (p. 74) • We suspect that NIBIN hits are rarely helpful in investigations because: • The hits come too late (only 10 percent of NIBIN hits are identified within 14 days of the crime) • Hit reports are often short and devoid of relevant information about the cases

23. A case study: NIBIN and a criminal group Linked by NIBIN hit, confirmed on Dec. 31, 2005 • September 12, 2005 Homicide. 9mm brass. • November 25, 2005 Double homicide. .40 cal. brass. • December 03, 2005 Homicide. 9mm and .40 cal. brass • December 16, 2005 Police/ATF raid Gun recovered .40 cal. • The police/ATF raid on Dec. 16, 2005 produces 7 arrests. • The suspects are charged in 10 homicides between November 2004 and December 2005. Linked by NIBIN hit, confirmed on Apr. 04, 2006

24. IV: NIBIN as a source of strategic intelligence • Strategic intelligence- “…collected over the long term and can be used to identify patterns and trends” (Gagliardi 2010, p. 5). • Few sites are mapping NIBIN hits. • Few are using social network analysis. • Only 8 percent of NIBIN sites send hit reports to a crime analysis unit or fusion center.

25. Strategic uses of NIBIN • Some NIBIN sites use hits to identify and prosecute violent criminal networks. • NIBIN hits that indicate gun sharing within gangs are used as evidence in RICO prosecutions • “They have community guns here. If you have a gun that has been used in 5 incidents, you might not be able to tie it to a person but you can tie it to a group. It is a phenomenal tool. It is one of the most powerful tools in law enforcement” • Interviewee in Onondaga County, NY.

26. NIBIN hits in Kansas City: A simple network analysis1 1. Source: Andrew Fox, University of Missouri, Kansas City.

27. Recommendations • Enhance ATF’s capacity to monitor the performance of local NIBIN sites using meaningful metrics. • Foster the strategic and tactical uses of NIBIN by local law enforcement agencies by improving NIBIN data and making them available to agencies. • Establish an ongoing program of research and development intended to discover effective practices and improve performance at NIBIN sites. • Provide funds and resources to establish NIBIN centers of excellence.

28. OPENING THE BLACK BOX OF NIBIN: A DESCRIPTIVE PROCESS AND OUTCOME EVALUATION OF THE USE OF NIBIN AND ITS EFFECTS ON CRIMINAL INVESTIGATIONSNIJ, Research for the Real World, July 15, 2014. William King, Ph.D.Sam Houston State University William Wells, Ph.D.Sam Houston State University Charles Katz, Ph.D. Arizona State University Edward Maguire, Ph.D. American University James Frank, Ph.D. University of Cincinnati

29. References • Cork, Daniel L., Eugene S. Meieran, and Carol V. Petrie (eds.). (2008). Ballistic Imaging: Accuracy and Technical Capability of a National Ballistics Database Committee to Assess the Feasibility, Accuracy and Technical Capability of a National Ballistics Database. National Research Council. Washington, DC: National Academies Press. • Gagliardi, Pete. 2010. Thirteen Critical Tasks: An Inside-out Approach to Solving More Gun Crime. Montreal, Canada: Forensic Technology Inc. • Nichols, Ronald G. 2007. Defending the Scientific Foundations of the Firearms and Tool Mark Identification Discipline: Responding to Recent Challenges. J. of Forensic Sciences, 52, 3: 586-594. • Thompson, Robert M. 2010. Firearm Identification in the Forensic Science Laboratory. Alexandria, VA: National District Attorneys Association. • U.S. Department of Justice, Office of the Inspector General. (2005). The Bureau of Alcohol, Tobacco, Firearms, and Explosives’ National Integrated Ballistic Information Network program. Washington, DC: U.S. Department of Justice.

30. The Journey of Physical Evidence from Gun Crime: From Crime Scene to Conversion to Intelligence

31. Elapsed times should be assessed by multiple methods • The mathematical mean is inappropriate • The median is better but multiple indicators are best • Some sites have old backlogs, but also quickly process new cases • Improper assessment of elapsed times may motivate sites to stop working on their old backlogs • For example:

32. Elapsed Time Percentiles

33. Elapsed Time Percentiles

34. Elapsed Time Percentiles

35. Elapsed Time Percentiles

36. Can sites change their elapsed times?Median days between most recent crime and NIBIN hit report: Houston PD (n=1,129 hits)

37. Median days between most recent crime and NIBIN hit report: Kansas City PD (n=414 hits)