Confocal Microscopy and Striated Tool Marks: A Statistical Study and

1. Confocal Microscopy and Striated Tool Marks: A Statistical Study and Potential Software Tools For Practitioners Carol Gambino, Patrick McLaughlin, Loretta Kuo, Peter Diaczuk, Gerard Petillo, FraniKammerman, Lauren Claytor, Peter Shenkin, Nicholas Petraco, James Hamby and Nicholas D. K. Petraco

2. Outline • Introduction • Details of OurApproach • Data acquisition • Confocal Microscopy for Surfaces • Surface/Profile Pre-processing • Results of Statistical Discrimination/Error Rate Estimates for Primer Shears • Suggested Operating Guidelines

3. Introduction • DNA profiling the most successful application of statistics in forensic science. • Responsible for current interest in “raising standards” of other branches in forensics…?? • No protocols for the application of statistics to comparison of tool marks. • Our goal: application of objective, numerical pattern comparison to tool marks Caution: Statistics is not a panacea!!!!

4. Background Information • All impressions made by tools and firearms can be viewed as numerical patterns • Machine learning trains a computer to recognize patterns • Can give “…the quantitative difference between an identification and non-identification”Moran • Can yield identification error rate estimates

5. Data Acquisition • Obtain striation/impression patterns from 3D confocal microscopy • Store files in ever expanding database • Data acquisition is labor intensive and time consuming • Data files will be made available to practitioner community through web interface

6. Confocal Microscopy Increasing surface height All-in-Focus 2D Image Overlay confocal “z-stack”

7. 3D confocal image of portion of chisel striation pattern

8. Glock Fired Cartridges Bottom of Firing pin imp. Glock 19 fired cartridge cases

9. Five Consecutively Manufactured Chisels G. Petillo Lead impression media Striation patterns generated at 32o • 70 striation patterns total: • 20 for traditional comparison • 50 for confocal microscopy G. Petillo

10. Experimental Research Design Striation patterns in lead Striation patterns in wax Craftsman Screwdrivers

11. Extended ImageJ Functionality

12. 2D profiles 3D surfaces (interactive)

13. Experimental Research Design Experimental Research Design • Software can detect edges of significant “lines”: • Or software can turn any profile into a “barcode”: barcode profile

14. Website Downloadable • 3D surfaces • ImageJ visualization • 2D and 3D • ImageJ measurements • R scripts/programs for statistical analysis • Preprints of papers

15. Striation pattern processing: Hamby 4 • Form removal • Register and optionally shift skewed profiles • Use max CCF Claytor 1 • Optional filter surface into waviness and roughness components • Cubic spline filter:

16. Primer Shear Primer shear from the sameGlock 19

17. Research Results Primer shear from two different Glock 19s

18. Experimental Research Design Mean total profile: Mean “waviness” profile: Mean “roughness” profile:

19. What Statistics Can Be Used? • Statistical pattern comparison! • Modern algorithms are called machine learning • Idea is to measure features that characterize physical evidence • Train algorithm to recognize “major” differences between groups of features while taking into account natural variation and measurement error.

20. Primer Shear • Primer shears (82-91 profiles) • PCA-SVM, CPT at the 95% level of confidence • Empirical error rate was 4.7% • No “uninformative” intervals were returned • PCA-SVM, HOO-CV • Error rate estimate is 0.0%-4.4%, depending on • the number of replicates • PLS-DA, Bootstrap (>10 replicates only) • 95% confidence interval for error rate: [0%, 0%] • PLS-DA, HOO-CV • Error rate estimate is 0.0%-4.3%, depending on the number of replicates • Results so far are on par with expectations • More samples are being prepared for analysis

21. Preliminary suggested operating guidelines • Visualization is MOST important • Trained examiner assessment • 3D microscopy and visualization • For statistical analysis: • # of replicates VERY important. • Train “machine learning method” on suspect tool and tools producing “similar marks” (close in data space) • SVM, PLS-DA • Get I.D. error rate estimates from various methods • Large test sets, bootstrap, cross-validation • Classify an unknown form a crime scene • Use CPT for a level of confidence for the “association”

22. References • Biasotti AA. A statistical study of the individual characteristics of fired bullets. J Forensic Sci 1959;4(1):34-50. • Efron B, Tibshirani RJ. An introduction to the bootstrap. 1st ed. Boca Raton: Chapman & Hall/CRC, 1993. • Geradts Z, Keijzer J, Keereweer I. A new approach to automatic comparison of striation marks. J Forensic Sci 1994;39(4):974-80. • AFTE. Theory of Identification as it Relates to Toolmarks. AFTE J. 1998;30(1):89-8. • Moran B. A report on the AFTE theory of identification and range of conclusions for tool mark identification and resulting approaches to casework. AFTE J 2002;34(2):227-35. • Vovk V, Gammerman A, Shafer G. Algorithmic learning in a random world. 1 ed. New York: Springer, 2005. • Neel M, Wells M. A comprehensive statistical analysis of striated tool mark examinations. Part 1: Comparing known matches to known non -matches. AFTE J 2007;39(3):176-98. • Gammerman A, Vovk V. Hedging predictions in machine learning. The Computer J 2007;50(7):151-77.

23. References • Schafer G, Vovk V. A tutorial on conformal prediction. J Machine Learning Research 2008;9:371-421. • Howitt D, Tulleners F, Cebra K, Chen S. A calculation of the theoretical significance of matched bullets. J Forensic Sci 2008;53(4):868-75. • Chumbley LS, Morris MD, Kreiser MJ, Fisher C, Craft J, Genalo LJ Davis S, Faden D, Kidd J. 2010. Validation of Tool Mark Comparisons Obtained Using a Quantitative, Comparative, Statistical Algorithm. J Forensic Sci 2010;55(4):953-961. • Bachrach B, Jain A, Jung S, Koons RD. A statistical validation of the individuality and repeatability of striated tool marks: Screwdrivers and tongue and groove pliers. J Forensic Sci 2010;55(1):348-57. • Chu W, Song J, Vorburger T, Yen J, Ballou S, Bachrach B. Pilot study of automated bullet signature identification based on topography measurements and correlations. J Forensic Sci 2010;55(2):341-7. • Petraco N. Color Atlas of Forensic Toolmark Identification. 1st ed. Boca Raton: Chapman & Hall/CRC, 2010. • Petraco NDK, Shenkin P,Speir J, Diaczuk P, Pizzola PA, Gambino C, Petraco N. Addressing the National Academy of Sciences’ Challenge: A Method for Statistical Pattern Comparison of Striated Tool Marks. J Forensic Sci 2011, (accepted). • Gambino C, McLaughlin P, Kuo L, Kammerman F, Shenkin P, Diaczuk P, Petraco N, Hamby J, Petraco NDK. Forensic Surface Metrology: Tool Mark Evidence. Scanning, 2011 (accepted).

24. Acknowledgements • National Institute of Justice • New York City Police Department Crime Lab • John Jay College of Criminal Justice • Research Team: • Mr. Peter Diaczuk • Ms. Carol Gambino • Dr. James Hamby • Dr. Thomas Kubic, Esq. • Off. Patrick McLaughlin • Mr. Jerry Petillo • Mr. Nicholas Petraco • Dr. Peter A. Pizzola • Dr. Jacqueline Speir • Dr. Peter Shenkin • Mr. Peter Tytell • Ms. Alison Hartwell, Esq. • Ms. Lauren Claytor • Helen Chan • Manny Chaparro • Aurora Ghita • Eric Gosslin • Frani Kammerman • Brooke Kammrath • Loretta Kuo • Dale Purcel • Stephanie Pollut • Rebecca Smith • Elizabeth Willie • Chris Singh • Melodie Yu • Greg Frasier

25. Website Information and Reprints/Preprints: npetraco@gmail.com npetraco@jjay.cuny.edu