1 / 23

Current Research in Forensic Toolmark Analysis

Current Research in Forensic Toolmark Analysis Helping to satisfy the “new” needs of forensic scientists with state of the art microscopy, computation and statistics. Outline. Introduction Instruments for 3D toolmark analysis 3D t oolmark data The statistics:

molimo
Download Presentation

Current Research in Forensic Toolmark Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Current Research in Forensic Toolmark Analysis Helping to satisfy the “new” needs of forensic scientists with state of the art microscopy, computation and statistics

  2. Outline • Introduction • Instruments for 3D toolmark analysis • 3D toolmark data • The statistics: • Identification Error Rates • “Match” confidence • “Match” probability • Statistics from available practitioner data

  3. Quantitative Criminalistics • All forms of physical evidence can be represented asnumerical patterns • Toolmark surfaces • Dust and soil categories and spectra • Hair/Fiber categories and spectra • Craniofacial landmarks • Triangulated fingerprint minutiae • Machine learning trains a computer to recognize patterns • Can give “…the quantitative difference between an identification and non-identification”Moran • Can yield identification error rate estimates • May be even confidence measures for I.D.s

  4. Data Acquisition For Toolmarks Confocal Microscope Scanning Electron Microscope Focus Variation Microscope Comparison Microscope

  5. Screwdriver Striation Patterns in Lead 3D surfaces (interactive) 2D profiles

  6. Bullets Bullet base, 9mm Ruger Barrel Bullet base, 9mm Glock Barrel

  7. Close up: Land Engraved Areas

  8. What can we do with all this microscope data? • Statistical pattern comparison! • Modern algorithms are called machine learning • Idea is to measure features that characterize physical evidence • Train algorithm to recognize “major” differences between groups of features while taking into account natural variation and measurement error.

  9. Visually explore: 3D PCA of 760 real and simulated mean profiles of primer shears from 24 Glocks: • ~45% variance retained

  10. Support Vector Machines • Support Vector Machines (SVM) determine efficient association rules • In the absence of specific knowledge of probability densities SVM decision boundary

  11. 18D PCA-SVM Primer Shear I.D. Model, 2000 Bootstrap Resamples Refined bootstrapped I.D. error rate for primer shear striation patterns= 0.35% 95% C.I. = [0%, 0.83%] (sample size = 720 real and simulated profiles)

  12. How good of a “match” is it? Conformal PredictionVovk • Can give a judge or jury an easy to understand measure of reliability of classification result • Confidence on a scale of 0%-100% • Testable claim: Long run I.D. error-rate should be the chosen significance level 80% confidence 20% error Slope = 0.2 99% confidence 1% error Slope = 0.01 95% confidence 5% error Slope = 0.05 • This is an orthodox “frequentist” • approach • Roots in Algorithmic Information Theory Cumulative # of Errors • Data should be IID but that’s it Sequence of Unk Obs Vects

  13. Conformal Prediction • For 95%-CPT (PCA-SVM) confidence intervals will not contain the correct I.D. 5% of the time in the long run • Straight-forward validation/explanation picture for court Empirical Error Rate: 5.3% Theoretical (Long Run) Error Rate: 5% 14D PCA-SVM Decision Model for screwdriver striation patterns

  14. How good of a “match” is it? Efron Empirical Bayes’ • An I.D. is output for each questioned toolmark • This is a computer “match” • What’s the probability the tool is truly the source of the toolmark? • Similar problem in genomics for detecting disease from microarray data • They use dataandBayes’ theorem to get an estimate

  15. A Bayesian Hierarchical Model: Believability Curve JAGS MCMC Bayesian over-dispersed Poisson with intercept, on test set

  16. Bayes Factors/Likelihood Ratios • In the “Forensic Bayesian Framework”, the Likelihood Ratio is the measure of the weight of evidence. • LRs are called Bayes Factors by most statistician • LRs give the measure of support the “evidence” lends to the “prosecution hypothesis” vs. the “defense hypothesis” • From Bayes Theorem:

  17. Bayes Factors/Likelihood Ratios • Using the fit posteriors and priors we can obtain the likelihood ratiosTippett, Ramos Known match LR values Known non-match LR values

  18. Available Large Scale Practitioner Studies • Two large scale published studies • 10-Barrel TestHamby: • 626 practitioners (24 countries) • 15 “unknowns” per test set • At least one bullet from each of the 10 consecutively manufactured barrels • # examiner errors committed = 0 • GLOCK Cartridge Case TestHamby: • 1632 9-mm Glock fired cartridge cases • 1 case per Glock • All cartridge cases pair-wise compared • # of pairs of cartridge cases judged to have enough surface detail agreement to be (falsely) “matching” = 0 • AFTE Theory of Identification standard used: www.swggun.org

  19. Available Large Scale Practitioner Studies • 0% error rate is the “frequentist” estimate • We looked to sports statistics for low scoring games • “Bayesian” statistics provide complementary methods for analysis • Can work much better in estimating small probabilities So does that mean the error rate is 0%?

  20. Available Large Scale Practitioner Studies • For 10-Barrel we need to estimate a small error rate • For GLOCK we need to estimate a small random match probability (RMP) • Use Bayesian “Beta-binomial” method when no “failures” are observed (Schuckers)

  21. Available Large Scale Practitioner Studies • Basic idea of the reverend Bayes: Updated Knowledge Prior Knowledge × Data = a a + b Error Rate/RMP = Uninf(a,b) × Beta-Binomial(data | a,b) Posterior(a,b | data) Get updated estimates of Error rate/RMP

  22. Available Large Scale Practitioner Studies • So given the observed data and assuming “prior ignorance” • Posterior error rate/RMP distributions: Posterior Dist. 10-Barrel Posterior Dist. GLOCK RMP 0.000086% [0.0000020%, 0.00031%] Average Examiner Error Rate 0.011% [0.00023%, 0.040%]

  23. Acknowledgements • Professor Chris Saunders (SDSU) • Professor Christophe Champod (Lausanne) • Alan Zheng (NIST) • Research Team: • Dr. Martin Baiker • Ms. Helen Chan • Ms. Julie Cohen • Mr. Peter Diaczuk • Dr. Peter De Forest • Mr. Antonio Del Valle • Ms. Carol Gambino • Dr. James Hamby • Ms. Alison Hartwell, Esq. • Dr. Thomas Kubic, Esq. • Ms. Loretta Kuo • Ms. Frani Kammerman • Dr. Brooke Kammrath • Mr. Chris Lucky • Off. Patrick McLaughlin • Dr. Linton Mohammed • Mr. Nicholas Petraco • Dr. Dale Purcel • Ms. Stephanie Pollut • Dr. Peter Pizzola • Dr. Graham Rankin • Dr. Jacqueline Speir • Dr. Peter Shenkin • Ms. Rebecca Smith • Mr. Chris Singh • Mr. Peter Tytell • Ms. Elizabeth Willie • Ms. Melodie Yu • Dr. Peter Zoon

More Related