The effects of a prior examiner’s status and findings on lay examiners’ shoeprint match decisions

1. The effects of a prior examiner’s status and findings on lay examiners’ shoeprint match decisions Nadja Schreiber Compo, Ph.D. (Associate Professor, Co-Director, Legal Psychology Graduate Program, FIU) Stephanie Stoiloff, Igor Pacheco, Thomas Fadul (Forensic Services Bureau, Miami-Dade Police Department) Jillian Rivard, Ph.D. (Barry University) Danielle Sneyd & Michelle Pena (Florida International University) 2018 Impression, Pattern and Trace Evidence Symposium FTCE

2. Wrongful convictions and forensic science • Problem of wrongful convictions was brought to the forefront of public (policy) consciousness via advances in forensic science, i.e., DNA analyses (Cutler, 2012) • National registry of exonerations has compiled a database of all US exonerations and its contributing factors since 1989 • Recent analyses of DNA exonerations reveal that errors related to forensic science can contribute to wrongful convictions (Dror & Bucht, 2012)

3. % EXONERATIONS BY CONTRIBUTING FACTOR % of exonerations National Registry of Exonerations 11/1/2017 Total = 2111

4. Errors in Forensic Science • Errors can result from misrepresentation or misinterpretation of forensic evidence at trial but in some cases, human errors known as “cognitive biases” can produce errors in forensic judgments (NAS, 2009) • NAS report: importance of collaborative research involving forensic scientists and academics on the role of interpretive bias and “sources of human error” across a broad range of forensic disciplines.

5. What is Cognitive Bias? • Group of systematic cognitive contamination errors made when processing (investigative) information. • Cognitive biases (including confirmation bias, hindsight bias, expectancy effect, Rosenthal effect and outcome bias) are mental shortcuts that can help guide and simplify human decision-making. • Shortcuts can be efficient, save time and help process information swiftly and accurately

6. What is Cognitive Bias? • Shortcuts can also be problematic: having preconceptions (e.g., that an event occurred or a particular person was involved) can cause an individual to focus on information that supports his or her preconceptions, while ignoring information that may contradict the initial belief. • These cognitive biases are natural and automatic, often operate beyond conscious awareness

7. Cognitive Bias and Forensic Science • Prior beliefs, motives, and situational contexts can influence how a piece of forensic evidence is perceived and interpreted during analysis (Kassin et al., 2013) • One forensic science discipline with a well-documented vulnerability to cognitive bias is fingerprint analysis (Dror & Cole, 2010) • Previous research on bias in fingerprint analysis has focused on a single examiner’s decision, largely ignoring the mandatory technical review step by a second examiner in the fingerprint examination protocol (SWGFAST, 2013)

8. Cognitive Bias and the Technical Review Process • Little research on the effects of prior information on reviewers’ decision-making during the verification stage of examinations • Limited data suggests that knowledge of a prior examiner’s decision can influence an examiner’s analysis of a fingerprint (e.g., Pacheco, Cerchiai, & Stoiloff, 2014) • When prints are from a different source, “inconclusive” responses can increase when examiners are aware of a prior examiner’s decision relative to a control condition with no information about a prior decision, especially when examiners were novices (Langenburget al., 2009)

9. Cognitive Bias and the Technical Review Process • Lab studies on cognitive bias in forensic examination have been criticized by practitioners for using student evaluators and stimuli arguably beyond their level of experience and training (e.g., fingerprint, handwriting, voice comparison; Kukucka & Kassin, 2016; Smalarz et al., 2016) • limits generalizability of findings to real world contexts

10. The present study: General Aims • address limitations of prior research • expand specifically on the understudied effects of the technical review step • test the potential for cognitive bias in the technical review process using a controlled analog lab-based paradigm

11. The present study: Specific Aims • examine potential for bias in technical review step using shoeprint comparisons, stimuli that were used to screen potential applicants for firearm and tool mark positions at the Miami-Dade Police Department Forensic Services Bureau (MDPD FSB) • using stimuli appropriate for novice evaluators in a controlled lab setting, we examined how knowledge of a prior examiner’s conclusion (in the absence of the prior examiner’s reasoning) and knowledge of the prior examiner’s level of experience affect lay reviewers’ evaluation of shoeprint comparisons

12. 401.81

13. 401.65

14. Method: Design • 4 (previous decision type: match vs. non-match vs. inconclusive vs. none) by 3 (prior information: forensic expert vs. student vs. anonymous) by 2 (stimuli type: match vs. non-match) mixed between-within participants design, with repeated measures on the last factor

15. Method: Materials and Procedure • Shoeprint comparison test was provided by the MDPD FSB and consists of 20 footwear impressions • Upon entering the lab, student volunteers were told that purpose of study was to assess students’ performance on a stimuli comparison task commonly used as an entrance exam for forensic examiners • Participants seated in front of a computer and instructed that they were about to view a series of shoeprint pairs that had been previously assessed

16. Method: Materials and Procedure • Random assignment to one of three prior information/source conditions: • receive information about a prior decision made by a forensic expert • receive information about a prior decision made by another student • receive information about a prior decision made by another person (status of person was not given)

17. Method: Materials and Procedure • For participants in the prior information conditions, prior decision was either identified as • a match by the prior examiner (correctly and incorrectly so) • a non-match (correctly and incorrectly so) • inconclusive (all incorrectly as ground truth is known) A fourth group of participants received no information regarding a previous examiner’s decision.

18. Method: Materials and Procedure • For each pair participants were asked whether the two stimuli are a match, non-match, or not sure/inconclusive • After each comparison, participants also provided a confidence and difficulty assessment (scale from 1-7) with lower numbers indicating less confidence/difficulty • At the end of the study participants answered demographic questions and were debriefed. After each comparison, participants provided a confidence and difficulty assessment (scale from 1-7) with lower numbers indicating less confidence/difficulty

19. Results: % Accuracy (N =119) % Accuracy (% correct out of all 10 matches): • Across all conditions, participants were more accurate when making decisions about matching (M = .84)than non-matching shoeprints (M = .54) • The type of decision made by the prior examiner (i.e., match, non-match, inconclusive or none) influenced the accuracy of the second examiner’s decision (prior decision X stimulus type): Only when shoeprint was a nonmatch did prior information matter: Participants were more accurate when told that the previous decision was a nonmatch compared to a match (p = .09)

20. Results: Count variables (N =119) #correct, #incorrect, #inconclusive: • Across all conditions, participants made more incorrect judgments when making decisions about nonmatching (M = 2.64) than matching shoeprints (M = 0.93), p < .001 • Trend: when shoeprint was a match, participants made more inconclusive decisions when the first examiner was an expert (M = 2.36) compared to no information about the prior examiner (M = 1.51), p = .082

21. Results: Agreements (N =119) Agreement (for those who received prior information): • Participants were more likely to agree with a prior decision if it was a match and they were told it was a match (.95) compared to other consistent information (e.g., it was a nonmatch and they were told it was a nonmatch)

22. Results: Summary • Lay examiners make more accurate decisions when comparing matching compared to non-matching shoeprints • When asking lay examiners to make decisions about forensic stimuli the type of decision made by the prior examiner (i.e., match, non-match, inconclusive or none) and the level of expertise can influence the second examiner’s decision • Present data paints a complex picture, that is, influence depends on whether second examiner is comparing matches versus non matches, the type of decision they are asked to make and the expertise and decision of the first examiner

23. Discussion and Outlook • This study design enables testing for cognitive bias in a controlled lab setting without losing applied relevance. • This initial study using an analog paradigm to examine the effects of prior examiner’s level of expertise and decision on a second examiner’s decision suggests that both status of first examiner and type of decision matter • Data collection is still ongoing to pattern of results may still be subject to change

24. Discussion and Outlook • Future research is needed to • Disentangle the respective importance of examiner status, and type of decision • Include real-world forensic examiners • Compare lay with expert examiner’s performance • Broaden the type of stimulus material to include fingerprints and DNA evidence