The impact of discredited evidence

1. The impact of discredited evidence David Lagnado Nigel Harvey Evidence project, UCL

2. Discredited evidence • How do people revise their beliefs once an item of evidence is discredited? • For example, in a murder trial, when the testimony of a key witness is shown to be fabricated, how does this affect juror’s beliefs about the testimony of other witnesses, or even other forensic evidence? • OJ Simpson trial

3. Normative Models • Bayesian network models • Normative model for combining probabilistic evidence • E.g., forensic DNA evidence; paternity cases (Dawid) • Formal modelling of ‘manipulated evidence’ (Baio) • Starting to be applied to crime cases • But a lot depends on network construction • No ‘normative’ method for this?

4. Crime case (Leucari, 2005)

5. P(S|C) > P(S) Finding out C raises probability of S S = Suspect commits crime S C F C = Suspect confesses P(S|C&F) < P(S|C) Finding out F too lowers the probability of S F = Police force confession Explaining away (Pearl, 1988) Despite its simplicity and ubiquity, this pattern of inference is hard to capture on most psychological models of inference (e.g., associative models, connectionist, mental models, mental logic)

6. Psychological models • Belief-adjustment model • Hogarth & Einhorn (1992) • Story model • Pennington & Hastie (1981, 1988, 1992, 1993)

7. Belief adjustment model • For Evaluation tasks • Evidence encoded as +ve or –ve relative to hypothesis • Adding model Sk = Sk-1 + wk s (xk) Sk = degree of belief in hypothesis given k items of evidence Sk-1 = prior opinion S (xk) = subjective evaluation of kth item (-1 ≤ s (xk) ≤ +1) wk = adjustment weight (0 ≤ wk ≤1)

8. Online vs. Global Processing • Two processing modes • Online (step-by-step) • Belief adjusted incrementally with each item of evidence • Global (end-of-sequence) • Belief adjusted by aggregate impact of all items Sk = S0 + wk [s (x1,…, xk)]

9. When is each process used? RESPONSE MODE PROCESS Processing load - Aggregation can be costly in terms of mental resources whereas step-by-step integration makes minimal demands

10. Evidence for Belief Adjustment • Order effects in online processing • Evaluation mode (adding not weighted average) • None with consistent evidence (e.g, ++ or --) • Recency with mixed evidence (-+ > +-) over-weight last item • Supported in Exps 1-5 • Model is quite flexible – designed to account for rich patterns of primacy and recency evidence • But does not address relations between evidence items (assumes independence?)

11. Story model • Evidence evaluated through story construction • Stories involve network of causal relations between events • Causal narratives not arguments • People represent events in the world, not inference process • Stories constructed prior to judgment or decision • Stories determine verdicts, and are not post hoc justifications

12. Evidence for story model • Verbal protocols • 85% of events causally linked • Verdicts covaried with story models • Recognition memory tests • More likely to falsely remember items consistent with story model • Story vs witness order • More likely to convict when prosecution evidence in causal order, defence in witness order, and vice-versa

13. Current experiments • Investigate effect of discredited evidence • Look at relations between items of evidence • Do these modulate how people revise their beliefs? • Once an item of evidence is discredited, do people simply return to their prior level of belief? • Or does this change permeate their belief network? • What factors affect this? • Relations between evidence • Order of presentation of evidence

14. EVIDENCE 1 Neighbour 1 says that S often loiters in area EVIDENCE 2 Neighbour 2 says S was outside house on night of crime ? Neighbour 2 is lying because he dislikes S Scenario: House burglary, local suspect S apprehended HYPOTHESIS: Suspect S did it P(S) Under-discounting Over-discounting

15. Generalisation • When do people generalize from the discrediting of one item to other items? • Dependent on relatedness of generating mechanisms? • SAME • E.g. two statements from same neighbour • SIMILAR • E.g. two statements from two different neighbours • DIFFERENT • E.g., one statement and one blood test

16. EVIDENCE 1 Footprints outside house match suspect’s EVIDENCE 2 Neighbour says S was outside house on night of crime ? Neighbour is lying because he dislikes S DIFFERENT Scenario: House burglary, local suspect S apprehended HYPOTHESIS: Suspect S did it P(S) Under-discounting Over-discounting

17. Experiment 1 • Each subject completes 12 problems (4 scenario types x 3 levels of relatedness) • ‘Relatedness’: SAME, SIMILAR, DIFFERENT • Four probability judgments (of guilt) • Background information • Evidence 1 (E1) • Evidence 2 (E2) • Discredit evidence 2 (D2) • Compare 2 and 4 (E2 vs. D2) • If D2 > E2 then under-discounting • If D2 < E2 then over-discounting

18. Example of SAME condition • Background You are a juror on a murder case. The victim is a middle-aged woman and the suspect is her ex-husband. You will need to judge whether or not the suspect is guilty on the basis of several pieces of evidence. The woman was found stabbed at her home. She was fully clothed and the murder weapon, a knife, was present at the crime scene • Evidence 1 The police have a statement from the current wife of the suspect, confessing that the suspect had previously revealed a desire for the victim to be dead • Evidence 2 The same police station has a confession from the suspect, admitting that he killed the victim • Discredit 2 The confession from the suspect was made under extremely pressured circumstances at the police station

19. Example of DIFF condition • Background You are a juror on a murder case. The victim is a middle-aged woman and the suspect is her ex-husband. You will need to judge whether or not the suspect is guilty on the basis of several pieces of evidence. The woman was found stabbed at her home. She was fully clothed and the murder weapon, a knife, was present at the crime scene • Evidence 1 Laboratory tests revealed that blood found at the crime scene matched the blood type of the suspect. • Evidence 2 The police station has a confession from the suspect, admitting that he killed the victim • Discredit 2 The confession from the suspect was made under extremely pressured circumstances at the police station

20. Exp 1: Results • Significant ‘over’-discounting (D2 < E1) in all conditions • SAME: t(23)=3.74, p<0.05; SIM: t(23)=2.71, p<0.05; DIFF: t(23)=3.13, p<0.05 • Amount of over-discounting greater in SAME vs. SIM, t(23)=1.91, p=0.07; no differences with SAME vs. DIFF, or SIM vs. DIFF • Main effect of DIFF due to physical test as E1

21. Individual analysis Over-discount (D2 < E1) None (D2 = E1) Under-discount (D2 > E1)

22. Conclusions • Difficult to interpret on either BA or Story model • BA model • Does not predict effect of relatedness • Predicts recency effect with mixed evidence • ++- (overweight last item) • Asymmetric rebound effect? • Story model • Predicts story construction only with global judgment • Does not predict over-discounting

23. Experiment 2 • Better test of two models • Order of evidence • LATE - discrediting info presented after both items B E1 E2 D • EARLY – discrediting info presented after first item B E2 D E1 • Relatedness • SAME, DIFFERENT

24. EVIDENCE 1 Footprints outside house match suspect’s EVIDENCE 2 Neighbour says S was outside house on night of crime ? Neighbour is lying because he dislikes S DIFFERENT & EARLY HYPOTHESIS: Suspect S did it P(S)

25. Model predictions • BA predicts recency • Final judgment for early > late • Because +-+ > ++- (overweight last item) • Story model • predicts recency with online but not global • SAME ≠ DIFF for global condition (take account of relatedness)

26. Example of SAME and EARLY • Background You are a juror on a murder case. The victim is a middle-aged woman and the suspect is her ex-husband. You will need to judge whether or not the suspect is guilty on the basis of several pieces of evidence. The woman was found stabbed at her home. She was fully clothed and the murder weapon, a knife, was present at the crime scene • Evidence 2 The police station has a confession from the suspect, admitting that he killed the victim • Discredit 2 The confession from the suspect was made under extremely pressured circumstances at the police station • Evidence 1 The police have a statement from the current wife of the suspect, confessing that the suspect had previously revealed a desire for the victim to be dead

27. Example of DIFF and EARLY condition • Background You are a juror on a murder case. The victim is a middle-aged woman and the suspect is her ex-husband. You will need to judge whether or not the suspect is guilty on the basis of several pieces of evidence. The woman was found stabbed at her home. She was fully clothed and the murder weapon, a knife, was present at the crime scene • Evidence 2 The police station has a confession from the suspect, admitting that he killed the victim • Discredit 2 The confession from the suspect was made under extremely pressured circumstances at the police station • Evidence 1 Laboratory tests revealed that blood found at the crime scene matched the blood type of the suspect.

28. Late condition – less sensitivity to relatedness (but note that over-discounting (E1 > U1) only sig for SAME not DIFF) Early condition –more sensitivity to relatedness NB no diff between B1 & U1 in EARLY rules out asymmetric rebound effect Online judgments:Early vs. Late

29. Problematic for both models • BA cannot explain early condition because does not consider relations between evidence • Story model needs to be applied/adapted to online processing, and somehow explain difference between early and late • Any other models? • Needed: online model that takes relations between items into account, but can also explain early/late difference

30. Speculations • Even with online processing people construct network fragments • As evidence is accumulated it is compactly stored /integrated • Natural to integrate items according to valence (+ve or –ve wrt hypothesis) • E.g., group +ve evidence together

31. C discredits both A and B (irrespective of relatedness) GUILT A B GUILT A&B + + + C Late condition • Positive evidence A and B integrated

32. + + GUILT A B GUILT A + C C Early condition DIFF • B unaffected by C’s discredit of A

33. + GUILT A A* GUILT A + C C Early condition SAME • A* discredited by C too (because similar to A)

34. Ongoing research • Look at both witness and alibi statements • E.g., How does discrediting of an alibi affect evaluation of a positive witness? • Are there asymmetries in dealing with positive vs. negative evidence? • More generally, look at positive and negative evidence (including forensic tests) • Are there differential affects of discrediting? • Can evidence integration idea explain these? • Manipulate deception vs error