Thinking about Evidence

1. Thinking about Evidence David Lagnado University College London

2. Leonard Vole accused of murdering a rich elderly lady Miss French Romaine, Vole’s wife, was to testify that he was with her at time of murder Vole had befriended French and visited her regularly including night of murder But instead Romaine appears as witness for prosecution Testifies that Vole was not with her, returned later with blood on his jacket, and said “I’ve killed her” Vole needed money French changed her will to include him; shortly after he enquired about luxury cruises Maid testified Vole was with French at time of death Blood on Vole’s jacket same type as French Letters written by Romaine to lover – reveals her plan to lie and incriminate Vole Vole is acquitted!

3. Evidential reasoning How do people reason with uncertain evidence? How do they assess and combine different items of evidence? What representations do they use? What inference processes? How do these compare with normative theories?

4. Reasoning with legal evidence Legal domain E.g. juror, judge, investigator, media Complex bodies of interrelated evidence Forensic evidence; witness testimony; alibis; confessions etc Need to integrate wide variety of evidence to reach singular conclusion (e.g. guilt of suspect)

5. Descriptive models of juror reasoning Belief adjustment model(Hogarth & Einhorn, 1992) Sequential weighted additive model Over-weights later items Ignores relations between items of evidence Story model(Pennington & Hastie, 1992) Evidence evaluated through story construction Holistic judgments based on causal models No formal, computational or process model

6. Descriptive models of juror reasoning Coherence-based models(Simon & Holyoak, 2002) Mind strives for coherent representations Evidential elements cohere or compete Judgments emerge through interactive process that maximizes coherence Bidirectional reasoning (evidence can be re-evaluated to fit emerging conclusions)

7. How should people do it? Bayesian networks? Nodes represent evidence statements or hypotheses Directed links between nodes represent causal or evidential relations Permits inference from evidence to hypotheses (and vice-versa) Guilt Maid Blood Cut Vole is guilty Blood on Vole’s cuffs Maid testifies that Vole was with Miss French Vole cut wrist slicing ham

8. Partial BN of ‘Witness for Prosecution’

9. Partial Bayesian net for Sacco and Vanzetti trial

10. Applicable to human reasoning? Vast number of variables Numerous probability estimates required Complex computations

11. Applicable to human reasoning? Fully-fledged BNs unsuitable as model of limited-capacity human reasoning BUT – a key aspect is the qualitative relations between variables (what depends on what) Judgments of relevance & causal dependency critical in legal analyses And people seem quite good at this! Blood match raises probability of guilt Alibi lowers it (not much!) Guilt Blood Alibi + -

12. Qualitative causal networks(under construction!) • People reason using small-scale qualitative networks • Require comparative rather than precise probabilities • Guided by causal knowledge • More formalized & testable version of story model?

13. Empirical studies • Discrediting Evidence • Alibi Evidence

14. Discredited evidence How do people revise their beliefs once an item of evidence is discredited? When testimony of one witness is shown to be fabricated, how does this affect beliefs about testimony of other witnesses, or even other forensic evidence? E.g., Romaine’s discredited testimony Involves a distinctive pattern of inference

15. Explaining away P(G|B) > P(B) Finding out B raises probability of G Guilt Vole is guilty of murder Blood on Vole’s cuffs Blood Cut P(G|B&C) < P(G|B) Finding out C too lowers the probability of G Vole cut himself Despite its simplicity and ubiquity, this pattern of inference is hard to capture on most psychological models of inference (e.g., associative models, mental models, mental logic)

16. Discrediting vs. direct evidence Guilt Guilt Blood Blood Weighted additive model Standard regression model Cut Cut Bayesian network model Causal model CUT only becomes relevant to guilt given BLOOD Important to distinguish ‘explaining away’ from simply adding (negative) evidence

17. Experimental questions Do people use causal models to reason with evidence in online tasks? Do they model discrediting evidence by ‘explaining away’? How does the discredit of one item of evidence affect other items?

18. EVIDENCE 1 Neighbour says that suspect has stolen previously EVIDENCE 2 Neighbour says he saw suspect outside house on night of crime EVIDENCE 1 Footprints outside house match suspect’s ? Does the discredit of item 2 affect item 1? Neighbour is lying because he dislikes suspect NO when different source Scenario: House burglary, local man arrested HYPOTHESIS: Local man did it YES when same source

19. Extension of discredit When do people extend the discredit of one item to other items? SAME E.g. two statements from same neighbour SIMILAR E.g. two statements from two different neighbours DIFFERENT E.g., one statement and one blood test Causal model approach would expect people to distinguish SAME from DIFFERENT cases

20. BN models Witness A Blood test Witness B Witness GUILT GUILT Discredit Discredit Same/Similar Different

21. Experiment 1 Mock jurors given simplified criminal cases Four probability judgments (of guilt) Baseline Stage 1 (Evidence 1) Footprint match Stage 2 (Evidence 2) Neighbour sees suspect Final (Discredit 2) Neighbour is lying Compare judgments at Final stage and Stage 1 Does discredit return judgments to Stage 1? Vary relations between items of evidence SAME, SIMILAR, DIFFERENT source

22. Witness1 Witness2 Discredit2 Both items undermined Forensic1 Witness2 Discredit2 Results • Final judgments significantly lower than at Stage 1 for all conditions • Discredit does not simply remove item 2; also affects belief in item 1 • When discredit presented LAST, it is extended regardless of relations between items

23. Summary Discrediting information extended regardless of relation to other evidence This pattern is consistent with Belief Adjustment model Recency effect leads to over-weighting of discrediting information Neglect relations between items Further test of BAM: manipulate order of evidence presentation

24. Experiment 2 Vary order of presentation of evidence LATE……E1 E2 D2 EARLY….E2 D2 E1 Both orders ‘ought’ to lead to same conclusions Relatedness SAME, DIFFERENT

25. Witness1 Witness2 Discredit2 Both items undermined Forensic1 Witness2 Discredit2 Results: Late condition • Final judgments lower than at Stage 1 for both conditions • Discredit does not simply remove item 2 • Replicates EXP 1 • When discredit presented LAST, it is extended regardless of relations between items

26. Witness1 Discredit1 Witness2 Both items undermined Discredit1 Only 1st item undermined Witness1 Forensic1 Results: Early condition • Pattern of judgments differ for SAME and DIFF • SAME • Final = Stage 2 • DIFF • Final > Stage 2 • Appropriate sensitivity to relation between items • When discredit presented EARLY, only extended to related items

27. Problematic for current models Why are people ‘rational’ in early but not late condition? Belief Adjustment model Cannot explain early condition because does not consider relations between evidence Story model Cannot explain bias in late condition (and needs to be adapted to online processing)

28. Coherence-based/grouping account Mind strives for most coherent representation Evidence grouped as +ve or -ve relative to guilt +ve and -ve groups compete, but within-group items mutually cohere (irrespective of exact causal relations) When an item of one group is discredited, this affects other items that cohere with it

29. LATE condition Incriminating evidence grouped together (regardless of source) Discredit affects the group (not just individual item) GUILT A B + + D + +

30. EARLY condition First item of evidence discredited Second item only discredited if from related source No grouping effect + GUILT B A + D + +

31. Study 3 Grouping hypothesis predicts that coherent groupings only emerge with elements that share the same direction (cf. Heider, 1946) Therefore discredit extended when evidence items both +ve or both -ve, but not with mixed items

32. Design Four evidence conditions A+, B+, discredit B+ A-, B-, discredit B- A+, B-, discredit B- A-, B+, discredit B+ Two levels of relatedness: similar and different Predictions 1&2 non-mixed -> discredit affects both items 3&4 mixed -> discredit affects only second item

33. Examples: Condition 2 - - different Lab tests reveal no footprint match Neighbour says she was with suspect at time of crime Neighbour lying because in love with suspect Evidence 1 Evidence 2 Discredit

34. Examples: Condition 3 + - different Neighbour lying because in love with suspect Lab tests reveal footprint match Neighbour says she was with suspect at time of crime Evidence 1 Evidence 2 Discredit

35. Results

36. Summary Grouping hypothesis supported Discredit extended when items share common direction, not when mixed Mutually coherent elements stand or fall together (even when no clear causal relation between them) Romaine & Agatha Christie knew this!

37. Alibi evidence Often crucial evidence (if true, absolves suspect) Treated with suspicion Hard to generate (even if innocent) Very little formal or empirical work Ongoing psychological studies – what makes a good alibi? (e.g., how much detail is best) Also interesting from normative viewpoint

38. Witness vs. Alibi models Suspect committed crime H E E* A E H Suspect motivated to lie Suspect at crime scene D Witness report of suspect at crime scene Suspect claims he was not at crime scene + + + + - + With impartial witness – knowing that suspect was at crime scene ‘screens off’ witness report from guilt judgment In alibi case – if suspect says he wasn’t there, but he was, this raises likelihood of guilt (beyond that if you just find out he was there) Even though P(H|A)<P(H) P(H|E&E*)=P(H|E) P(H|E&A)>P(H|E) To understand alibi evidence – need to represent potential deception

39. Pilot study Compare discredit of witness vs. alibi evidence Manipulate reason for discredit Deception (X was lying in his statement) Error (X was mistaken in his statement) Mock jurors given crime scenarios 3 judgments of guilt Baseline After statement (alibi/witness) After discredit of statement

40. Results Witness – discredit returns belief to baseline (j1 = j3) irrespective of reason Alibi – discredit returns belief to baseline in error condition, but greatly enhances guilt in deception condition Fits with causal network predictions

41. General alibi model H E A Suspect motivated to lie D Suspect claims he was not at crime scene Case 1: Suspect provides alibi Higher motivation to lie if guilty than if innocent (hence link from H to D) Given alibi, discovery of E incriminates via two routes E raises likelihood of H directly E raises likelihood of H indirectly (via its effect on D) + + + - No screening-off ie P(H|E&A) > P(H|E)

42. General alibi model H E A Friend motivated to lie D Friend claims suspect was not at crime scene Case 2: Close relative/friend provides alibi AND they know whether or not suspect is guilty Higher motivation to lie if guilty than if innocent (hence link from H to D) Given alibi, discovery of E incriminates via two routes + + - + No screening-off ie P(H|E&A) > P(H|E)

43. General alibi model H E A Case 3: Close relative/friend provides alibi BUT they do NOT know whether suspect is guilty Motivation to lie irrespective of actual guilt or innocence of suspect (effectively no link from H to D) Given alibi, discovery of E incriminates only via direct route + Friend motivated to lie D - + Friend claims suspect was not at crime scene Screening-off ie P(H|E&A) = P(H|E)

44. General alibi model H E A Case 4: Impartial stranger provides alibi AND they do NOT know whether suspect is guilty Low Motivation to lie AND this is unrelated to actual guilt or innocence of suspect (effectively no link from H to D) Given alibi, discovery of E incriminates only via direct route + Stranger motivated to lie D - + Stranger claims that suspect was not at crime scene Screening-off ie P(H|E&A) = P(H|E)

45. Experimental study Do people conform to these models? Background info: eg Victim is attacked on her way home … suspect is arrested Alibi: ‘suspect was elsewhere at time of crime’ Manipulate who provides the alibi Discredit Alibi e.g., suspect seen on CCTV near crime scene at time of crime

46. Results so far > = = = • Scenarios don’t clarify that close friend knows H (as shown by subjects’ judgments about this) • Strong order effects --- • ALIBI, CCTV >> CCTV, ALIBI

47. Conclusions so far • People construct and use causal models • ‘Explaining-away’ inferences • Grouping of variables can lead to biases • Sensitive to Alibi model • Puzzling order effect with Alibis • Judgment involves both causality and coherence?

48. Thank you! Leverhulme/ESRC Evidence project Nigel Harvey Phil Dawid Amanda Hepler Gianluca Baio Students Miral Patel Nusrat Uddin Charlotte Forrest