David Lagnado Division of Psychology and Language Sciences University College London

1. Stories and statistics: What can the Sally Clark case tell us about the psychology of evidential reasoning? David Lagnado Division of Psychology and Language Sciences University College London

2. Evidential reasoning • How do people assess and combine evidence to make decisions? • Legal, Medical, Financial, Social … • Cognitive science approach • What kinds of representations? • What kinds of inference patterns? • How do these compare with ‘normative’ or formal methods of evidential reasoning?

3. Reasoning with legal evidence Legal domain E.g. juror, judge, investigator, media Complex bodies of interrelated evidence Forensic evidence; witness testimony; alibis; confessions etc Need to integrate wide variety of evidence to reach singular conclusion (e.g. guilt of suspect)

4. Story model (Pennington & Hastie, 1986, 1991, 1992) • Evidence evaluated through story construction • ‘Stories involve human action sequences in which relationships of physical causality and intentional causality between events are central’ • Jurors use prior causal knowledge and expectations about story structure to fill in gaps in evidence • Active ‘sense-making’ process to construct an account of what happened • (Evaluative nature of story)

5. Evidential Reasoning • Reasoning from evidence • Use the evidence to construct ‘most plausible’ account of what happened • Generate a causal story based on the evidence • Reasoning about evidence • Assessing the strength/reliability/validity of the evidence • How well does the evidence support the putative hypotheses/stories? think-aloud protocols from jurors in simulated trials suggest that they predominantly engage in former

6. Stories: blessing or curse? • Story is concrete and categorical • Describes a singular causal process • Economy of representation • Easy to communicate • Clearcut basis for decision and action • Identify key variables to blame • Hard to simultaneously compare/evaluate multiple stories (cf Wigmore) • Danger of neglecting alternative accounts • Evidence often gathered/interpreted for a single story (confirmation bias) • The ‘truth’ might not make a good story

7. Binocular rivalry

8. Sally Clark case • Sally & Stephen Clark married, both solicitors • Son Christopher born in 1996 • Died suddenly at home aged 11 weeks • Sally alone with child; noticed he was unwell; ambulance called, but he could not be resuscitated • Postmortem (Dr Williams): • Death from natural causes - lung infection (and bruises consistent with resuscitation attempts) • Body was cremated

9. Sally Clark case • Harry born in 1997 • Died suddenly at home aged 8 weeks • Stephen at home with Sally; but Sally alone with child when discovered unwell; ambulance called, but he could not be resuscitated • Postmortem (Dr Williams): • Suspicious - death from shaking? • Re-examined death of Christopher • Concluded it too was unnatural, with evidence of smothering • Sally Clark charged with murder of both children

10. Prosecution case • Christopher & Harry were smothered • Nb change from Dr Williams’ initial claims of shaking for Harry (error in diagnosis of retinal haemorrhages) • Neither died from SIDS because there were unexplained injuries • Numerous similarities between the two deaths • ‘which would make it an affront to commonsense to conclude that either death was natural, and it was beyond coincidence for history to so repeat itself’

11. Prosecution case • ‘Similarities’ • Babies died at similar ages • Both found unconscious in same room; at same time; shortly after feed • Mother alone with child when found unwell • Father either away or due to go away • (Medical evidence of previous abuse & deliberate injury) • How unlikely are these given that mother is innocent? (beyond coincidence?)

12. Prosecution case: Injuries to Christopher Prior smothering Smothering nosebleed Blood in lungs Torn frenulum Bruises Between lip and jaw Small marks on arms and legs Both fresh & older blood

13. Prosecution case:Injuries to Harry Smothering Shaking/ prior abuse Hypoxic damage to brain Haemorrhages in eyelids Haemorrhages to eyes Rib injuries Spinal injuries Old fracture & dislocation Spinal bleeding & swollen cord

14. Prosecution case:Credibility of witnesses Sally’s testimony in doubt Sally Clark reliability Harry slumped in bouncy chair? Police surgeon says impossible for baby of 8 weeks to slump in bouncy chair Sally Clark states she found Harry slumped in bouncy chair

15. Stephen’s testimony in doubt Sally Clark smothered Harry Stephen lying to protect wife Opportunity/ Motive Stephen Clark reliability Sally Clark alone with Harry Taxi records show Stephen Clark returned home at 8.10pm Stephen Clark states he returned home at 5.30/5.45pm

16. Prosecution case:Statistical evidence • Professor Sir Roy Meadow (Paediatrics) • Report – ‘Sudden unexpected deaths in infancy’ • Risk factors – age of mother (<26), smoker in household, no wage earner • None applied to Clark family • Chance of one SIDS in family= 1 in 8,543 • Chance of two SIDS = 1/8543 x 1/8543 = 1/73 million • ‘…by chance that happening will occur about once every hundred years’

17. Defence case • Sally Clark did not kill her children • They died of natural but unexplained causes • Medical evidence amounts only to suspicion • Two of prosecution experts said cause of deaths ‘unascertained’ • Case hinges upon Dr William’s reliability and competence

18. Defence case: Injuries to Christopher Postmortem effects Haemoderosis Resuscitation attempts Blood in lungs Torn frenulum Bruises Change of opinion Poor conduct of postmortem Low quality photos etc Reliability of Dr Williams

19. Defence case: Injuries to Christopher Haemoderosis Resuscitation attempts Postmortem effects Blood in lungs Torn frenulum Bruises Report of Torn frenulum Report of Bruises Change of opinion Poor conduct of postmortem Low quality photos etc Report from police & hospital Reliability of Dr Williams

20. Defence case: Injuries to Harry Postmortem Natural causes post-death Hypoxic damage to brain Haemorrhages to eyelids Haemorrhages to eyes Rib injuries Spinal injuries Change of opinion Prior error with slides Reliability of Dr Williams

21. Explain Stephen testimony mistake Sally Clark smothered Harry Stephen very unlikely to lie to protect wife if she killed their children Opportunity/ Motive Stephen Clark reliability Sally Clark alone with Harry Stephen admitted lack of knowledge, and mentioned taxi records Stephen Clark states he returned home at 5.30/5.45pm Taxi records show Stephen Clark returned home at 8.10pm

22. Defence case:Statistical evidence Mother >26 No smokers Wage earner • Estimate for probability of one SIDS death questionable • Calculation for two deaths ignores possible genetic & environmental factors SIDS death SIDS death Familial or Environmental factors 2 SIDS death = significantly greater than 1/73 million

23. Verdict • Sally Clark found guilty by 10-2 majority • Imprisoned for life

24. First Appeal:Statistical evidence misleading • Calculation • 1/73 million figure flawed • Calculation ignores possible genetic & environmental factors • Relevance • Probability of two SIDS deaths needs to be compared with probability that mother murders both her children (by taking ratio) • Estimated incidence of this is much lower than of two SIDS deaths ‘it is clearly inadequate to concentrate on a single cause of death. If we make an assessment of the probability of two babies in one family both dying from SIDS, we must equally make a similar assessment of the probability of two babies in one family both being murdered (and so on, for any other causes that may be under consideration)…’ Dawid (2002)

25. Two alternative causes of the deaths (not exhaustive – other causes possible, also possible that one SIDS one murdered etc) Prior probability of murder is lower Prior probability of SIDS is low SIDS Murder 2 deaths Prior to other/medical evidence, probability of double SIDS greater than probability of double murder

26. Sally Clark motive etc Genetic or environmental factors • Two alternative causes (not exhaustive – other causes possible) probability of SIDS is very low SIDS1 Murder1 SIDS2 Murder2 probability of murder is MUCH lower Evidence1 Evidence2 Prior to medical evidence, probability of double SIDS much greater than probability of double murder

27. Appeal dismissed • Court of appeal judgment • No need for expert statisticians to give oral testimony –“it was hardly rocket science” • Defence already pointed out flaws in statistics • What matters is that probability of two SIDS deaths is very low, not exact figure • Statistic might have had larger impact on jury than it should have, but case against Sally Clark was nevertheless overwhelming

28. Second appeal • Discovery of new evidence • Harry had bacterial infection • Known by Dr Williams but not disclosed at trial! • (When jury asked about blood tests for Harry, Williams said norelevant test results) • Plausible cause of Harry’s death • according to 11 independent experts • Also casts doubt on Christopher’s death due to unreliability of Dr Williams

29. Harry’s death Bacterial infection Postmortem Natural causes post-death Micro-biological tests Hypoxic damage to brain Hemorrhages to eyelids Hemorrhages to eyes Rib injuries Spinal injuries Failure to disclose etc Reliability of Dr Williams Conclusions about Christopher

30. Second appeal • Conviction declared unsafe • Sally Clark released 2003 • Postscript • Several other similar convictions involving Meadow subsequently overturned • Meadow struck off medical register 2005; reinstated on appeal 2006 • Williams guilty of serious misconduct 2005 • Sally dies 2007

31. Lessons • Various repercussions for legal domain • Expert witnesses; statistical evidence • For evidential reasoning • Role of causal networks • Reliability of evidence (and experts) • Statistical evidence • Stories and blame • Counterfactual reasoning

32. Causal networks • Key role of causal reasoning borne out by Sally Clark case • But story model needs to be developed • Formal means for representing causal models and inference • Include representation of evidence and reliability (and their interrelations) • (Even if people don’t always do this- they can!)

33. Legal idioms • Evidential reasoning in terms of causal building blocks • Capture generic inference patterns • Reusable and combinable • Qualitative causal structure • Based on Bayesian networks • Akin to schema/scripts Neil, Fenton & Nielsen, 2000 Dawid, Hepler, Lecauri, 2007 Dawid, Hepler, Schum, 2011 Fenton, Lagnado & Neil, 2012

34. Legal idioms • Evidence idiom • Evidence depends on Hypothesis • Evidence is more likely if hypothesis is true • Observed evidence raises the probability of hypothesis Smother Hypothesis Bruises Evidence Smothering causes bruises (probabilistically)

35. Explaining away Evidence is often rebutted Legal idioms Smother Resuscitation Bruises Stephen report Evidence for alternative cause of bruises

36. Legal idioms • Distinguish event from report Hypothesis Christopher smothered Event Bruises Police / hospital report of NO bruises Report Williams report of bruises

37. Legal idioms • Evidence – Reliability idiom Impact of evidence on hypothesis is modulated by its reliability Hypothesis H Bruises Reliability of Williams Reliability Evidence E Williams report Williams slide errors

38. Legal idioms • Reliability of witness reports • Separate factors for reliability Bruises Reliability of Williams Is Williams mistaken? Is Williams biased? Williams report Objectivity Competence Is Williams honest? Veracity From Schum (2001)

39. Legal idioms • Opportunity idiom Sally alone with Harry Reliability Sally smothers Harry Stephen report Opportunity is often a pre-condition of guilt

40. Legal idioms • Motive idiom Sally career driven Sally resentful Use of irony rebuttal Sally murders baby Letters to parents evidence Motive is typically a pre-condition of guilt

41. Combining idioms – alibi evidence Opportunity Sally alone with Harry Stephen lying to protect wife? Sally smothered Harry Conflicting Evidence reports Reliability Stephen memory error? Stephen report 5.30 Taxi record 8.10

42. The big picture • Combining network fragments into a large-scale model

43. Key factors at trial

44. Prosecution case

45. Defence case

46. Cognitive Economy? • How do people do this? • Lab-based studies support the claim that they use idioms for small-scale problems (Lagnado, 2011; Lagnado et al., 2012) • But how does this scale-up? • Story-telling • Use of narrative to simplify? • Reasoning from but not about evidence

47. Stories and Blame • Stories constructed from causal networks • Cohesive narrative to explain events • To attribute blame for negative outcomes • Can involve simplifications, distortions, gap filling etc • Can these be captured by systematic transformations on causal networks? • Assume low probability variables are false (egnot SIDS) • Collapsing variables (eg natural causes->SIDS) • Leaving out weak causal links?

48. Stories and Blame • At trial • Prosecution presented one cohesive story – Sally smothered both babies • Explains most of the medical evidence • Explains unreliability of Stephen & Sally testimony • ‘Supported’ by statistical evidence • Defence did not present one single story, but numerous disconnected pieces to explain the different injuries etc

49. Possible line of juror reasoning? • Jurors reject SIDS due to extreme rarity • Neglect low base-rate of smothering because this was never raised at trial • Accept smothering because: • it gives ‘simple’ explanation of injuries • (and explains inconsistent testimonies) • Assigns blame to someone • A ‘plausible’ story?

50. Stories and Blame • Importance of causal story that assigns blame? • At second appeal • New ‘story’ in which Harry died from infection and Dr Williams & Meadow were blamed • Aftermath & Media • Professor Meadow’s statistical errors are highlighted NB exp to test people’s memory of case?