Juror Understanding of Random Match Probabilities

1. Juror Understanding of Random Match Probabilities Dale A. Nance Case Western Reserve University August, 2007

2. Focus of Presentation • What we know about how jurors react to testimony reporting a match between the defendant and the perpetrator and presenting a “random match probability” (RMP) • “Experiments” assessing juror reactions

3. Eight Common Hypotheses About Cognitive Error by Jurors • 1. The Prosecutor’s Fallacy • 2. Neglect of Lab Error • 3. Improper Combination Strategies • 4. Vividness • 5. Defense Attorney’s Fallacy • 6. Defense Attorney’s (Extreme) Fallacy • 7. The Inversion Fallacy • 8. Misaggregation

4. “The chance of a coincidental match with an innocent man is 1 in 40,000.” What the Expert Says 1. The Prosecutor’s Fallacy

5. “The chance of a coincidental match with an innocent man is 1 in 40,000.” What the Expert Says “The chance that the accused in innocent is 1 in 40,000, so the odds that he is guilty must be 39,999 to 1.” What the Jurors Think 1. The Prosecutor’s Fallacy

6. “The chance of a coincidental match with an innocent man is 1 in 40,000.” What the Expert Says 2. Neglect of Lab Error

7. “The chance of a coincidental match with an innocent man is 1 in 40,000.” What the Expert Says “The chance that the accused, though innocent, would be implicated by either coincidence or lab error is 1 in 40,000.” What the Jurors Think 2. Neglect of Lab Error

8. “The chance of a coincidental match with an innocent man is 1 in 40,000.” “The chance of a false positive lab error is about 1 in 1,000.” What the Expert Says 3. Combination Errors (Averaging)

9. “The chance of a coincidental match with an innocent man is 1 in 40,000.” “The chance of a false positive lab error is about 1 in 1,000.” What the Expert Says “The chance that the accused , though innocent, would be implicated by a coincidental match or lab error is 1 in 20,500.” What the Jurors Think 3. Combination Errors (Averaging)

10. “The chance of a coincidental match with an innocent man is one in a billion.” What the Expert Says 4. The Vividness Hypothesis

11. “The chance of a coincidental match with an innocent man is one in a billion.” What the Expert Says “One in a billion! That’s all I need to know. Hang the bastard!” What the Jurors Think 4. The Vividness Hypothesis

12. “The chance of a coincidental match with an innocent man is 1 in 40,000. Yes, out of 12,000,000 adult men, about 300 will match.” What the Expert Says 5. The Defense Attorney’s Fallacy

13. “The chance of a coincidental match with an innocent man is 1 in 40,000. Yes, out of 12,000,000 adult men, about 300 will match.” What the Expert Says “If 300 men will match, then this DNA evidence tells us nothing. I should just decide the case on the eyewitness evidence.” What the Jurors Think 5. The Defense Attorney’s Fallacy

14. “The chance of a coincidental match with an innocent man is 1 in 40,000. Yes, out of 12,000,000 adult men, about 300 will match.” What the Expert Says 6. The Defense Attorney’s (Extreme) Fallacy

15. “The chance of a coincidental match with an innocent man is 1 in 40,000. Yes, out of 12,000,000 adult men, about 300 will match.” What the Expert Says “If 300 men will match, then the chance the accused is guilty must be only 1 in 300.” What the Jurors Think 6. The Defense Attorney’s (Extreme) Fallacy

16. “The chance of a coincidental match with an innocent man is 1 in 40,000.” What the Expert Says 7. The Inversion Fallacy

17. “The chance of a coincidental match with an innocent man is 1 in 40,000.” What the Expert Says “The chance that the accused in guilty is just 1 in 40,000. This prosecutor must be from Durham.” \ What the Jurors Think 7. The Inversion Fallacy

18. “The chance of a coincidental match with an innocent man is 1 in 40,000.” What the Expert Says 8. Misaggregation

19. “The chance of a coincidental match with an innocent man is 1 in 40,000.” What the Expert Says “Without the DNA evidence, I would place the odds of guilt at 2:1 against. With this DNA evidence, the odds of guilt are about 2:1 for.” What the Jurors Think 8. Misaggregation

20. For a RMP = 1 in 40,000, and considering only the chance of: Coincidental match, posterior odds should be 40,000 times the prior odds: Coincidental match or lab error (at a rate of 1 in 1,000), posterior odds should be about 1000 times the prior: Coincidental match, lab error, or other sources of error (like police planting of evidence), assessed by the average juror at about 1 in 50, the posterior should be about 40 times the prior: PRIOR → POST. ODDS ODDS 1:2 → 20,000:1 1:2 → 500:1 1:2 → 20:1 8. Misaggregation: How Bad Is It?

21. 8. Misaggregation:What Can Be Done About it? • 1. Give RMP testimony in the form of probabilities focused on the defendant, rather than frequencies focused on the population: • “The probability that defendant would match if he were innocent is 1 in 40,000.” rather than • “1 in 40,000 people in the population share this DNA profile.”

22. 8. Misaggregation:What Can Be Done About it? • 2. Give testimony explaining the RMP by showing results of hypothetical Bayes’ Rule calculations. For example, with RMP= 1 in 40,000 and ignoring other sources of error: Prior Probability→ Posterior Probability 1/10 of 1% → 97.56% 1% → 99.75% 20% → 99.99% 50% → 99.99% 70% → 99.99%

23. 8. Misaggregation:What Can Be Done About it? • Incorporating information about lab error rates into the calculation produces lower posterior probabilities: Prior Prob.→ Post. Prob. Post. Prob. (ignoring lab error) (incorp. lab error) 1/10 of 1% → 97.56% 49.42% 1% → 99.75% 90.79% 20% → 99.99% 99.59% 50% → 99.99% 99.90% 70% → 99.99% 99.96%

24. Conclusions • Pro-prosecution fallacies: extant but correctible by argument or by restrictions on form of RMP presentation • Pro-defense fallacies: extant but of declining importance as RMP becomes very small • Pro-defense error (misaggregation): serious but potentially amenable to Bayesian instruction