Cold Hit Statistics: United States v. Jenkins

1. Cold Hit Statistics: United States v. Jenkins The Science of DNA Profiling: A National Expert Forum Dayton, OH August 13, 2005 Edward J. Ungvarsky, Special Counsel (202) 824-2301, eungvarsky@pdsdc.org Public Defender Service for the District of Columbia

2. Overview • Our cases are prosecuted by the United States Attorney’s Office (USAO) and the Federal Bureau of Investigation (FBI) • We litigated the admissibility of statistics associated with cold hit DNA evidence at a week-long hearing in March-April 2005 in D.C. Superior Court

3. Goals for This Session • Discuss litigation history of U.S. v. Raymond Jenkins • Set forth statistical issues unique to cold hit cases • Extrapolate from Jenkins admissibility hearing to suggest preparation of admissibility challenges more generally

4. Murder Scene – June 4, 1999 • Dennis Dolinger was found in his basement • 25 stab wounds to head • Blood in basement, main floor, stairs to second floor, second floor, outside front of house • Wallet missing from person and house

5. High Profile Case • Public Life • Local Politician • Neighborhood Activist • Scourge of Drug Dealers

6. Steven Watson • Street Musician • Drug Addict • Long Criminal Record • Drug Offenses • Property Offenses • Assaultive Offenses

7. Evidence Against Mr. Watson • Caught using Mr. Dolinger’s credit cards next day in Alexandria, Virginia • Had Mr. Dolinger’s wallet and personal effects in apartment • Observed with blood on his person on the day of the murder • Shortly after murder, observed nearby • Nervous and agitated; credit cards

8. Crime Solved • Mr. Watson is arrested • Charged with the murder • Held in jail pending trial

9. DNA  Second Perpetrator Theory • FBI develops single source DNA profile from crime scene evidence • FBI excludes Mr. Watson as contributor of DNA • USAO develops second perpetrator theory of the crime

10. DNA Evidence Begins to Drive Investigation • FBI searches Combined DNA Index System (CODIS): No hit • Prosecutors ask VA to search VA database: Hit = Raymond Jenkins • Mr. Jenkins is arrested and charged with murder • Jenkins is jailed pending trial in January 2000

11. Starting Over • Charges against Steven Watson are dropped • Mr. Watson is released from jail • Mr. Watson disappears • PDS gets case. • What’s a cold hit?

12. Generation of Match Statistic • Investigators recover biological material from crime scene. • Lab obtains a nuclear DNA profile from that biological material. • Typically a 13 loci profile in recent cases; fewer in older cases. • Analyst generates a Random Match Probability (RMP) statistic, signifying the likelihood that a randomly selected unrelated person from the population would match that DNA profile

13. Traditional DNA Case: Confirmatory ID • Non-DNA evidence exists that a particular suspect committed the crime (e.g., witness ID) • DNA sample taken from already identified suspect and compared to DNA sample taken from crime scene • If the two DNA samples match: confirmatory ID • Report out RMP for that DNA profile

14. Typical Cold Hit Case: Database Search • Compare crime scene DNA profile to a database of convicted-offender profiles • Law enforcement receives notice of any “hits” on individual in database  suspect • Prosecution uses hit to get search warrant for a new DNA sample and profile from the suspect • If new profile matches the profile in convicted offender database and the profile from the crime scene, suspect  defendant • FBI reports out RMP on evidence DNA profile

15. Confirmatory ID Case Suspect is first identified by non-DNA evidence DNA evidence is used to corroborate traditional police investigation Cold Hit Case Suspect is first identified by search of DNA database DNA evidence is used to identify suspect as perpetrator, to exclusion of others, from the outset Traditional police work is no longer focus Difference Between How Suspect Identified

16. Wait! Since the ID of the suspect in a cold hit case isn’t random, do we report something other than the match statistic?

17. Questions • Do you report out same statistics in cold hit case as in confirmatory ID case? • Does fact suspect first identified by search of database provide meaningful information that changes the appropriate statistical methodology?

18. Legal Standard in D.C. for Admissibility of DNA Evidence • Method must be “generally accepted in the relevant scientific community” (Frye v. United States) • Not reliability, not Daubert • In DC, statistical methodology must independently pass muster

19. “General Acceptance” • Consensus versus controversy • Scientists significant in number or expertise • No “Flat Earth-ers”: no crank science • Scientific literature as index of measurement

20. “Relevant Scientific Community” • Relevant scientific community is not just forensic scientists • Forensic scientists “accept, with no qualifier” forensic evidence • Statisticians, mathematicians, geneticists, molecular biologists, population geneticists, etc.

21. Factual Development of Cold Hit Argument • Articles • Treatises • Laboratory Protocols • Meetings • Consultations • David Lynch; scientists • Transcripts • Affidavits  Motion to Exclude DNA Results from Database Search  Hearing: April 2005

22. Surveying the Three (or Four) Statistical Approaches • NRC I : 1992 National Research Council Report • NRC II: 1996 National Research Council Report • Bayesian (aka Balding and Donnelly): Widespread in UK and Western Europe • (DAB): 2000 DNA Advisory Board to FBI

23. The Problem: Ascertainment Bias • First three approaches differ in how they take into account ascertainment bias, a byproduct of identifying an individual from a database search. • Ascertainment bias is statistical effect of fact suspect first identified by search of a database • How must RMP be modified • Consequently, the methods yield results differing by several orders of magnitude. • DAB approach effectively ignores ascertainment bias altogether by allowing reporting of unmodified RMP along with DMP.

24. NRC I & NRC II • Position: Both say ascertainment bias makes the link between suspect and crime scene DNA weaker—less probative. • Rationale: As the size of the database searched increases, so does the chance that you will find a match to the crime scene profile by chance. • Example: If you are looking for someone named “Bill Thompson,” the likelihood of finding match(es) greatly increases if you search US census data versus a local phone book. And your amazement at finding another “Bill Thompson” decreases as database increases

25. NRC I & NRC II (cont.) • NRC I Solution: Set aside previous DNA testing and test the sample at additional loci. If the new testing produces a match, it will be free of ascertainment bias. (Must re-test!) • New, untainted loci • NRC II Solution: Multiply the number of profiles in the database searched by the rarity of the profile in the general population and you get a “database match probability,” or the probability of finding a match in a database of size N • (n(DB) x RMP= DMP)

26. Bayesian (Donnelly/Balding) • Position: Says the ascertainment bias makes the link between suspect and crime scene DNA stronger—more probative. • Rationale: Each additional profile searched in a database eliminates another suspect, making the posterior odds of the suspect being the actual source higher than if the match were random.

27. Bayesian (cont.) • Example: While the likelihood of finding random people named “Bill Thompson” increases if you search US census data versus a local phonebook, you also eliminate most of the population as “not-Bill Thompsons.” • Bayesian Solution: Mathematics is complicated, but the result approximates the Random Match Probability (~RMP).

28. DAB • Position: (Re-)interprets NRC II to allow reporting of RMP statistic about the rarity of the crime scene profile in the general population, in addition to or instead of DMP. • Rationale: The rarity of a profile is always relevant, apart from the database search that links a crime scene profile to a database profile.

29. DAB (cont.) • Example: On one hand, the likelihood of finding a coincidentally matching Bill Thompson increases if you search US census data versus the local phone book. On the other hand, the ex ante estimate of the number of Bill Thompsons in the US should also be reported because estimate is sufficiently rare that we are “close enough.” • DAB Solution: Report RMP and, if you want, also report NRC II (n(DB) x RMP = DMP). • Depends on question that you ask

30. Cold Hit Admissibility Hearing Key question: What statistical methodology for determining significance of match between crime scene sample and suspect sample may be used, where crime scene evidence was first linked to the suspect by a database search?

31. Main Prosecution Arguments • There are at least two questions that statistics can answer in a cold hit case that may be of interest to jurors – RMP and DMP • There is more than one generally accepted way of answering each question, although not all are equally favored • The issue defendant raises involves at most a debate over whether a jury needs to be given DMP in addition to RMP • The Frye question is not what question to ask but whether the means or answering the various different questions are generally accepted (e.g., using product rule to generate RMP)

32. Main Defense Arguments • A database “cold hit” should be used to identify potential suspects but not supply the basis for frequency estimates • Cold hit DNA evidence: • Is statistically less probative than confirmatory IDs • Makes reporting of the rarity of the crime scene DNA profile in the general population (RMP) irrelevant and misleading • must be reported differently than confirmatory hits, with RMP modified to account for nature of search; there is no agreement in the scientific community on how to do this. • Only statistical calculation recommended by NRC I, and the relatively conservative statistics that it generates, is generally accepted

33. Scientific Issues in Hearing • Does NRC II mean what it says or what DAB says it means? • Is there a lowest common statistic everyone can agree on? • Since labs must follow DAB for CODIS access, how much weight should be given to their practices? • Do claims of uniqueness or source attribution make ascertainment bias irrelevant, like fingerprint databases? • Do the statistical methodologies reflect differences in science or in policy, or both?

34. Sources of Answers • Articles and other scientific literature • Affidavits • More broadly you look, the better • Last minute affidavits may be excluded on notice grounds • Transcripts from other cases • Testimony

35. Dr. Chakraborty’s Testimony To present the whole picture, both RMP and DMP should be given to jurors, per DAB. Presentation of RMP alone is not generally acceptable. NRC I is overly conservative.

36. Dr. Bieber’s Testimony Because of advances in technology to 13 loci testing, there no longer is a basis for ascertainment bias. Reporting RMP alone is safe. NRC I and II are overly conservative.

37. AND THEN …

38. Dr. Krane’s Testimony • Based upon review of literature and affidavits, NRC I is feasible today and is the only method everyone would agree is sufficiently conservative.

39. The Ruling • “Today there is a controversy about the method that should be used and it’s not a manufactured controversy, it’s not an insignificant one because there are esteemed members of the community on, I would say, both sides but on all the several sides of this issue.” • “While the Court believes the DAB view of NRC II is appropriate, it would be disingenuous to say there is no controversy here.”

40. The Ruling (cont.) • Court: “Now, where does that leave us? . . . I would like to hear if the government wishes to be heard what [its] view is of whether the Court can find that the NRC I approach is generally accept[ed] . . . .” • AUSA: “I don’t believe there is general acceptance [of NRC I]. You can[not] find general acceptance of a principle that’s never been . . . implemented by any forensic lab.”

41. The Ruling (cont.) • Court: “Where does that leave us, counsel?” • AUSA: “Probably in the Court of Appeals, Your Honor.”

43. Court of Appeals • USAO filed 70-page appeals brief • PDS filed opposition on July 28, 2005 • Oral argument expected in October 2005 • Trial continued to 2006

44. Court of Appeals (correction)

45. Lessons Learned

46. Scrutinize the Statistics • Compared to many other laboratory procedures, statistical methods have been subject to less scrutiny by both prosecution and defense attorneys. • There are ways to question both the general acceptance (Frye) and soundness of the methods (Daubert and Frye).

47. Research, Research, Research • Trust and rely upon others. • Early and often • Locate published research to bolster Frye challenge. • Share bibliographies and pleadings, give referrals to experts, and talk through strategies. • All PDS materials from these cases and other resources are available to defenders are at the NLADA/NACDL forensic website: http://www.nlada.org/Defender/forensics/

48. Expand the Scientific Community • Must look beyond forensic science and population genetics to related fields (e.g., molecular anthropology, statistics, medicine). • Must look to foreign countries’ practices and researchers. • Must contact researchers, especially in related (but neglected) fields.

49. Control the Focus of the Hearing • Don’t get yourself into a battle of the testifying experts • Don’t forget literature • That’s where scientists argue • Most compelling • Have appendices organized into user-friendly format • Affidavits • Use Expert to channel opinions of other scientists • Dan Krane

50. Timely Challenges • It is crucial to get in front of these issues before a mountain of precedent blindly accepts these forensic practices.