The Method Skeptic Debate

1. The Method Skeptic Debate For and Against

2. Forensic Concepts • The nature of expert testimony • Admissibility is determined by legal statute and court precedent; the judge is the gatekeeper • Established criteria for validity and reliability (e.g., Daubert) • These criteria reduce the possibility of biasing the trier of fact • The role of the forensic neuropsychologist (expert) • To provide expert opinion that assists the trier of fact in reaching a legal judgment • The trier of fact, not the expert, determines the truth

3. Forensic Concepts • The nature of expert testimony • Admissibility is determined by legal statute and court precedent; the judge is the gatekeeper • Established criteria for validity and reliability (e.g., Daubert) • These criteria reduce the possibility of biasing the trier of fact • The role of the forensic neuropsychologist (expert) • To provide expert opinion that assists the trier of fact in reaching a legal judgment • The trier of fact, not the expert, determines the truth • Question: can neuropsychologists adhere to these principles?

4. The Method Skeptics • “Neuropsychological evidence generally lacks scientifically demonstrated value for resolving legal issues, and thus, if admitted into court, should be accorded little or no weight.” (Faust, 1991)

5. The Method Skeptics • “Neuropsychological evidence generally lacks scientifically demonstrated value for resolving legal issues, and thus, if admitted into court, should be accorded little or no weight.” (Faust, 1991) • The role of neuropsych: limited in a legal context (diagnosis and research relates minimally to legal questions; clinician’s role unclear)

6. The Method Skeptics • “Neuropsychological evidence generally lacks scientifically demonstrated value for resolving legal issues, and thus, if admitted into court, should be accorded little or no weight.” (Faust, 1991) • The role of neuropsych: limited in a legal context (diagnosis and research relates minimally to legal questions; clinician’s role unclear) • Neuropsych data: poorly predicts everyday functioning; insufficient methods for determining base rates or prior functioning

7. The Method Skeptics • “Neuropsychological evidence generally lacks scientifically demonstrated value for resolving legal issues, and thus, if admitted into court, should be accorded little or no weight.” (Faust, 1991) • The role of neuropsych: limited in a legal context (diagnosis and research relates minimally to legal questions; clinician’s role unclear) • Neuropsych data: poorly predicts everyday functioning; insufficient methods for determining base rates or prior functioning • Neuropsych process: clinical judgment, not data, is the primary guide of decision-making; little value above actuarial calculation

8. The Method Skeptics • “Neuropsychological evidence generally lacks scientifically demonstrated value for resolving legal issues, and thus, if admitted into court, should be accorded little or no weight.” (Faust, 1991) • The role of neuropsych: limited in a legal context (diagnosis and research relates minimally to legal questions; clinician’s role unclear) • Neuropsych data: poorly predicts everyday functioning; insufficient methods for determining base rates or prior functioning • Neuropsych process: clinical judgment, not data, is the primary guide of decision-making; little value above actuarial calculation • Neuropsych practitioner:lacks standard practices; poor capacity to integrate complex data; experience unrelated to diagnostic ability

9. The Method Skeptics • “Neuropsychological evidence generally lacks scientifically demonstrated value for resolving legal issues, and thus, if admitted into court, should be accorded little or no weight.” (Faust, 1991) • The role of neuropsych: limited in a legal context (diagnosis and research relates minimally to legal questions; clinician’s role unclear) • Neuropsych data: poorly predicts everyday functioning; insufficient methods for determining base rates or prior functioning • Neuropsych process: clinical judgment, not data, is the primary guide of decision-making; little value above actuarial calculation • Neuropsych practitioner:lacks standard practices; poor capacity to integrate complex data; experience unrelated to diagnostic ability • Validity of neuropsych: low outside of cases confirmable through other methods (e.g., MRI); hard to evaluate objectively

10. Reply to Method Skeptics • “Clinical evidence can usefully inform legal decision-making… the modern trend has been for courts to be increasingly open to such expert testimony.” (Barth, Ryan, & Hawk, 1991)

11. Reply to Method Skeptics • “Clinical evidence can usefully inform legal decision-making… the modern trend has been for courts to be increasingly open to such expert testimony.” (Barth, Ryan, & Hawk, 1991) • The role of neuropsych: utility has led to a general expansion since 1963 (Jenkins vs. US ruling)

12. Reply to Method Skeptics • “Clinical evidence can usefully inform legal decision-making… the modern trend has been for courts to be increasingly open to such expert testimony.” (Barth, Ryan, & Hawk, 1991) • The role of neuropsych: utility has led to a general expansion since 1963 (Jenkins vs. US ruling) • Neuropsych data: supports a “clinical description,” not a diagnosis per se; clinical judgment is well-informed based on experience

13. Reply to Method Skeptics • “Clinical evidence can usefully inform legal decision-making… the modern trend has been for courts to be increasingly open to such expert testimony.” (Barth, Ryan, & Hawk, 1991) • The role of neuropsych: utility has led to a general expansion since 1963 (Jenkins vs. US ruling) • Neuropsych data: supports a “clinical description,” not a diagnosis per se; clinical judgment is well-informed based on experience • Neuropsych process: importance of data integration is appreciable in individual cases, not in group data; utility goes beyond actuarial calculation

14. Reply to Method Skeptics • “Clinical evidence can usefully inform legal decision-making… the modern trend has been for courts to be increasingly open to such expert testimony.” (Barth, Ryan, & Hawk, 1991) • The role of neuropsych: utility has led to a general expansion since 1963 (Jenkins vs. US ruling) • Neuropsych data: supports a “clinical description,” not a diagnosis per se; clinical judgment is well-informed based on experience • Neuropsych process: importance of data integration is appreciable in individual cases, not in group data; utility goes beyond actuarial calculation • Neuropsych practitioner: experience is poorly operationalized in empirical studies; “watered down” in group data; is important in answering questions beyond “abnormal or not?”

15. Reply to Method Skeptics • “Clinical evidence can usefully inform legal decision-making… the modern trend has been for courts to be increasingly open to such expert testimony.” (Barth, Ryan, & Hawk, 1991) • The role of neuropsych: utility has led to a general expansion since 1963 (Jenkins vs. US ruling) • Neuropsych data: supports a “clinical description,” not a diagnosis per se; clinical judgment is well-informed based on experience • Neuropsych process: importance of data integration is appreciable in individual cases, not in group data; utility goes beyond actuarial calculation • Neuropsych practitioner: experience is poorly operationalized in empirical studies; “watered down” in group data; is important in answering questions beyond “abnormal or not?” • Validity of neuropsych: evaluated as high, especially with the HRB

16. Concepts Revisited • The nature of expert testimony • Bigler (2007): “Unreliable ‘scientific’ testimony… can have an extremely prejudicial impact on the jury in part because of the way in which the jury perceives an expert as an unbridled authority figure.” • Barth et al. (1992): expert testimony “not limited to scientific knowledge;” neuropsychology experts can assist trier of fact by testifying to technical or specialized aspects of their practice • The role of the forensic neuropsychologist (expert) • Faust (1991): Content of testimony is usually confirmable by more valid methods, such as CT or MRI; neuropsych testimony should be considered with “little or no weight” beyond other evidence • Barth et al. (1992): the neuropsychologist offers greater capacity to interpret variables related to mental status than does the layperson

17. Two Questions • Is neuropsychology “a science that does not yet exist?” • Is neuropsychological evidence (and expert opinion) biasing or helpful to the trier of fact? • Keep in mind: the purpose of the case is to answer the legal question at hand, not to establish the accuracy of a diagnosis

18. Skepticism • The method skeptics “remind us that our science is not perfect, and that we must not become so content with our knowledge base, research findings, validity and reliability of methods, and clinical judgments based upon data training and experience, that we fail to engage in critical self-examination.” • “Neuropsychology will continue to grow and have considerably greater impact in the forensic arena if we are not afraid to evaluate the findings objectively, recognize their merit while challenging their assumptions, and learn to improve our methods and understanding from the debate.” - Barth, Ryan, & Hawk, 1991

19. Method Skeptic Debate Anti Pro Admission of neuropsychology expert testimony is growing Clinical judgment is a useful, valid tool Accuracy shown to be high Clinical interpretation leads to accurate prediction of function Established norms allow for population-level comparisons Growing compendium of standardized tests Experience leads to greater capacity to interpret complex cases Neuropsychologist must take on responsibility of ensuring objectiveness • Applicability to courtroom is limited • Decisions are based primarily on clinical judgment • Validity is low in difficult cases • Prediction of function is low • Little knowledge of base-rates • Lack of standardized assessment methods • Poor at detecting malingering • Poor at determining prior function • Limited capacity for complex data integration • Self-fulfilling bias