Statistical Issues for Developing Alzheimer Screening Tests

1. Statistical Issues for DevelopingAlzheimer Screening Tests J. Wesson Ashford, M.D., Ph.D. Stanford / VA Alzheimer’s Center VAMC, Palo Alto, California & Helena C. Kraemer, Ph.D. Department of Psychiatry and Behavioral Sciences Stanford University, Palo Alto, California June 19, 2005 Washington, D.C. Slides at: www.medafile.com(Dr. Ashford’s lectures)

2. Screening for Alzheimer’s DiseaseWhat does it mean? • Annual assessments (or bi- or semi-) • Positive screen recommends more tests • Contrast with current lack of system • 50% not diagnosed until moderate AD • Screening will progressively improve • Change over time can be detected

3. AD Can Be Readily Diagnosed • A diagnosis of Alzheimer’s disease can be made with a high degree of certainty • Using NINCDS-ADRDA criteria, accuracy in autopsy-verified cases is approximately 90% • Diagnosis is a 2-step process: • Detection through screening (test vs. family concern) • Confirmation through patient history and physical, caregiver interview, brain imaging, and appropriate laboratory studies McKhann G et al. Neurology. 1984;34:939-944. Kazee AM et al. Alzheimer Dis Assoc Disord. 1993;7:152-164. Ashford JW et al, Psychiaric Annals, 1996;26:262-268.

4. AD is Under-diagnosed • Early Alzheimer’s disease is subtle, the diagnosis continues to be missed • it is easy for family members to avoid the problem and compensate for the patient • physicians tend to miss the initial signs and symptoms • Less than half of AD patients are diagnosed • Estimates are that 25% to 50% of cases remain undiagnosed • Diagnoses are missed at all levels of severity: mild, moderate, severe • No definitive laboratory test for diagnosing AD exists • Efforts to develop biomarkers, early recognition by brain scan have not provided a screening methodology Evans DA. Milbank Quarterly. 1990; 68:267-289

5. Reasons to Diagnose Alzheimer’s Disease Early Social • Undiagnosed AD patients face avoidable problems • social, financial • Early education of caregivers • how to handle patient (choices, getting started) • Advance planning while patient is competent • will, proxy, power of attorney, advance directives • Reduce family stress and misunderstanding • caregiver burden, blame, denial • Promote safety • driving, compliance, cooking, etc. • Patient’s and Family’s right to know • especially about genetic risks • Promote advocacy • for research and treatment development

6. Reasons to Diagnose Alzheimer’s Disease Early Medical • Early diagnosis and appropriate intervention may lessen disease burden and early treatment may improve overall course substantially • Neurophysiological pathways in patients with AD are still viable and are a target for treatment • Specific treatments now available (anti-cholinesterases, memantine) • Improve cognition • Improve function (ADLs) • Delay conversion to AD from Mild Cognitive Impairment • Slow underlying disease process, the sooner the better • Decreased development of behavior problems • Delay nursing home placement, possibly over 20 months • Delay nursing home placement longer if started earlier

7. UNDERLYING CONTINUUM OF ALZHEIMER SEVERITY- progressive disruption of neuroplasticity -(Ashford & Jarvik, 1985; Ashford et al., 1995; Ashford, 2004)?? unidimensional, but how to measure ?? • CROSS-SECTIONAL MEASURES • DEMENTIA SEVERITY (cognitive, ADL) • COGNITIVE SCALE SCORE • Z-SCORE • PRINCIPAL COMPONENT ANALYSIS • BRAIN ATROPHY, DYSFUNCTION • AUTOPSY MEASURES: plaques, tangles • TIME TO DEATH • LONGITUDINAL MEASUREMENT • TIME INTO THE DISEASE PROCESS • Considerable heterogeneity in disease clinical presentation and brain distribution

8. ALZHEIMER’S DISEASE CONTINUUM derived from CERAD dataset AAMI / MCI/ early AD -- + -- DEMENTIA Ashford et al., 1995; Ashford & Schmitt, 2001

9. Alzheimer’s Disease Categories NORMAL MILD COGNITIVE IMPAIRMENT (MCI) CRITERIA • Memory complaint, preferably corroborated by an informant • Objective memory impairment • Normal general cognitive function • Intact activities of daily living • Not demented American Academy of Neurology Petersen et al., 2001 – Neurology 56:1133 Earlier descriptions by: Jonker, Hooyer, 1990 Flicker, Ferris, Reisberg, 1991 Zaudig, 1992 DEMENTIA OF ALZHEIMER TYPE (DSM-IV - APA) Kraemer et al., 2004

10. ALZHEIMER’S DISEASE CATEGORIZATION Yesavavage et al., 2002 Markov Chain model

11. Current Approaches to Early Assessment • Genetic vulnerability testing (trait not diagnosis) • Vulnerability factors • education, occupation, head injury (not diagnosis) • Early concern • Alzheimer Association’s 10 warning signs, ADL dysfunction • Dementia severity assessment tools (lack early power) • Positive diagnostic tests (too invasive for screening) • CSF – tau levels elevated, amyloid levels low • Brain scan – PET – • NFTs: DDNP • Amyloid: Thioflavin-S, Congo-red derivatives Need new screening tools (6th vital sign in elderly)

12. Available Short TestsUsed for Screening • MMSE 10 -- 15 min • Too long • 7-Minute Screen 7 – 10 min • Too complex • Clock Drawing Test 2 – 4 min • Not sensitive • Mini-cog 3 – 5 min • To be considered • Memory Impairment Screen 4 min • To be considered • Need to know the cost of administering the test • Need to have prospective measurement of sensitivity and specificity for the target population • Need to develop progressively better tests

13. Control: What happens without screening? Total Population Risk=P P’ P Have AD No effective intervention Do not have AD Helena Kraemer, 2003

14. Testing: What happens with testing? Total Population P’ P AD No AD Se Se’ Sp’ Sp Unnecessary intervention OK No effective intervention Effective intervention $ Testing $Testing $ Testing $ Testing $ Intervention $ Intervention Iatragenic Damage? Clinical Wash Clinical Wash Clinical Gain Major(?) Loss Minor (?) Loss Minor(?) Loss Major(?) Gain Some gain Helena Kraemer, 2003

15. $W = Cost–Worthiness Calculation • I = incidence (new occurrences each year, by age) • $T = cost of test, time to take (Subject, Tester) • Se = sensitivity of test = True positive / I • Sp = specificity of test = True negative / (1-I) • Cost: • $B = benefit of a true positive diagnosis • Estimate: (100 years – age ) x $1000 • Save $50,000 NH cost / 1year (after treatment cost deduction) • $C = cost of a false positive diagnosis • $500 for further evaluation (time, stress of suspecting dementia) • True negative (real peace of mind) (no price) • False negative = false peace of mind (no price) $W = ($B x I x Se) – ($C x (1-I) x (1-Sp)) - $T Kraemer, Evaluating Medical Tests, Sage, 1992

16. JW Ashford, MD PhD, 2003

17. Se, Sp Se, Sp

18. Sp, Se

19. See: Raber et al., 2004

23. (CERAD dataset)

24. Mendiondo et al., 2003 JW Ashford, MD PhD, 2003

25. Issues in Screening • ROC analysis provides independent values of test performance • how the screening test values affect the normal and patient populations • plots of their relationship with respect to each other (specificity vs sensitivity) • data must be derived from the represented population!!! • The value of the test must be calculated with respect to the risk of the disease • In the specific population to which it is being applied • Risk is affected by age, genotype, many other factors • Accounting for a priori probability is Bayesian analysis • The cost-benefit must be assessed: • Apply the test cost and the costs of false positive and false negative results • Apply the benefits of correct positive and negative results • Alzheimer’s disease is not a dichotomous diagnosis but a continuum • Diagnosis would be better described with a probabilistic statement • Item Response Theory would better calculate probability (Modern Test Theory) Item Response Theory and Factor Analysis allow combination of test components Kraemer, 1992; Ashford & Schmitt, 2001

26. MMSE items Mini-Mental State Exam items Based on Ashford et al., 1989; 1995; applied to CERAD data set

27. See: Hambleton et al., 1990; Ashford & Schmitt, 2001)

28. MMSE Item-Response Analysis

29. Summary Requirements for Screening Test Evaluation • Sensitivity, specificity for the population to which test is to be applied • values change with level of disease being screened (must be prospective) • Portion of population at risk • Risks according to age, gender, genotype, family hx, education, etc. • Cost of test -$1, $10, $100, $1000 • Need brief screen to replace MMSE (ASAP) - $10 Medicare allowance • Costs of false positive, false negative tests • Benefits of true positive, true negative • Longevity is increasing over time, needs to be factored in. • Cascaded tests – preliminary screen, confirmatory exam • Longitudinal tests may provide much more reliable information • Different tests for clinic (cascaded), research (outcomes) • Clinic: brief screen, brain scan; Research: CSF - is disease stopped? • Benefit to society relative to other societal needs.

30. Dementia Screening Test Requirements for the Future • For Alzheimer’s disease (other tests for non-AD) • Validated against more stringent tests – brain scans, CSF measures • Specificities and sensitivities valid for comparison with other tests • Item Response Theory analysis of discriminatory power on disability continuum • Multiple platforms: • Doctor’s offices • Best if computerized for rapid, objective assessment • World-Wide Web – based testing, • CD-distribution • KIOSK administration – drug stores, shopping malls • Very brief (about 1-minute) • Multiple test forms • so it can be repeated often, e.g., every 3 months • Cost-effective yearly after age 50 • repeatable every 3 months over 65 years of age or with concerns • Sensitive to change over time • Nominal cost