1 / 22

The Future of Mental Health Measurement

The Future of Mental Health Measurement. Robert D. Gibbons University of Chicago. David J. Kupfer David J. Weiss Paul Pilkonis. Ellen Frank R. Darrell Bock.

atristan
Download Presentation

The Future of Mental Health Measurement

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Future of Mental Health Measurement Robert D. Gibbons University of Chicago David J. KupferDavid J. Weiss Paul Pilkonis Ellen Frank R. Darrell Bock Supported by NIMH Grant R01-MH-66302. The CAT-MH is distributed by Adaptive Testing Technologies (www.adaptive testingtechnologies.com) for which Drs. Gibbons, Kupfer, Frank, Weiss and Pilkonis have financial interests.

  2. Classical vs. IRT Measurement Classical Measurement Model

  3. Classical vs. IRT Measurement IRT

  4. What is CAT? Arithmetic Algebra Calculus Imagine a 1000 Item Math Test

  5. Background: Bi-factor Model • Psychiatry issue: Multidimensionality = excess correlation within domains violating conditional independence assumption • Fitting unidimensional models to multidimensional data. • Small item banks (e.g. PROMIS 28 items for depression) • Underestimate posterior variance • Greater variability of scores between and within individuals • Solution is to base CAT on multidimensional IRT

  6. Quality of Life Example Gibbons R.D., Bock R.D., Hedeker D., Weiss D., Segawa E., Bhaumik D.K., Kupfer D., Frank E., Grochocinski V., Stover A. Full-Information Item Bi-Factor Analysis of Graded Response Data. Applied Psychological Measurement, 31, 4-19, 2007.

  7. Quality of Life Example

  8. CAT • Traditional – all subjects get all items • Subjects get different items based on severity • Smallest number of items for fixed precision • Develop large item banks that completely characterize a disorder such as depression • Select items dynamically based on responses

  9. Paradigm Shift • Traditional Measurement • Fix items allow precision to vary • IRT-Based CAT • Fix precision allow items to vary • Change precision depending on application • Epidemiology – fewer items lower precision (se=.4) • Primary care screening – medium precision (se=.3) • RCTs – more items high precision (se=.2) • Suicide Screen – C-SSRS items (2-4)

  10. CAT-DI and CAD-MDD • Focus on Major Depressive Disorder (MDD) • MDD is a leading factor in US health care costs • Created 1008 item bank (DEP, ANX, MANIA) • Over 800 items remained after calibration • CAT-DI – Measure Depressive Severity • CAD-MDD – Diagnose Depression • CAT-ANX (anxiety) CAT-MANIA (bipolar)

  11. CAT-DI RESULTS Gibbons R.D., Weiss D.J., Pilkonis P.A., Frank E., Moore T., Kim J.B., Kupfer D.K. The CAT-DI: A computerized adaptive test for depression. Archives of General Psychiatry, 69, 1104-1112, 2012. Gibbons R.D., Weiss D.J., Pilkonis, P.A., Frank E., Moore T., Kim J.B., Kupfer D.J. Development of the CAT-ANX: A computerized adaptive test for anxiety. American Journal of Psychiatry, 171, 187-194, 2014. Achtyes E.D., Halstead S., Smart L., Moore T., Frank E., Kupfer D., Gibbons R.D. Validation of computerized adaptive testing in an outpatient non-academic setting. Psychiatric Services, in press.

  12. CAT-DI Depression Scores vs. SCID Diagnosis

  13. HAM-D Scores vs. SCID Diagnosis

  14. PHQ-9 Scores vs. SCID Diagnosis

  15. CAD-MDD – Decision Tree • Computerized Adaptive Diagnosis • Begin with 100 DSM-IV MDD items • Obtain SCID DSM-IV MDD Diagnosis • Fit Decision Tree using Random Forest • Develop live CAD-MDD and cross validate

  16. CAD-MDD – Decision Tree

  17. CAD-MDD – Results Sensitivity: 0.95 Specificity: 0.87 Average of 4 Items Max=6 Gibbons et.al., J. of Clinical Psychiatry (2013)

  18. Rates of Detection and Service Utilization • Emergency Department U of Chicago (n=1000) • 26% MDD positive screens (>50% confidence) • 22% MDD positive screens (>90% confidence) • 7% MDD Positive + moderate or severe CAT-DI • 3% suicide screen positive • 3-fold increase in ED visits in past year • moderate/severe vs. none/mild • 4-fold increase in hospitalizations in past year moderate/severe vs. none/mild • None of these patients had a psychiatric indication • Primary Care Spain and US Latino Samples (n=1000) • 33% MDD positive screens (>50% confidence) • 25% MDD positive screens (>90% confidence) • 9% MDD Positive + moderate or severe CAT-DI

  19. Independent Validation Study • Highly Comorbid Community MH sample (n=150) • High sensitivity 0.96 maintained for entire sample • Specificity 1.0 for MDD vs Control • CAT-DI, CAT-ANX, CAT-MANIA all predict Dx • MDD 28-fold across scale • GAD and current BP each 12-fold across scale • 97% accurately reflected mood • 86% preferred computer interface (10% preferred pp) • 97% Comfortable taking CAT-MH • 98% Answered honestly Achtyes E.D., Halstead S., Smart L., Moore T., Frank E., Kupfer D., Gibbons R.D. Validation of computerized adaptive testing in an outpatient non-academic setting. Psychiatric Services, in press.

  20. Future Directions Screening and monitoring in primary care Inexpensive phenotyping for GWAS studies Psychiatric epidemiology Comparative effectiveness and safety Differential Item Functioning – Global Health Kiddie CAT - Developmental shifts – vertical scaling Spend billions on biology but validate using stone age clinical measurements Autism, PTSD, RDoC, … Military – Suicide RR=4 within 4 years of discharge Cloud computing environments

  21. Relevant Publications Gibbons R.D., & Hedeker D.R. Full-information item bi-factor analysis. Psychometrika, 57, 423-436, 1992. Gibbons R.D., Bock R.D., Hedeker D., Weiss D., Segawa E., Bhaumik D.K., Kupfer D., Frank E., Grochocinski V., Stover A. Full-Information Item Bi-Factor Analysis of Graded Response Data. Applied Psychological Measurement, 31, 4-19, 2007. Gibbons R.D., Weiss D.J., Kupfer D.J., Frank E., Fagiolini A., Grochocinski V.J., Bhaumik D.K., Stover A. Bock R.D., Immekus J.C. Using computerized adaptive testing to reduce the burden of mental health assessment. Psychiatric Services, 59, 361-368, 2008. Gibbons R.D., Weiss D.J., Pilkonis P.A., Frank E., Moore T., Kim J.B., Kupfer D.K. The CAT-DI: A computerized adaptive test for depression. Archives of General Psychiatry, 69, 1104-1112, 2012. Gibbons R.D.,Hooker G., FinkelmanM.D., Weiss D.J., Pilkonis P.A., Frank E., Moore T., Kupfer D.J. The CAD-MDD: A computerized adaptive diagnostic screening tool fordepression. Journal of Clinical Psychiatry, 74, 669-674, 2013. Gibbons R.D., Weiss D.J., Pilkonis, P.A., Frank E., Moore T., Kim J.B., Kupfer D.J. Development of the CAT-ANX: A computerized adaptive test for anxiety. American Journal of Psychiatry, 171, 187-194, 2014. Achtyes E.D., Halstead S., Smart L., Moore T., Frank E., Kupfer D., Gibbons R.D. Validation of computerized adaptive testing in an outpatient non-academic setting. Psychiatric Services, published on-line.

More Related