Validation of Intermediate Measures VIM : RAND Panel, Study Organization and Design

1. Validation of Intermediate Measures (VIM) :RAND Panel, Study Organization and Design Nina R. Schooler, Ph.D. Department of Psychiatry & Behavioral Sciences SUNY Downstate Medical Center Department of Veterans Affairs, VISN 6 Mental Illness, Research, Education and Clinical Center (MIRECC)

3. Step 1:Identification of Evaluation Criteria for Measures Test-retest reliability Correlation with measures of “real-life” functioning Correlation with cognitive performance Practicality - the tester’s perspective Tolerability – the test-taker’s perspective Utility as a repeated measure Sensitivity to change Applicability to international clinical trials

4. Step 2Solicit Nominations of Intermediate Measures Definition of intermediate measures Suitable for use in short-term studies Candidate for cognition co-primary Measure functional capacity or assess cognition Identification Wide solicitation from field Review of literature using Pub-Med and other sources Distinction Performance Interview

5. Step 3 Selection of Candidate MeasuresFunctional Capacity – performance based Everyday Functioning Battery Micro-Module Learning Test Maryland Assessment of Social Competence Social Skills Performance Assessment Test of Adaptive Behavior in Schizophrenia UC-SD Performance Based Skills Assessment UC-SD Performance Based Skills Assessment – Brief Everyday Problems Test Independent Living Skills

6. Step 3Selection of Candidate Measures Interview Based Measures of Cognition Clinical Global Impression of Cognition in Schizophrenia (CGI-CogS Schizophrenia Cognition Rating Scale (SCoRS) Cognitive Assessment Interview (CAI) Cognitive Functioning Scale (CogFu) Global Assessment of Cognition Functioning (GACF) CGI for Cognitive Impairment – clinician version

7. Step 4Creation of Data Base for RAND Panel Evaluation The available literature for each measure is reviewed In addition to published literature, authors and “users” are contacted for unpublished data A data matrix of evidence for each criterion identified in Step 1 and each measure identified in Steps 2 and 3 is generated e.g. test re-test reliability for each measure Problem Inadequate data to fill the matrix for interview measures

9. VIM Rand Panel Members – Feb 2008 Deanna Barch – Washington U. John Brekke - USC Judith Cook – U IL Chicago Patrick Corrigan – IL Inst. Tech Michael Egan - Merck Helena Kraemer – Stanford U. William Lawson – Howard U. Andy Leon – Cornell U Steve Romano - Pfizer Larry Seidman – Harvard U Sophia Vinogradov – UC SF

10. Evaluation Criteria for RAND PANEL Test-retest reliability Correlation with measures of “real-life” functioning Correlation with cognitive performance Practicality - the tester’s perspective Tolerability – the test-taker’s perspective Utility as a repeated measure Sensitivity to change Applicability to international clinical trials

11. RAND Panel Evaluation CriteriaRating Scale 1 Poor 2 3 Fair 4 5 6 Good 7 8 9 Super b

12. RAND PanelPerformance based Measures Results

13. RAND PanelInterview-based MeasuresChallenges for Review Concept of interview –based assessment of cognition is new First publication in the area is Bilder et al (2003) Data regarding the formal evaluation criteria are limited Instead of formal ratings, panelists held a focused review and discussion evaluating the criteria No formal ranks were given Addressed questions about interviewing for cognition Need for informants Global judgment vs. domain-based assessment

14. RAND PanelInterview-based MeasuresRecommendations Cognitive Assessment Interview CAI - domain based Empirically derived from CGI-CogS and SCoRS Patient as only source of information Global Assessment of Cognitive Function - 100 pt scale Novel , experimental measure based on interview Attractive format related to GAF CGI for cognition – 7-point clinician rated measure Can clinicians assess cognition

15. Step 6Selection of Measures for VIM studyPerformance based Measures Test of Adaptive Behavior in Schizophrenia (TABS) UCSD Performance-based Skills Assessment (UPSA) Independent Living Scales (ILS) Shorter versions assessed UPSA Brief Finances Communication TABS Brief Medication management Work & Productivity ILS factors extracted

16. Step 6Selection of Measures for VIM studyInterview Measures Cognitive Assessment Interview (CAI ) Clinical Global Impression (1-7 pt scale) CGI for Cognitive Impairment rated by psychopathology assessor

17. Step 7 VIM Study Specific Aims Examine psychometric properties of the measures Reliability - test-retest and inter rater Repeatability - utility as a repeated measure within the time frame of a clinical trial absence of practice effects that yield ceiling effects) Examine validity of the measures Correlation with measures of “real-life” functioning Correlation with cognitive performance Examine practicality and tolerability of the measures ease of set up, tester training, and scoring, missing data, assessment duration, ratings of subject satisfaction with measures

18. The VIM Study Sites / PIs Project: PI – Green; Co-PI – Schooler Sites and Principal Investigators UCLA / Los Angeles VA Robert Kern, Michael Green Collaborative Neuroscience Network David Walling, Ph.D. Harvard University, Deaconess Beth Israel Hospital Larry Seidman, Ph.D., William Stone, Ph.D. Uptown Research, Chicago John Sonnenberg, Ph.D.

19. VIM Study Design 160 subjects 40 at each of four sites Two assessments Baseline Four weeks Intermediate measures based on RAND panel review MATRICS Consensus Cognitive Battery (MCCB) Quality of Life Scale to assess functioning PANSS to assess psychopathology Three independent interviewer/raters MCCB and intermediate performance measures Interview based intermediate measures PANSS

20. VIM Study Timeline Spring/Summer 2008 Finalize study design Identify sites Prepare study protocol September 2008 Train research staff October 2008 – May 2009 Recruit participants June 2009 End of data collection July – September 2009 Data analysis October 2009 Presentation of results