Global Appraisal of Individual Needs (GAIN): A Standardized Biopsychosocial Assessment Tool

1. Global Appraisal of Individual Needs (GAIN):A Standardized Biopsychosocial Assessment Tool Michael Dennis, Ph.D. Chestnut Health Systems, Normal, IL Workshop at 16th Annual Drug Court Training Conference of the National Association of Drug Court Professionals, Boston, MA, June 3, 2010.. This presentation reports on treatment & research funded by the Center for Substance Abuse Treatment (CSAT), Substance Abuse and Mental Health Services Administration (SAMHSA) under contracts 270-2003-00006 and 270-07-0191, as well as several individual CSAT, NIAAA, NIDA and private foundation grants. The opinions are those of the author and do not reflect official positions of the consortium or government. Available on line at www.chestnut.org/LI/Posters or by contacting Joan Unsicker at 448 Wylie Drive, Normal, IL 61761, phone: (309) 451-7801, Fax: (309) 451-7763, e-mail: junsicker@Chestnut.Org

2. Provide an overview of the role of the GAIN as a piece of infrastructure in support the move toward both evidence based practice and practice based evidence Describe each of the measures, the reports that they use to help the assessment guide clinical decision making and illustrate how they provide a successively more detailed picture of client needs Highlight our current work to using actuarial estimates of outcomes to improve placement decisions Summarize the status of efforts to make the data available for secondary analysis and translate the software, measures and reports from English into Spanish, French, Portuguese and other languages Goals of this Presentation are to

3. Part 1. Provide an overview of the role of the GAIN as a piece of infrastructure in support the move toward both evidence based practice and practice based evidence

4. The Global Appraisal of Individual Needs (GAIN) is .. A family of instruments ranging from screening, to quick assessment to a full Biopsychosocial and monitoring tools Designed to integrate clinical and research assessment Designed to support clinical decision making at the individual client level Designed to support evaluation and planning at program level Designed to support secondary analyses and comparisons across individuals and programs

5. GAIN Collaborators in the U.S (4/10) NH VT WA ME MT MN ND MA OR WI ID SD NY RI MI WY CT PA IA NV NE NJ OH UT IL IN CA DE CO WV MO VA MD KY KS DC NC TN OK NM State or Regional System GAIN-Short Screener GAIN-Quick GAIN-Full No of GAIN Sites AR AZ SC GA AL None (Yet) MS 1 to 14 TX LA 15 to 30 AK 31 to 164 FL HI More in BZ, CA, CN, JP, MX VI PR 3/10 5

6. Some numbers as of April 2010 1,368 Licensed GAIN administrative units from 49 states (all by ND) and 7 countries 2,853 users in 405 Agencies using GAIN ABS 43,968 intake assessments (largest in field) 22,045 (88% w 1+ follow-up) from 219 CSAT grantees (largest follow-up data set in field) 22 states, 12 Federal, 6 provincial, and 3 foundations mandate or strongly encourage its use 4 dozen researchers have published 186 GAIN-related research publications to date Medicaid, Health Canada, several states, and private insurance systems accept it as evidence based 6

7. So what does it mean to move the field towards Evidence Based Practice (EBP)? Introducing explicit intervention protocols that are Targeted at specific problems/subgroups and outcomes Having explicit quality assurance procedures to cause adherence at the individual level and implementation at the program level Introducing reliable and valid assessment that can be used At the individual level to immediately guide clinical judgments about diagnosis/severity, placement, treatment planning, and the response to treatment At the program level to drive program evaluation, needs assessment, performance monitoring and long term program planning Having the ability to evaluate client and program outcomes For the same person or program over time, Relative to other people or interventions

8. Key Issues that we try to address with the GAIN Instruments and Coordinating Center High turnover workforce with variable educationbackground related to diagnosis, placement, treatment planning and referral to other services Heterogeneous needs and severitycharacterized by multiple problems, chronic relapse, and multiple episodes of care over several years Lack of access to or use of data at the program levelto guide immediate clinical decisions, billing and program planning Missing, bad or misrepresented datathat needs to be minimized and incorporated into interpretations Lack of Infrastructure that is needed to support implementation and fidelity

9. 1. High Turnover Workforce with Variable Education Questions spelled out and simple question format Lay wording mapped onto expert standards for given area Built in definitions, transition statements, prompts, and checks for inconsistent and missing information. Standardized approach to asking questions across domains Range checks and skip logic built into electronic applications Formal training and certification protocols on administration, clinical interpretation, data management, coordination, local, regional, and national “trainers” Above focuses on consistency across populations, level of care, staff and time On-going quality assurance and data monitoring for the reoccurrence or problems at the staff (site or item) level Availability of training resources, responses to frequently asked questions, and technical assistance Outcome: Improved Reliability and Efficiency

10. 2. Heterogeneous Needs and Severity Multiple domains Focus on most common problems Participant self description of characteristics, problems, needs, personal strengths and resources Behavior problem recency, breadth , and frequency Utilization lifetime, recency and frequency Dimensional measures to measure change with interpretative cut points to facilitate decisions Items and cut points mapped onto DSM for diagnosis, ASAM for placement, and to multiple standards and evidence- based practices for treatment planning Computer generated scoring and reports to guide decisions Treatment planning recommendations and links to evidence-based practice Basic and advanced clinical interpretation training and certification Outcome: Comprehensive Assessment

11. 3. Lack of Access to or use of Data at the Program Level Data immediately available to support clinical decision making for a case Data can be transferred to other clinical information system to support billing, progress reports, treatment planning and on-going monitoring Data can be exported and cleaned to support further analyses Data can be pooled with other sites to facilitate comparison and evaluation PC and web based software applications and support Formal training and certification on using data at the individual level and data management at the program level Data routinely pooled to support comparisons across programs and secondary analysis Over three dozen scientists already working with data to link to evidence-based practice Outcome: Improved Program Planning and Outcomes

12. 4. Missing, Bad or Misrepresented Data Assurances, time anchoring, definitions, transition, and question order to reduce confusion and increase valid responses Cognitive impairment check Validity checks on missing, bad, inconsistency and unlikely responses Validity checks for atypical and overly random symptom presentations Validity ratings by staff Training on optimizing clinical rapport Training on time anchoring Training answering questions, resolving vague or inconsistent responses, following assessment protocol and accurate documentation. Utilization and documentation of other sources of information Post hoc checks for on-going site, staff or item problems Outcome: Improved Validity

13. 5. Lack of Infrastructure Direct Services Training and quality assurance on administration, clinical interpretation, data management, follow-up and project coordination Data management Evaluation and data available for secondary analysis Software support Technical assistance and back up to local trainer/expert Development Clinical Product Development Software Development Collaboration with IT vendors (e.g., WITS) Over 36 internal & external scientists and students Workgroups focused on specific subgroup, problem, or treatment approach Labor supply (e.g., consultant pool, college courses) Outcome: Implementation with Fidelity

14. Across measures, the GAIN has a Common Factor Structure of Psychopathology Source: Dennis, Chan, and Funk (2006) CFI=.92, RMSEA=.06 allowing for age

15. Alcohol and Other Drug Abuse, Dependence and Problem Use are Age Related Over 90% of use and problems start between the ages of 12-20 It takes decades before most recover or die 100 People with drug dependence die an average of 22.5 years sooner than those without a diagnosis 90 Percentage 80 70 60 Severity Category 50 Other drug or heavy alcohol use in the past year 40 30 Alcohol or Drug Use (AOD) Abuse or Dependence in the past year 20 10 0 65+ 12-13 14-15 16-17 18-20 21-29 30-34 35-49 50-64 Age Source: 2002 NSDUH and Dennis & Scott, 2007, Neumark et al., 2000

16. Co-occurring Mental Health Problems are Common, but the Type of Problems also Changes with Age Internalizing Disorders go up with age Externalizing Disorders go down with age (but do NOT go away) Source: Chan, YF; Dennis, M L.; Funk, RR. (2008). Prevalence and comorbidity of major internalizing and externalizing problems among adolescents and adults presenting to substance abuse treatment. Journal of Substance Abuse Treatment, 34(1) 14-24 .

17. Progressive Continuum of Measurement (Common Measures) Screening to Identify Who Needs to be “Assessed” (5-10 min) Focus on brevity, simplicity for administration & scoring Needs to be adequate for triage and referral GAIN Short Screener for SUD, MH & Crime ASSIST, AUDIT, CAGE, CRAFT, DAST, MAST for SUD SCL, HSCL, BSI, CANS for Mental Health LSI, MAYSI, YLS for Crime Quick Assessment for Targeted Referral (20-30 min) Assessment of who needs a feedback, brief intervention or referral for more specialized assessment or treatment Needs to be adequate for brief intervention GAIN Quick ADI, ASI, SASSI, T-ASI, MINI Comprehensive Biopsychosocial (1-2 hours) Used to identify common problems and how they are interrelated Needs to be adequate for diagnosis, treatment planning and placement of common problems GAIN Initial (Clinical Core and Full) CASI, A-CASI, MATE Specialized Assessment (additional time per area) Additional assessment by a specialist (e.g., psychiatrist, MD, nurse, spec ed) may be needed to rule out a diagnosis or develop a treatment plan or individual education plan CIDI, DISC, KSADS, PDI, SCAN More Extensive / Longer/ Expensive Screener Quick Comprehensive Special

18. Longer assessments identify more areas of unmet need Most substance users have multiple problems Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192) 18

19. Part 2. Describe each of the measures, the reports that they use to help the assessment guide clinical decision making and illustrate how they provide a successively more detailed picture of client needs

20. Next slides will • Describe the difference in the breadth of information you get with different levels of assessment • Summarize validation studies to date • Illustrate the difference using data from a single sample (Reclaiming futures project) • Demonstrate that multi-morbidity is the norm and varies by type of client and program

21. GAIN-Short Screener (GSS) • Administration Time: A 3- to 5-minute screener • Purpose: Used in general populations to • identify or rule-out clients who will be identified as having any behavioral health disorders on the 60-120 min versions of the GAIN • triage area of problem • serve as a simple measure of change • Easy for administration and interpretation by staff with minimal training or direct supervision • Mode: Designed for self- or staff-administration, with paper and pen, computer, or on the web • Scales: Four screeners for Internalizing Disorders, Externalizing Disorders, Substance Disorders, Crime/Violence, and a Total

22. GAIN-Short Screener (GSS) (continued) • Response Set: Recency of 20 problems rated past month (3), 2-12 months ago (2), more than a year ago (1), never (0) • Interpretation: Combined by cumulative time period as: • Past month count (3s) to measure of change • Past year count (2s or 3s) to predict diagnosis • Lifetime count (1s, 2s or 3s) as a measure of peak severity • Can be classified within time period low (0), moderate (1-2) or high (3) • Can also be used to classify remission as • Early (lifetime but not past month) • Sustained (lifetime but not past year) • Reports: Narrative, tabular, and graphical reports built into web based GAIN ABS and/or ASP application for local hosting

23. GAIN-Short Screener (GSS)

24. GAIN SS Psychometric Properties Low Mod. High 100% Prevalence (% 1+ disorder) 90% Sensitivity (% w disorder above) 80% Specificity (% w/o disorder below) 70% (n=6194 adolescents) 60% Using a higher cut point increases prevalence and specificity, but decreases sensitivity 50% 40% At 3 or more symptoms we get 99% prevalence, 91% sensitivity, & 89% specificity 30% 20% 10% 0% 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Total Disorder Screener (TDScr) Total score has alpha of .85 and is correlated .94 with full GAIN version Source: Dennis et al 2006

25. Moderate (1+) gives best result for sensitivity High (3+) gives best result for specificity GSS Performance by Subscale and Disorders Prevalence Sensitivity Specificity Screener/Disorder 1+ 3+ 1+ 3+ 1+ 3+ Internal Disorder Screener (0-5) Any Internal Disorder 81% 99% 94% 55% 71% 99% Major Depression 56% 87% 98% 72% 54% 94% Generalized Anxiety 32% 56% 100% 83% 44% 83% Suicide Ideation 24% 43% 100% 84% 41% 79% Mod/High Traumatic Stress 60% 82% 94% 60% 55% 90% External Disorder Screener (0-5) Any External Disorder 88% 97% 98% 67% 75% 96% AD, HD or Both 65% 82% 99% 78% 51% 85% Conduct Disorder 78% 91% 98% 70% 62% 90% Substance Use Disorder Screener (0-5) Any Substance Disorder 96% 100% 96% 68% 73% 100% Dependence 65% 87% 100% 91% 30% 82% Abuse 30% 13% 89% 25% 14% 28% Recommend Triage as 0=Not likely 1-2 Possible 3+=Likely Crime Violence Screener (0-5) Any Crime/Violence 88% 99% 94% 49% 76% 99% High Physical Conflict 31% 46% 100% 70% 38% 77% Mod/High General Crime 85% 100% 94% 51% 71% 100% Total Disorder Screener (0-5) Any Disorder 97% 99% 99% 91% 47% 89% Any Internal Disorder 58% 63% 100% 98% 8% 28% Any External Disorder 68% 75% 100% 99% 10% 37% Any Substance Disorder 89% 92% 99% 92% 20% 51% Any Crime/Violence 68% 73% 100% 96% 10% 32%

26. GAIN SS Total Score is Correlated With Level Of Care Placement: Adolescents

27. GAIN SS Total Score is Correlated With Level Of Care Placement: Adults

28. Track Gap Between Prior and current Lifetime Problems to identify “under reporting” Track progress in reducing current (past month) symptoms) GAIN SS Can Also be Used for Monitoring 20 12+ Mon.s ago (#1s) 2-12 Mon.s ago (#2s) 16 Past Month (#3s) Lifetime (#1,2,or 3) 11 12 10 10 9 9 8 8 3 4 2 2 0 Intake 3 6 9 12 15 18 21 24 Mon Mon Mon Mon Mon Mon Mon Mon Total Disorder Screener (TDScr) Monitor for Relapse

29. GAIN Short Screener Profile: Reclaiming Futures (Range based on 0/1-2/3+ Symptoms) Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)

30. GAIN Short Screener Number of Problems Mod/Hi in Reclaiming Futures 93% endorsed one or more problems (40% 4 or more) Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)

31. Construct Validity of GSS Internalizing Disorder Screener Source: Education Service District 113 (n=979) and King County (n=1002)

32. Construct Validity of GSS Externalizing Disorder Screener Source: Education Service District 113 (n=979) and King County (n=1002)

33. Construct Validity of GSS Substance Disorder Screener Source: Education Service District 113 (n=979) and King County (n=1002)

34. Construct Validity of GSS Crime/Violence Screener Source: Education Service District 113 (n=979) and King County (n=1002)

35. Adolescent Rates of High (2+) Scores on Mental Health (MH) or Substance Abuse (SA) Screener by Setting in Washington State Problems could be easily identified Comorbidity is common Source: Lucenko et al (2009). Report to the Legislature: Co-Occurring Disorders Among DSHS Clients. Olympia, WA: Department of Social and Health Services. Retrieved from http://publications.rda.dshs.wa.gov/1392/

36. Adult Rates of High (2+) Scores on Mental Health (MH) or Substance Abuse (SA) Screener by Setting in Washington State Lower than expected rates of SA in Mental Health & Children’s Admin Source: Lucenko et al (2009). Report to the Legislature: Co-Occurring Disorders Among DSHS Clients. Olympia, WA: Department of Social and Health Services. Retrieved from http://publications.rda.dshs.wa.gov/1392/

37. Adolescent Client Validation of Hi Co-occurring from GAIN Short Screener vs Clinical Records by Setting in Washington State Two page measure closely approximated all found in the clinical record after the next two years Source: Lucenko et al (2009). Report to the Legislature: Co-Occurring Disorders Among DSHS Clients. Olympia, WA: Department of Social and Health Services. Retrieved from http://publications.rda.dshs.wa.gov/1392/

38. Higher rate in clinical record in Mental Health and Children’s Administration (But that was past on “any use” vs. “abuse/dependence” and 2 years vs. past year Adult Client Validation of Hi Co-occurring from GAIN Short Screener vs Clinical Records by Setting in Washington State Source: Lucenko et al (2009). Report to the Legislature: Co-Occurring Disorders Among DSHS Clients. Olympia, WA: Department of Social and Health Services. Retrieved from http://publications.rda.dshs.wa.gov/1392/

39. Other Validations Confirmatory Factor Analysis • Dennis, Chan & Funk (2006) found that the 20 item GSS and its four subscales were highly correlated (.84 to .94) with the full scale, had 90% sensitivity and over 90% area under the curve relative to the full GAIN; Confirmatory factors analysis also found it to be consistent with the overall model of psychopathology after allowing for age (CFI=.92; RMSEA=.06). Substance Disorders: • McDonnell and colleagues (2009) found that the 5-item GAIN SS Substance Disorder Screener had 92% sensitivity and 85% correct classification relative to the Diagnostic Inventory Scale for Children (DISC) Predictive Scales (DPS; Lucas et al 2001) and 88% sensitivity and 88% correct classification relative to the CRAFFT (Knight et al 2001) Internalizing Disorders: • McDonnell and colleagues (2009) found that the 5-item GAIN SS Internalizing Disorder Screener had 100% sensitivity and 75% correct classification relative to the Youth Self Report (YSR; Achenbach et al, 2001) and that the 5-item GAIN SS Externalizing Disorder Screener had 89% sensitivity and 65% correct classification to the YSR. • Riley and colleagues (2009) found that the 5-item GAIN SS’s Internalizing Disorder Screener had 92% sensitivity and 80% area under the curve relative to the Structured Clinical Interview for DSM (SCID) and was more efficient relative to 11 item Addiction Severity Index (ASI) psychiatric composite score (McLellan et al., 1992), 10 item K10 (Kessler et al., 2002) and the 87 item Psychiatric Diagnostic Screening Questionnaire (PDSQ; Zimmerman and Mattia, 2001)

40. GAIN Quick (GQ): Version 2 • Strengths:Length (20-30min) in desired range, range of topics, efficiently categorizes, narrative reports to support screening, brief intervention, and referral to treatment • Problems: • Lacks scales or recency to support analyses or outcomes related to “change over time” • Item response choices do not provide information about lifetime problems important when someone has been incarcerated for more than a few weeks • Current Personal Feedback Report focuses only on substance use and does not address the other content areas of the GAIN-Q • Only about 60% of the items can be directly imported into the GAIN-I

41. GAIN Quick (GQ): Changes from Version 2 to 3: • Kept focus on screening, brief intervention and referral to treatment • Broke out sections for Crime/Violence, HIV risk, Work and School problems • Subsumed GSS and added similar screeners in other GAIN Q areas with recency response to address change and lifetime issues • Change measures for each symptom count and days items • Created reasons for change items in each area to support brief intervention, reducing number of items in substance use • Make all questions importable into full GAIN • Expand narrative report to have more treatment planning statements and to allow motivational interviewing within each area

42. GAIN-Quick (GQ) Version 3 • Administration Time: about 25 minutes for core (varies depending on severity) and on average 25-45 minutes using full with motivational interview questions (depending on number of problem areas probed). • Training Requirements: 1 day (train the trainer) training plus certification within 1-2 months for administration certification, and 2 days of motivational intervention training plus 1-3 months for clinical certification. • Mode: Generally staff-administered on computer (can be done on paper or self-administered with proctor). • Purpose: Designed for use in targeted populations for more detailed screening, to support brief intervention or referral for further assessment or behavioral intervention.

43. GAIN-Quick (GQ) • Scales: • GAIN SS scales + similar scales for school, work, physical health, psychosocial stress, and HIV risks • Additional “days” items and scale for measuring behavioral change • Recency and past 90 day measures of service utilization in each area to aid in placement, track implementation and estimate quarterly costs to society • Reasons for change to support motivational interviewing in each area • Life Satisfaction Scale and interview quality documentation • Response Set: Recency (“the last time” scale), Breadth (past year symptom counts for behavior and lifetime for utilization) and Prevalence (past 90 days)

44. GAIN-Quick (GQ) Problem Profile(Range based on 0/1-2/3+ Symptoms) Source: GAIN-Q Pilot (n=138)

45. Number of Problems 99% endorsed one or more problems (94% 4 or more problems) Source: GAIN-Q Pilot (n=138)

46. Distribution of Summary Indices Source: GAIN Q Pilot . (n=138)

47. Problem Prevalence Index • Percent of days of problems in each of 8 areas summed, divided by range (23 items), and multiplied by 100 to get a score from 0 to 100. • Problem Prevalence Index can be interpreted continuously where up is bad and subjectively unpleasant. • It can also be triaged to low (0-5), moderate (6-24) or high (100) based on roughly 50%, 40% and 10% of the clinical population. • Mean score here was 12.7 (10.2 Std. Dev).

48. Quarterly Costs to Society • The frequency of using tangible services in the 90 days before intake (e.g., health care utilization, days in detention, probation, parole, days of missed school) in each of the 8 areas valued in 2009 dollars, and summed. • Quarterly costs to society can be used continuously with up as more expensive to society. • It can also be triaged as low ($0 to $1999), moderate ($2000 to $9999) and high ($10000 or more) based on average costs of outpatient and residential treatment respectively. • The 138 clients here cost society an average of $6,118 (SD=$12,382) per person in the quarter before intake and $24,471 (SD=$49,551) in the year before intake. Source: GAIN-Q Pilot (n=138)

49. Quality of Life Year (QOLY) • Triage of past year problem count in each of the 9 screeners weighted as No or Low=2, Moderate=1 and High=0; summed, divided by range 18, and multiple by 100 to get a quality of life score from 0 to 100. • Quality of Life Year measure can be interpreted continuously where up is good and subjectively pleasant. • It can also be triaged to low (0-36), moderate (37-69) or high (70 to 100) based on roughly 50%, 40% and 10% of the clinical population. • Mean score here was 39.7 (18.8 Std. Dev).

50. Life Satisfaction Index • Likert rating from strongly disagree (1) to strongly agree (5) in 5 areas (sexual relationship, living situation, family relationships, school/work situation, how life is going so far, general level of happiness) is summed and ranges from 6 to 30 • Life Satisfaction Index can be interpreted continuously where up is good • It can also be triaged to low (6-15), moderate (16-27) or high (28-30) on roughly 50%, 40% and 10% of the clinical population • Mean score here was 19.3 (5.7 Std. Dev) and was related to other summary indices