1 / 71

Global Appraisal of Individual Needs (GAIN) Logic Model and its Short Screener

Global Appraisal of Individual Needs (GAIN) Logic Model and its Short Screener. Michael Dennis, Ph.D. Chestnut Health Systems, Normal, IL December 3, 2009 Presentation at the Electronic Health Record Content Standards for Behavioral Health

skyla
Download Presentation

Global Appraisal of Individual Needs (GAIN) Logic Model and its Short Screener

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Global Appraisal of Individual Needs (GAIN) Logic Model and its Short Screener Michael Dennis, Ph.D. Chestnut Health Systems, Normal, IL December 3, 2009 Presentation at the Electronic Health Record Content Standards for Behavioral Health Expert Panel and Stakeholders Meeting, Rockville, MD, December 3-4, 2009. The presentation reports on treatment & research funded by the Center for Substance Abuse Treatment (CSAT), Substance Abuse and Mental Health Services Administration (SAMHSA, under contracts 270-2003-00006 and 270-07-0191), the state of Washington, King county, PSESD, and ESD113.. The opinions are those of the author and do not reflect official positions of the consortium or government. Available on line at www.chestnut.org/LI/Posters or by contacting Michael Dennis, Chestnut Health Systems, 448 Wylie Drive, Normal, IL 61761, phone 309-451-7801, fax 309-451-7765, e-Mail: mdennis@Chestnut.Org Questions about the GAIN can also be sent to gaininfo@chestnut.org

  2. Part 1. Overview of the Global Appraisal of Individual Needs (GAIN) collaboration and logic model

  3. The Global Appraisal of Individual Needs (GAIN) is .. A family of instruments ranging from screening, to quick assessment to a full Biopsychosocial and monitoring tools Designed to integrate clinical and research assessment Designed to support clinical decision making at the individual client level Designed to support evaluation and planning at program level and secondary analysis A key piece of infrastructure in the move towards evidenced based practice and a key source of practice based evidence

  4. As of June 30, 2009, there were 1127 administrative units (agencies, grantees, counties, states) collaborating to use the GAIN in the U.S., State or County System GAIN-Short Screener GAIN-Quick GAIN-Full

  5. Canada and other countries 1-10 Sites in Other Countries: Brazil China Mexico Japan

  6. So what does it mean to move the field towards Evidence Based Practice (EBP)? Introducing explicit intervention protocols that are Targeted at specific problems/subgroups and outcomes Having explicit quality assurance procedures to cause adherence at the individual level and implementation at the program level Introducing reliable and valid assessment that can be used At the individual level to immediately guide clinical judgments about diagnosis/severity, placement, treatment planning, and the response to treatment At the program level to drive program evaluation, needs assessment, performance monitoring and long term program planning Having the ability to evaluate client and program outcomes For the same person or program over time, Relative to other people or interventions

  7. Key Issues that we try to address with the GAIN Instruments and Coordinating Center High turnover workforce with variable educationbackground related to diagnosis, placement, treatment planning and referral to other services Heterogeneous needs and severitycharacterized by multiple problems, chronic relapse, and multiple episodes of care over several years Lack of access to or use of data at the program levelto guide immediate clinical decisions, billing and program planning Missing, bad or misrepresented datathat needs to be minimized and incorporated into interpretations Lack of Infrastructure that is needed to support implementation and fidelity

  8. 1. High Turnover Workforce with Variable Education Questions spelled out and simple question format Lay wording mapped onto expert standards for given area Built in definitions, transition statements, prompts, and checks for inconsistent and missing information. Standardized approach to asking questions across domains Range checks and skip logic built into electronic applications Formal training and certification protocols on administration, clinical interpretation, data management, coordination, local, regional, and national “trainers” Above focuses on consistency across populations, level of care, staff and time On-going quality assurance and data monitoring for the reoccurrence or problems at the staff (site or item) level Availability of training resources, responses to frequently asked questions, and technical assistance Outcome: Improved Reliability and Efficiency

  9. 2. Heterogeneous Needs and Severity Multiple domains Focus on most common problems Participant self description of characteristics, problems, needs, personal strengths and resources Behavior problem recency, breadth , and frequency Utilization lifetime, recency and frequency Dimensional measures to measure change with interpretative cut points to facilitate decisions Items and cut points mapped onto DSM for diagnosis, ASAM for placement, and to multiple standards and evidence- based practices for treatment planning Computer generated scoring and reports to guide decisions Treatment planning recommendations and links to evidence-based practice Basic and advanced clinical interpretation training and certification Outcome: Comprehensive Assessment

  10. 3. Lack of Access to or use of Data at the Program Level Data immediately available to support clinical decision making for a case Data can be transferred to other clinical information system to support billing, progress reports, treatment planning and on-going monitoring Data can be exported and cleaned to support further analyses Data can be pooled with other sites to facilitate comparison and evaluation PC and web based software applications and support Formal training and certification on using data at the individual level and data management at the program level Data routinely pooled to support comparisons across programs and secondary analysis Over three dozen scientists already working with data to link to evidence-based practice Outcome: Improved Program Planning and Outcomes

  11. Progressive Continuum of Measurement (Common Measures) Screening to Identify Who Needs to be “Assessed” (5-10 min) Focus on brevity, simplicity for administration & scoring Needs to be adequate for triage and referral GAIN Short Screener for SUD, MH & Crime ASSIST, AUDIT, CAGE, CRAFT, DAST, MAST for SUD SCL, HSCL, BSI, CANS for Mental Health LSI, MAYSI, YLS for Crime Quick Assessment for Targeted Referral (20-30 min) Assessment of who needs a feedback, brief intervention or referral for more specialized assessment or treatment Needs to be adequate for brief intervention GAIN Quick ADI, ASI, SASSI, T-ASI, MINI Comprehensive Biopsychosocial (1-2 hours) Used to identify common problems and how they are interrelated Needs to be adequate for diagnosis, treatment planning and placement of common problems GAIN Initial (Clinical Core and Full) CASI, A-CASI, MATE Specialized Assessment (additional time per area) Additional assessment by a specialist (e.g., psychiatrist, MD, nurse, spec ed) may be needed to rule out a diagnosis or develop a treatment plan or individual education plan CIDI, DISC, KSADS, PDI, SCAN More Extensive / Longer/ Expensive Screener Quick Comprehensive Special

  12. Part 2. Overview of the GAIN Short Screener and a Summary of Major Validation Studies

  13. The Movement to Increase Screening Screening, Brief Intervention and Referral to Treatment (SBIRT) has been shown to be effective in identifying people not currently in treatment, initiating treatment/change and improving outcomes (see http://sbirt.samhsa.gov/ ) The US Preventive Services Task Force (USPSTF, 2004; 2007), National Quality Forum (NQF, 2007), and Healthy People 2010 have each recommended regular screening, brief intervention, and referral to treatment (SBIRT) for tobacco and alcohol abuse in general medical settings for everyone SBIRT for drug use in high risk populations (e.g., adolescents, pregnant and post partum women, people with HIV, and people with co-occurring psychiatric conditions) CSAT and NIDA are both funding several demonstration and research projects to develop and evaluate models for doing this Washington State mandated screening in all adult and adolescent substance abuse treatment, mental health, justice, child welfare and student assistant programs 13

  14. GAIN-Short Screener (GSS) • Administration Time: A 3- to 5-minute screener • Purpose: Used in general populations to • identify or rule-out clients who will be identified as having any behavioral health disorders on the 60-120 min versions of the GAIN • triage area of problem • serve as a simple measure of change • Easy for administration and interpretation by staff with minimal training or direct supervision • Mode: Designed for self- or staff-administration, with paper and pen, computer, or on the web • Scales: Four screeners for Internalizing Disorders, Externalizing Disorders, Substance Disorders, Crime/Violence, and a Total

  15. GAIN-Short Screener (GSS) (continued) • Response Set: Recency of 20 problems rated past month (3), 2-12 months ago (2), more than a year ago (1), never (0) • Interpretation: Combined by cumulative time period as: • Past month count (3s) to measure of change • Past year count (2s or 3s) to predict diagnosis • Lifetime count (1s, 2s or 3s) as a measure of peak severity • Can be classified within time period low (0), moderate (1-2) or high (3) • Can also be used to classify remission as • Early (lifetime but not past month) • Sustained (lifetime but not past year) • Reports: Narrative, tabular, and graphical reports built into web based GAIN ABS and/or ASP application for local hosting

  16. GAIN-Short Screener (GSS)

  17. Expected Factor Structure of Psychopathology and Psychopathy Source: Dennis, Chan, and Funk (2006)

  18. Co-occurring Mental Health Problems are Common, but the Type of Problems also Changes with Age Internalizing Disorders go up with age Externalizing Disorders go down with age (but do NOT go away) Source: Chan, YF; Dennis, M L.; Funk, RR. (2008). Prevalence and comorbidity of major internalizing and externalizing problems among adolescents and adults presenting to substance abuse treatment. Journal of Substance Abuse Treatment, 34(1) 14-24 .

  19. Knowing both is a better predictor (high –high group is 5.5 times more likely than low low) Intake Crime/ Violence Severity Predicts Recidivism Intake Substance Problem Severity Predicts Recidivism Any Illegal Activity in the Next Six Months by Intake Severity on Crime/Violence and Substance Problem Scales While there is risk, most (42-80%) actually do not commit additional crime Source: CSAT 2008 V5 dataset Adolescents aged 12-17 with 3 and/or 6 month follow-up (N=9006)

  20. GAIN SS Psychometric Properties Low Mod. High 100% Prevalence (% 1+ disorder) 90% Sensitivity (% w disorder above) 80% Specificity (% w/o disorder below) 70% (n=6194 adolescents) 60% Using a higher cut point increases prevalence and specificity, but decreases sensitivity 50% 40% At 3 or more symptoms we get 99% prevalence, 91% sensitivity, & 89% specificity 30% 20% 10% 0% 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Total Disorder Screener (TDScr) Total score has alpha of .85 and is correlated .94 with full GAIN version Source: Dennis et al 2006

  21. Moderate (1+) gives best result for sensitivity High (3+) gives best result for specificity GSS Performance by Subscale and Disorders Prevalence Sensitivity Specificity Screener/Disorder 1+ 3+ 1+ 3+ 1+ 3+ Internal Disorder Screener (0-5) Any Internal Disorder 81% 99% 94% 55% 71% 99% Major Depression 56% 87% 98% 72% 54% 94% Generalized Anxiety 32% 56% 100% 83% 44% 83% Suicide Ideation 24% 43% 100% 84% 41% 79% Mod/High Traumatic Stress 60% 82% 94% 60% 55% 90% External Disorder Screener (0-5) Any External Disorder 88% 97% 98% 67% 75% 96% AD, HD or Both 65% 82% 99% 78% 51% 85% Conduct Disorder 78% 91% 98% 70% 62% 90% Substance Use Disorder Screener (0-5) Any Substance Disorder 96% 100% 96% 68% 73% 100% Dependence 65% 87% 100% 91% 30% 82% Abuse 30% 13% 89% 25% 14% 28% Recommend Triage as 0=Not likely 1-2 Possible 3+=Likely Crime Violence Screener (0-5) Any Crime/Violence 88% 99% 94% 49% 76% 99% High Physical Conflict 31% 46% 100% 70% 38% 77% Mod/High General Crime 85% 100% 94% 51% 71% 100% Total Disorder Screener (0-5) Any Disorder 97% 99% 99% 91% 47% 89% Any Internal Disorder 58% 63% 100% 98% 8% 28% Any External Disorder 68% 75% 100% 99% 10% 37% Any Substance Disorder 89% 92% 99% 92% 20% 51% Any Crime/Violence 68% 73% 100% 96% 10% 32%

  22. Track Gap Between Prior and current Lifetime Problems to identify “under reporting” Track progress in reducing current (past month) symptoms) GAIN SS Can Also be Used for Monitoring 20 12+ Mon.s ago (#1s) 2-12 Mon.s ago (#2s) 16 Past Month (#3s) Lifetime (#1,2,or 3) 11 12 10 10 9 9 8 8 3 4 2 2 0 Intake 3 6 9 12 15 18 21 24 Mon Mon Mon Mon Mon Mon Mon Mon Total Disorder Screener (TDScr) Monitor for Relapse

  23. Status of Translations

  24. Construct Validity of GSS Internalizing Disorder Screener Source: Dennis 2009, Education Service District 113 (n=979) and King County (n=1002)

  25. Construct Validity of GSS Externalizing Disorder Screener Source: Dennis 2009, Education Service District 113 (n=979) and King County (n=1002)

  26. Construct Validity of GSS Substance Disorder Screener Source: Dennis 2009, Education Service District 113 (n=979) and King County (n=1002)

  27. Construct Validity of GSS Crime/Violence Screener Source: Dennis 2009, Education Service District 113 (n=979) and King County (n=1002)

  28. Adolescent Rates of High (2+) Scores on Mental Health (MH) or Substance Abuse (SA) Screener by Setting in Washington State Problems could be easily identified Comorbidity is common Source: Lucenko et al (2009). Report to the Legislature: Co-Occurring Disorders Among DSHS Clients. Olympia, WA: Department of Social and Health Services. Retrieved from http://publications.rda.dshs.wa.gov/1392/

  29. Where in the System are the Adolescents with Mental Health, Substance Abuse and Co-occurring? There are more kids with mental health issues than substance use Source: Lucenko et al (2009). Report to the Legislature: Co-Occurring Disorders Among DSHS Clients. Olympia, WA: Department of Social and Health Services. Retrieved from http://publications.rda.dshs.wa.gov/1392/

  30. Where in the System are the Adolescents with Mental Health, Substance Abuse and Co-occurring? 2/3rd of the teens with mental health issues are seen in substance abuse treatment or student assistance programs <1% <1% <1% student assistance programs Represent 1/3rd of the behavioral health system Source: Lucenko et al (2009). Report to the Legislature: Co-Occurring Disorders Among DSHS Clients. Olympia, WA: Department of Social and Health Services. Retrieved from http://publications.rda.dshs.wa.gov/1392/

  31. Adolescent Client Validation of Hi Co-occurring from GAIN Short Screener vs Clinical Records by Setting in Washington State Two page measure closely approximated all found in the clinical record after the next two years Source: Lucenko et al (2009). Report to the Legislature: Co-Occurring Disorders Among DSHS Clients. Olympia, WA: Department of Social and Health Services. Retrieved from http://publications.rda.dshs.wa.gov/1392/

  32. Adult Rates of High (2+) Scores on Mental Health (MH) or Substance Abuse (SA) Screener by Setting in Washington State Lower than expected rates of SA in Mental Health & Children’s Admin Source: Lucenko et al (2009). Report to the Legislature: Co-Occurring Disorders Among DSHS Clients. Olympia, WA: Department of Social and Health Services. Retrieved from http://publications.rda.dshs.wa.gov/1392/

  33. Where in the System are the Adults with Mental Health, Substance Abuse and Co-occurring? More Mental Health than Substance Abuse Source: Lucenko et al (2009). Report to the Legislature: Co-Occurring Disorders Among DSHS Clients. Olympia, WA: Department of Social and Health Services. Retrieved from http://publications.rda.dshs.wa.gov/1392/

  34. Where in the System are the Adults with Mental Health, Substance Abuse and Co-occurring? More Mental Health Treated in Substance Abuse Treatment Source: Lucenko et al (2009). Report to the Legislature: Co-Occurring Disorders Among DSHS Clients. Olympia, WA: Department of Social and Health Services. Retrieved from http://publications.rda.dshs.wa.gov/1392/

  35. Higher rate in clinical record in Mental Health and Children’s Administration. But that was based on • “any use” vs. “week use + abuse/dependence” • - and 2 years vs. past year Adult Client Validation of Hi Co-occurring from GAIN Short Screener vs Clinical Records by Setting in Washington State Source: Lucenko et al (2009). Report to the Legislature: Co-Occurring Disorders Among DSHS Clients. Olympia, WA: Department of Social and Health Services. Retrieved from http://publications.rda.dshs.wa.gov/1392/

  36. Other Validations Confirmatory Factor Analysis • Dennis, Chan & Funk (2006) found that the 20 item GSS and its four subscales were highly correlated (.84 to .94) with the full scale, had 90% sensitivity and over 90% area under the curve relative to the full GAIN; Confirmatory factors analysis also found it to be consistent with the overall model of psychopathology after allowing for age (CFI=.92; RMSEA=.06). Substance Disorders: • McDonnell and colleagues (2009) found that the 5-item GAIN SS Substance Disorder Screener had 92% sensitivity and 85% correct classification relative to the Diagnostic Inventory Scale for Children (DISC) Predictive Scales (DPS; Lucas et al 2001) and 88% sensitivity and 88% correct classification relative to the CRAFFT (Knight et al 2001) Internalizing Disorders: • McDonnell and colleagues (2009) found that the 5-item GAIN SS Internalizing Disorder Screener had 100% sensitivity and 75% correct classification relative to the Youth Self Report (YSR; Achenbach et al, 2001) and that the 5-item GAIN SS Externalizing Disorder Screener had 89% sensitivity and 65% correct classification to the YSR. • Riley and colleagues (2009) found that the 5-item GAIN SS’s Internalizing Disorder Screener had 92% sensitivity and 80% area under the curve relative to the Structured Clinical Interview for DSM (SCID) and was more efficient relative to 11 item Addiction Severity Index (ASI) psychiatric composite score (McLellan et al., 1992), 10 item K10 (Kessler et al., 2002) and the 87 item Psychiatric Diagnostic Screening Questionnaire (PDSQ; Zimmerman and Mattia, 2001)

  37. Total Disorder Screener Severity by Level of Care: Adolescents Outpatient Median=6.0 (30% at 10+) Residential Median= 10.5 (59% at 10+) Few missed (1/2-3%) Source: SAPISP 2009 Data and Dennis et al 2006 37

  38. Total Disorder Screener Severity by Level of Care: Adults Outpatient Median=4.5 (29% at 10+) Residential Median= 8.5 (59% at 10+) 10% of adult OP missed) Source: SAPISP 2009 Data and Dennis et al 2006 38

  39. Part 3. Detailed Results from the Student Assistance Prevention and Intervention Services Program (SAPISP) in Washington State

  40. Student Assistance Prevention and Intervention Services Program (SAPISP) • Core funding is funneled from DASA via OSPI and combined with a variety of other local, state, and federal funding sources (eg, DFSCA, SSHS, SPF-SIG). • 13 grantees (the 9 ESDs and 4 largest school districts) hire specialists to serve about 75% of MS and HS statewide. • Specialists conduct some primary prevention activities and serve about 16,000 students specifically referred for assistance related to mental health, alcohol or drug use, tobacco use or other behavioral problems • Screening using the GAIN-SS was first implemented in the 2007-2008 school year. • Reporting is optional for “Quick” referrals that are seen only once or twice. • Data Presented here are for the 2008 to 2009 school year

  41. SAPISP Results: State Wide (n=10,924) GAIN SS uses triage: 0=Low 1-2=Mod 3+=High WA State dichotomizes as 0-1=Low 2+=High Source: SAPISP 2009 Data

  42. Total Disorder Screener Severity by Level of Care About 30% of OP & SAP are in the high severity range more typical of residential Outpatient & Student Asst. Prog. are Similar (Median 6.0 vs. 6.4) Residential Median (10.5) is higher Well Targeted 95% 1+ 85% 3+ Source: SAPISP 2009 Data and Dennis et al 2006 42

  43. Total Disorder Screener by Level of Care SAP Similar to OP/IOP on Total Source: SAPISP 2009 Data and CSAT 2008 Full subset to Adolescent Intakes

  44. Internalizing Disorder Screener by Level of Care SAP Higher on Internalizing Disorders Source: SAPISP 2009 Data and CSAT 2008 Full subset to Adolescent Intakes

  45. Externalizing Disorder Screener by Level of Care SAP Mod-Hi on Externalizing Disorders Source: SAPISP 2009 Data and CSAT 2008 Full subset to Adolescent Intakes

  46. Substance Disorder Screener by Level of Care SAP Lower on Substance Disorders Source: SAPISP 2009 Data and CSAT 2008 Full subset to Adolescent Intakes

  47. Crime/Violence Screener by Level of Care SAP Lower on Crime/Violence Source: SAPISP 2009 Data and CSAT 2008 Full subset to Adolescent Intakes

  48. Count of Problems (0-4) with Mod/High Severity by Level of Care Source: CSAT 2008 Full subset to Adolescents and Intake

  49. Count of Problems (0-4) with Mod/High Severity by Demographics (n=10,924) Source: SAPISP 2009 Data 49

  50. SAPISP Results: Females (n=5,363) Higher than average on Internalizing Disorders and Lower than average on Crime/Violence Source: SAPISP 2009 Data 50

More Related