Quality of Life Measures: Mental Health Services and Consumer Satisfaction

1. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Quality of Life Measures: Mental Health Services and Consumer Satisfaction For further information, please contact: Dennis McBride:(253) 756-2335 | dmcb@uw.edu Catherine M. Wilson: (253) 761-7573 | cmw27@uw.edu

2. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Part 1Consumer Satisfaction SurveysThe Mental Health Statistical Improvement Program(MHSIP) For further information, please contact: Dennis McBride:(253) 756-2335 | dmcb@uw.edu Catherine M. Wilson: (253) 761-7573 | cmw27@uw.edu

3. MHSIP Emerges • 1976 – MHSIP Advisory Group (developed data standards for public mental health systems) • 1987 – National Research Institute (NRI) to Support MHSIP • 1989 – MHSIP Policy Group (Refined Data Standards for Mental Health Systems) • 1996 – MHSIP Consumer-Oriented Report Card (Developed base survey we now use). National Association of State Mental Health Program Directors (NASMHPD) was created to further develop the Application of MHSIP. • 1997 – Five State Feasibility Project (refined survey) • 1998-2001 – 16 State Study on Mental Health Performance Measures (refined survey) • 2001- 2014 – Current Survey

4. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 Goal & Audiences The consumer surveys are designed to examine service quality issues related to Washington State’s delivery of state-funded mental health services. Using Survey Data for: • Federal Reporting • Performance Indicator Work Groups • Provider Agencies and RSNs (e.g., EQROs) • Consumer Groups

5. Federal Uses • Data Infrastructure Grants (DIGS) • National Outcome Measures (NOMS) • Government Performance and Results Act (GPRA) • Uniform Reporting System (URS). The MHSIP Consumer Survey is now being implemented in 55 states/territories for the adult survey and 54 states/territories for youth surveys.

6. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 Surveys to Meet CMS Requirements

7. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 2013 MHSIP Survey Samples • Persons receiving at least one hour of service within a 6 month time frame: May-Oct 2012 Adult Survey: 18+ years old Youth Survey: 13-20 years old Family Survey: <13 years old • Stratified Random Sample (Stratified by RSN, Age, Minority Status • Probability Proportionate to Size (Size adjusted for small RSNs)

8. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 WIMHRT’s CATI Lab System

9. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 Sources of Data

10. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 Selecting Participants • Child & Family Consumer Survey (CFCS) • Sample • Frame • Drawn Sample • Respondent • Sample • Adult Consumer Survey (ACS) • Sample Frame • Drawn Sample • Respondent • Sample F=604 Y=320

11. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 ACS & CFCS Disposition

12. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 Response & Cooperation Rates Over Time

13. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 States’ MHSIP Completion Rates 2008 Source: Sampling and the MHSIP Consumer Surveys: Techniques, Models, Issues. Mary E. Smith & Vijay Ganju (June, 2008). *Removes non-contacts from the denominator.

14. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 2013 Calls & Completions Total number of attempts/calls: 32,628 ACS: 22,023 CFCS: 10,605 Average number of attempts/calls per completion: 14.45

15. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 Total Calls & Completions Total number of attempts/calls: 341,078 ACS: 231,995 CFCS: 109.083 Total number of completions: 24,005 ACS: 15,592 CFCS: 8,413

16. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 Representativeness ACS CFCS

17. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 Adult/Youth Perception of Outcomes Alpha = 0.90 As the result of services I received: • I am better at handling daily life. • I get along better with family members. • I get along better with friends and other people. • I am doing better in school and/or work. • I am better able to cope when things go wrong. • I am satisfied with my family life right now. Scoring: 1-Strongly Disagree; 2-Disagree; 3-Undecided; 4-Agree; 5-Strongly Agree Scale

18. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 Scale Construction *Added in 2007

19. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 ACS Scales

20. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 CFCS Scales

21. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 Other Relationships • Demographics and Total Service Hours • Living Situation by Age, Gender, & Ethnicity • Employment Status by Age, Gender, & Ethnicity • Employment Status by Age, Gender, & Ethnicity • Medical/Insurance Information by Age, Gender, & Ethnicity • Satisfaction Scales by Age, Gender, & Ethnicity • Satisfaction Scales by RSN

22. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 What two things do you like the most about the services you received?

23. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 Open-Ended Responses: Adult – Like Most Support: ”The support that they give me. If I need someone to talk with, they are there. If not, they call right back. They are good listeners and very understanding .” ”You are not rejected on account of anything you might be experiencing/feeling. If there’s one counselor that you’re just not clicking with, they will rotate you to another. They’ll bear with you and just keep going until you can identify your feelings and handle them.”

24. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 What two things do you like the least about the services you received?

25. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 Open-Ended Responses: Family – Like Least Access: ”The location and difficulty in getting after school appointments .” ”Long waiting list and it takes months to see a psychiatrist.” ”I liked least that they are short of time. They had so few choices of time. My child would miss school for appointments.’

26. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 STIGMA

27. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 ACS Perceived Stigma by Gender/Age

28. Consumer Satisfaction Surveys 2013 ACS Perceived Stigma by Gender/Ethnicity

29. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 ACS Perceived Service Outcomes by Stigma

30. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING ACS Consumer Satisfaction Surveys 2012 -13 Correlations: Service Perception by Stigma N = 2578 p < .001

31. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Part 2Trends

32. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 ACS Perceived Stigma by Year

33. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 Trends in Mean ACS Satisfaction Scores Over Time

34. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 Trends in Mean CFCS Satisfaction Scores Over Time

35. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 Functioning/Connectedness Scores Over Time

36. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 States’ Adult MHSIP Scores 2010 Source: 2010 CMHS Uniform Reporting System Output Tables Note: Scores represent the average of those positively agreeing or strongly agreeing with each scale.

37. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 Summary of Findings: ACS • Satisfaction scores on most scales remain high, with at least 70% of consumers on most scales saying they are satisfied or very satisfied with services. • Since 2002, satisfaction score trends have shown slight but steady improvement over time. • Across years, Perceived Outcomes has remained the lowest, with more than one third of consumers being undecided or dissatisfied in this area. • Consumers continue to be most satisfied with Appropriateness and Quality of Services. • Perceived Stigma affects nearly half of consumers and has remained relatively constant since 2007.

38. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 Summary of Findings: CFCS • Satisfaction scores on most scales remain high, with at least three quarters of consumers on most scales saying they are satisfied or very satisfied with services. • Since 2002, all areas of youth and family service satisfaction have continued to improve. • Cultural Sensitivity of Staff and Satisfaction with Staff continue to be the highest rated areas. • Perceived Outcomes is consistently the lowest rated area.

39. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2013 https://depts.washington.edu/pbhjp/dev/projects-programs/page/mental-health-statistics-improvement-program-consumer-satisfaction-surveys