Consumer Satisfaction Surveys

1. THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Consumer Satisfaction Surveys 2009 Survey For further information contact Dennis McBride (253) 756-2335 or dmcb@u.washington.edu

2. Consumer Satisfaction Surveys 2009 Survey THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Survey Goal The Consumer Surveys are designed to examine quality issues related to Washington State’s delivery of state-funded mental health services

3. Consumer Satisfaction Surveys 2009 Survey THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Audiences • MHD for Federal Reporting • PIWG and other MHD Indicator Groups • Provider Agencies & RSNs (e.g., EQRO) • Consumer Groups

4. Surveys to meet CMS requirements

5. Consumer Satisfaction Surveys 2009 Survey THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Samples • Persons receiving at least one hour of service within the 6 month time frame. • Adult Consumer Survey • - Adult: 18+ • Youth & Family Survey • - Youth: 13 – 20 • - Family: Children < 13 • Stratified Random Sample • Probability Proportionate to Size

6. Consumer Satisfaction Surveys 2009 Survey THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING WIMHRT’s CATI System

7. Consumer Satisfaction Surveys 2009 Survey THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Sources of Data MHD Consumer Information System Contact Information • ACES Barcode data – Economic Services of DSHS • Regional Support Networks • Individual provider agencies (contractors)

8. Adult Consumer Satisfaction 2009 Survey Selecting Participants Sample Frame Drawn Sample Respondent Sample 1,565 6,078 4,513

9. Consumer Satisfaction Surveys 2009 Survey ACS Disposition Incorrect numbers: incorrect, disconnected, and those respondents for whom contact data were never obtained. Other: claimed had already responded to survey, requested a mailed survey but didn’t respond, requested callback but unavailable after numerous attempts to do so.

10. Consumer Satisfaction Surveys 2009 Survey Response Rates & Cooperation Rates Over Time: ACS

11. Consumer Satisfaction Surveys 2009 Survey THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Total number of calls = 38,619 ACS: 27,091 YFS: 11,528 Average number of calls per completion: 16.27

12. Consumer Satisfaction Surveys 2009 Survey THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING “Representativeness” ACS YFS Respondent Sample Drawn Sample Sample Frame Service Hours Service Hours Age Age Female Minority Female Minority

13. Consumer Satisfaction Surveys 2009 Survey Sample Scale Creation Alpha = 0.82 Perception of Access The location of services was convenient. Staff returned my calls within 24 hours Scale Staff were willing to see me as often as I felt it was necessary. Services were available at times that were good for me. Scoring : 1 Strongly Disagree; 2 Disagree; 3 Undecided; 4 Agree; 5 Strongly Agree

14. Alpha General Satisfaction .87 Appropriateness and Quality of Services .87 Participation in Treatment Goals .57 Perceived Outcomes .90 Perception of Access .82 NOMS Functioning .86 NOMS Social Connectedness .79 Stigma .87 Adult Consumer Satisfaction 2009 Survey Scale Construction • 1-5 MHSIP: Mental Health Statistical Improvement Project • 6-9 Added in 2007

15. Adult Consumer Satisfaction 2009 Survey THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING ACS Scales Percent Satisfied Average Score Quality of Service Participation in Treatment Perception of Access Perceived Outcomes General Satisfaction

16. Adult Consumer Satisfaction 2009 Survey THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING ACS Perceived Stigma by Gender/Age Percent Satisfied Average Score Male 76 + 18 to <21 21 to <41 41 to <61 61 to <76 Female Age (years)

17. r * General Satisfaction -.134 Appropriateness and Quality of Services -.183 Participation in Treatment Goals -.153 Perceived Outcomes -.260 Perception of Access -.185 NOMS Functioning -.246 NOMS Social Connectedness -.304 Adult Consumer Satisfaction 2009 Survey Stigma Scale Correlations • P < .001 • N = 4018

18. Consumer Satisfaction Surveys 2009 Survey THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Stepwise Regression Dependent Variable: Stigma Scale Predictors: NOMS Social Connectedness Perceived Outcome of Service R2 = .103

19. Youth & Family Survey 2009 Survey THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING YFS Scales Percent Satisfied Average Score General Satisfaction Voice in Service Delivery Perceived Outcomes Staff Satisfaction Appropriate-ness of Services Staff Cultural Sensitivity Access to Services

20. Consumer Satisfaction Surveys 2009 Survey THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Other Relationships • Demographics and Total Service Hours • Living Situation by Age, Gender, & Ethnicity • Employment Status by Age, Gender, & Ethnicity • Medical/Insurance Information by Age, Gender, & Ethnicity • Satisfaction Scales by Age, Gender, & Ethnicity • Satisfaction Scales by RSN

21. Consumer Satisfaction Surveys 2009 Survey THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING What two things do you like the most about the services you received? ACS: First Response

22. Consumer Satisfaction Surveys 2009 Survey THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Open-ended Responses: LikeMost Services: “I appreciated the fact that, after my intake interview, I was placed with a counselor *immediately* who seemed very much in touch or able to address my specific problems. That was fast and accurate work.” Professional Staff: “Current counselor has been the most helpful in the short amount of time, calls and checks on me to make sure that I’m ok. Open to phone calls when I can’t come in.”

23. Consumer Satisfaction Surveys 2009 Survey THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING What two things do you like the least about the services you received? YFS: Family First Response

24. Consumer Satisfaction Surveys 2009 Survey THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Open-ended Responses: LikeLeast (Family) Access: “The appointment times are difficult due to being between 9-4. Must take vacation time from work to go to appointments and it took away from school time.” Services: “They seem to have a rigid program/mold where they give the same drugs for everything. It is not individualized…”

25. TRENDS

26. Consumer Satisfaction Surveys THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Trends in Mean ACS Satisfaction Scores Over Time

27. Consumer Satisfaction Surveys THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING ACS Perceived Stigma by Year Percent Satisfied Average Score 2007 2008 2009

28. Consumer Satisfaction Surveys THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING ACS Functioning/Connectedness Scores Over Time

29. Consumer Satisfaction Surveys 2009 Survey THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Trends in Mean YFS Satisfaction Scores Over Time

30. Consumer Satisfaction Surveys 2009 Survey THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Summary of Findings (ACS) • Service satisfaction trends remained consistent since 2002. • General satisfaction scores were relatively high with three quarters of consumers saying they are satisfied. • Across years, Perceived Outcomes has remained the lowest with more than one third of consumers being undecided or dissatisfied in this area. • Consumers continue to be most satisfied with the appropriateness and quality of services.

31. Consumer Satisfaction Surveys 2009 Survey THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING Summary of Findings (YFS) • All areas of youth and family services have improved since 2002. • Staff Cultural Sensitivity and Staff Satisfaction continue to be the highest rated area. • Perceived outcomes is consistently the lowest rated area.