Evaluating Mental Health Services

1. Evaluating Mental Health Services Christine B. Kleinpeter, MSW., Psy.D., Professor CSU, Long Beach Rob Bachmann, RN, MN, Director Golden West College

2. Presentation Overview I. Current Study II. Prior Studies III. Recommendations

3. I. Current Project Stages of Change and Therapeutic Alliance among College Students Seeking Mental Health Services in an Integrated Health Center

4. GWC Student Demographics N=15,798 Students Males=46.4% Females=53.0% African-American=1.9% Asian=27.5% White=32.5% Hispanic=27.6% Multicultural=4.0% Unreported=3.8% Mean age=24.7%

5. Mental Health Demographics N=130 (2012-2013) Males=28.7% Females=71.3% Asian=27.0% Hispanic=18.3% White=52.2% Other=2.6% Mean age=26.2

6. Purpose of the Study • Describe who we treat and what type of treatment they receive? • Compare our students in the health center to the general population of GWC students to see if we are serving our campus in all of its diversity? • To compare our current year (2012-2013) data to prior years for program improvement • To study the relationship between therapeutic bond and treatment progress

7. Methods • Exploratory/descriptive study • Closed case review of mental health data (2012-2013)using a data retrieval form • GWC data was provided by University Research • Treatment progress was measured by the Rhode Island Change Assessment (URICA) administered pre and post treatment • The therapeutic bond between clients and therapists was measured by the Working Alliance Inventory (WAI)at discharge • SPSS was used to analyze data

8. Results: Demographics • MH Students are female (71.3%) and slightly older (26.2 vs 24.7) • Whites are over-represented in the MH group (52.2% vs 32.5%) • Asians are similar in both groups (27.5% vs 27.0%) • Latinos were under-represented in the MH group (18.3% vs 27.6%)

9. Results: Clinical • MH Students received M=4.2 sessions, range (0-11) 0=nurse intake only • MH Students presented with depression (24.%), anxiety (15.0%), and stress (13%). • 6.9% Dropped out after the nurse intake, 18.5% after 1 session /Therapist • Students reported strong therapeutic bond with therapists, WAI (M=77) for women and (M=80) for men • Students show improvement in Action Stage (URICA) from intake to discharge (t=5.82, p=.001) • Students who reported strong therapeutic bonds with their counselors had higher levels of treatment progress: Bond (r=.29, p=.02), Task (r=.40, p=.001), and Goal (r=.44, p=.001)

10. Discussion Outreach needs to focus on men and Latinos The total drop out rate is 25.4%, however the therapists’ drop out rate is 18.5% The study findings are consistent with the literature; that is, clients who have a strong therapeutic alliance are more likely to stay in treatment and have positive therapy outcomes. Using Motivational Interviewing by both nurses and therapists is consistent with other studies that have seen positive outcomes with MI in school settings.

11. Administrative Implications In an academic environment where accountability is a key administrative issue, the chart audit provides continuous feedback for program improvement. In addition to reviewing our current program services we have also added new services where we discovered service gaps.

12. II. Prior Studies: GWC • 3 publications • 1 current study under review (URICA) and (WAI) • 1 study in progress (PHQ-9) health and mental health samples

13. Baseline Audit • Baseline audit revealed lack of consistent documentation: • DSM diagnosis • Short-term, measurable goals • Active interventions • Client response to treatment • Case disposition • High drop out rate (44%) • Uneven standard of care by provider

14. Administrative Response #1 • Uniform charting expectations • Case disposition follow-up calls • Staff training in Motivational Interviewing • Staff supervision regarding deficiencies • Nurses intake includes client motivation and goal setting • Front desk staff make reminder calls to clients

15. Second Audit: Replication Study • Drop out rate decreased to 25% • Most charts contained: diagnosis, goals, treatment plan, and active interventions • Interventions of evidenced-based practice were clearly documented • Outcomes were measured: goals met, partially met, or not met • A symptom checklist was given pre and post, documenting a decrease in symptoms • We served 146 clients, up from 118 last year

16. Administrative Response #2 • Improving outcomes was rewarding to the staff • We identified 2 gaps in services in this audit: medication evaluation and groups • Our physician agreed to meet the need for medication evaluation and management • We began a Veteran’s Support Group and a Student Nurses Support Group

17. Third Audit • Low drop out rate of 15.9% • Served 132 students • High Client Satisfaction (CSQ-8) (29.79 out of 32 possible) • Students need help with positive coping skills (CSI) (High Avoidance)

18. Administrative Response #3 • The efforts made in reducing no- shows and cancellations is holding as seen in the low drop out rate • The documentation continues to be complete and accurate • Coping skills lectures have been provided by Health Center staff in individual classrooms • The need for a platform to teach coping skills to the larger student body is needed

19. Summary • The audit provides important information about service gaps and areas in need of improvement • The staff receives feedback that leads to improved patient care • The program improves by ensuring that services offered are current and evidenced-based • The students benefit when we monitor client satisfaction and ask for their input

20. III. Recommendations • Useful research questions • Practical Instruments/surveys • Research that improves services

21. First Steps • Take a baseline audit of your charts: • Number of students seen per year • Demographics of students seen • Number of therapy sessions per client • Drop out rate • Number of service hours provided each year by the clinic

22. Comparison of your data • Comparing your MH students’ demographics to your college demographics to demonstrate you serve all students on your campus • Comparing your data with prior years to monitor program improvements over the years • Comparing your data to other colleges to learn from each other

23. Use Standardized Instruments • Using standardized instruments means that they are valid and reliable. This allows you to compare your data with other colleges and universities • Using client checklists is an easy way to get started because they take little time to score and clients can complete them while they wait for their appointment

24. Research Questions • Client satisfaction is a very good place to start. We used the Client Satisfaction Questionnaire (CSQ-8), as a measure of consumer satisfaction. • Therapeutic Alliance is a good measure of the client/counselor bond, we used the Working Alliance Inventory (WAI) (12 items). • Measuring outcome can be done with a pre and post test. We used the Symptoms Checklist (20 items) (Bartone, et., al., 1989) • Measuring outcome can be done with the URICA, using Action subscale pre and post treatment. (32-item, no easy to score)

25. What about specific aspects of my program? • If you have specific questions, you can design a survey that focuses on your program, for example a group for recovering substance users: • There are standardized surveys written for many types of therapy treatments and can be used for objective questions (e.g., Were the objectives of the group met?) • You can design open-ended questions to cover things like: “What two new things did you learn in this group?”

26. What about outcomes with certain types of clients? • You could choose to look at how well you are doing when you treat depressed clients. If so, you can use the Beck Inventory (BDI) 21-items or (CES-D) 20-items, or Zung (20- item). All are self-report checklists. (BDI requires a Psychologist)

27. Future Research • Clinical outcomes that are specific and standardized have been adopted • Larger samples including multiple sites to determine the generalizability of the findings • Studies that include the health clients as well as mental health clients in an effort to become more integrated in our practice • Next year: The PHQ-9 Study for all SHC clients, Would you like to participate??

28. Thank You! Christine B. Kleinpeter, MSW., Psy.D., Professor CSU, Long Beach Rob Bachmann, RN, MN, Director, GWC, Huntington Beach