A Framework for the Evaluation of Recovery-Focused Case Management Making Gains Conference , 2009

1. A Framework for the Evaluation of Recovery-Focused Case Management Making Gains Conference , 2009 Jason Newberry, PhD, Research Director, Centre for Community Based Research jason@communitybasedresearch.ca Allan Strong Recovery Education Coordinator stronga@self-help.ca

2. Interviewer: When you started your recovery plan, what role did you have in developing it? Service User: I had a big role in it, actually. They allowed me to come up with the idea that education was the main thing I would concentrate on. I just needed the help, kind of like a hand up to do that. Interviewer: What are some of the useful things you’ve learned through your recovery process? Service User: I’ve learned I should always have a goal.

3. Today’s Presentation • History and background • Overview of the training • The evaluation process • Summary of findings • Next steps • Questions and discussion

4. Background and History • 2005 - Ministry of Health and Long Term care provided funding for the enhancement of case management in the area ( 1.13 million dollars) • Decision about how the money should be allocated must be made by consensus • Planning process involved 3 community agencies, 3 hospitals and an alliance of consumer - survivor groups

5. Organizations Involved in Planning • Waterloo Regional Homes for Mental Health • Grand River Hospital • Trellis Mental Health and Developmental Services • Homewood Health Centre • Cambridge Memorial Hospital • CMHA Grand River Branch • The Self Help Alliance

6. How did we do it? • Developed and agreed to a decision making process • Adopted a statement of values and principles to guide our work • Designed services that were “true” to guiding principles and values 6

7. The Values and Principles • Statement was developed by the Self Help Alliance • Statement addresses how the system will look if values and principles of recovery put in place 7

8. A recovery oriented mental health and addiction system will : • Value Empowerment – The system will provide opportunities for individuals to exercise control and power with respect to their lives. • Value and Instill Hope – Recovery can not occur without hope. The system will encourage hope and in doing so will focus on skills and abilities.

9. Value Self Determination – The system will recognize and accept that consumers will make their own decisions about their life and they will be in control of those decisions. • Work toward the Elimination of Prejudice and Discrimination – The system will work toward the elimination of prejudice and discrimination toward people with mental health issues.

10. Value Meaningful Choice – The system will recognize and accept that individuals will make their own decisions about their life and they will be in control of those decisions. From these key values, principles were developed, some of which include:

11. Recovery principles • Development of recovery skills • Development of recovery communities • Development of recovery relationships • Development of recovery identities

12. This approach was different because....... • Started from a shared set of values and principles that were developed by consumers. This has not been usually been the case. • Values and principles provide not only a common understanding but also something to hold each other accountable. 12

13. Setting the stage • Trip to Ohio - service providers went to Ohio • Self Help Alliance had been developing its own programming based on recovery (independent of service providers) 13

14. Consumer and Case Manager Training • A central part of system transformation. • Training of staff in recovery values and principles, and practice. • Training of consumers about recovery and their role in planning 14

15. Consumer and Case Manager Training Since April 2006, 37 staff training sessions 0ver 250 staff (case managers, housing staff, ACTT, management). Since January 2007, 100 consumer training sessions. Over 250 consumers participated. 15 15

16. Two Day Training • Day One • Focus is the development of an understanding of recovery values and principles and what does it mean for the work • This is achieved by lecture, interactive discussion, review of articles, interview with a consumer • How do you see this changing your work

17. Recovery Training • Day Two • Learning the planning process, focus is on the process and the conversation • It is about the process, discussion and the relationship • This is achieved by learning the planning process, practice and role play and review 17 17

18. One Day Refresher • Opportunity to review the process and to reflect upon and to discuss the experience of doing the work • To think critically about the challenges and opportunities presented by doing recovery oriented work

19. Supervisor’s Training • Beginning in January 2010 • Training to provide supervisor’s from agencies to learn skills and approaches that will support staff to do recovery oriented work

20. Consumer Education • Provides the opportunity for those receiving service to learn about the planning process • Also provides the opportunity to discuss the notion of “recovery’ and what does that mean for them

21. Future directions for training • Developing some education for members of the Board of Directors – how does this shift affect governance • Working with GRH to develop training/educational opportunities for in-patient staff – to be provided in hospital • Continue to look for opportunities to work with other community agencies

22. The Overarching Evaluation Question: “To understand how a recovery focus in the Waterloo-Wellington-Dufferin mental health system leads to positive changes for individuals using case-management services”

24. 1. How and to what extent is “recovery focused” approach to case management helping people gain the skills and knowledge they need to develop their recovery plan and pursue their own goals? Increased knowledge of the concept & process of recovery Increased skills & knowledge to pursue goals Willingness to take risks, to fail, & to learn from it Acting on one’s own plan for recovery (practicing options) Increased confidence & faith in own decisions, needs, values 4. To what extent are people feeling an increased sense of control and choice regarding their health and lives? STAFF RELATIONSHIPS KNOWLEDGE CONTROL 3. How and to what extent is recovery focused approach to case management helping to alter the traditional relationships between case managers and people they support? What is being done to ensure reciprocal, honest and open relationships between case managers and service users? Increased sense of control & choice regarding their health, lives Increased feelings of trust, comfort, and respect for lived experience 1 4. 2 2. Are people beginning to understand the idea of recovery and what it can mean for their own goals? A sense of diminished professional boundaries, informal relationships 3 Increased confidence in workers & the system Increased feelings of hope & ability to dream about recovery 6 5. To what extent are people beginning to dream about recovery and are hopeful that they will reach their goals? 5 ELEMENTS OF RECOVERY 6. To what extent are people active in their recovery plan – taking risks, practicing options, and achieving learning and confidence?

25. Increased knowledge of the concept & process of recovery Increased skills & knowledge to pursue goals Willingness to take risks, to fail, & to learn from it Sharing the process & success in relationship to others Greater participation in regular, day-to-day community life Acting on one’s own plan for recovery (practicing options) Increased confidence & faith in own decisions, needs, values Increased sense of control & choice regarding their health, lives Increased feelings of trust, comfort, and respect for lived experience A sense of diminished professional boundaries, informal relationships Increased confidence in workers & the system Increased feelings of hope & ability to dream about recovery 7. To what extent are people experiencing a greater sense of support from others in thie personal lives and participating in community life Increased connections & sense of support in community, social & family life 7 SOCIAL SUPPORT & COMMUNITY INTEGRATION ELEMENTS OF RECOVERY

26. Accountability statements regarding expectations of policy Build partnerships mental health sector & across social service sectors Structure and support practice consistent with policies, values Training of staff re: principles of recovery Greater coordination, improved partnerships among regional partners Improved staff knowledge & skills in doing recovery action planning Improved attitudes regarding values of recovery among staff Greater sharing & consistency of language & values; recognition of organizational differences Increased empathy, compassion, sensitivity of CMs Less diagnosis focus in system Services are individualized, flexible Greater continuity of services across SPs - “similarly principled service” but retaining unique roles Accessible, inclusive, & responsive services Staff are free & willing to have (and see value in having) more flexible boundaries within their professional relationships System recognizes & value importance of lived experience & recovery • What are the experiences, successes, & challenges of the organizations within partnership in building a collaborative, system wide approach to recovery-focused case management? • What is the impact of the training on the knowledge and skills of case managers to promote a recovery focus and engage service users with recovery action planning? What are the successes and challenges? • Is a recovery focus apparent in the practice? Are there attitudinal or organizational barriers? System and Service Provider Model

27. Design & Methodology – Service Users • Service User Survey Interview • A mix of demographics, existing & customized scales focused on outcomes, and qualitative questions focused on service use. • Page 15 of the report has a table that matches measures to outcomes. • This is our main “pre-post” to examine change. • 86 at baseline; 101 follow-up; 62 both Phases. • Service User Qualitative Interviews • Qualitative interviews to explore ideas about recovery, its relationship to case management, and expected outcomes. • 14 participants (8 Well.-Duff.; 6 Wat.Reg.)

28. Design & Methodology – Case Managers • Case Manager Online Survey • A combination of custom scale items and open-ended questions to assess: • the implementation of the recovery training • use of the recovery action plan tool in practice • perceptions and attitudes regarding recovery values and practice. • 57 at pre-test; 31 at post test; 16 both (9 “don’t remember”) • Supervisor Key Informant Interviews • Review changes to recovery planning and the rationale; reflect on the process so far. 10 participants, all organizations represented.

29. Challenges in Recovery Planning “I think we got caught up in the tool and the resources, the recovery book. I think we felt that we had to follow that, we had to pull out the book, we had to do this routinized process, this is how we have to follow it. I think in using it, we lost sight of the flexibility of the conversation of the excitement, and we got kind of stuck on a tool.” - Supervisor “The Recovery tools presented in the original training were too confusing and made the process difficult to facilitate with those supported. The recovery tools could actually be a barrier to doing the planning.” – Case manager

30. Challenges in Recovery Planning Gaining buy-in: “I find it difficult with some of the people I support to commit to planning. For some, priorities change quickly and the planning can't keep up (or doesn't get completed because the topic is dropped). For some, it's beyond what they want to deal with. For some I've struggled to spark recovery alive for them and they are not willing to plan to change anything in their lives.” - Case manager

31. Challenges in Recovery Planning • “Recovery Readiness” • Attitude still persists in that some individuals, in some circumstances are not ready for recovery planning, especially those in ACTT. • When people are experiencing a lot of symptoms, substance use, other crises. • “Motivation” was also invoked as a reason. • Supervisors attempting to address recovery in the most challenging circumstances – what does it look like?

32. Challenges in Recovery Planning • “Recovery Readiness” • Recovery readiness may have shifted in response to the tool itself. Because the tool was complex, certain people may be selected out. • The potential problem was that case managers and service users started equating “recovery” with “completing the tool”. • Organizations have begun to make efforts to simplify the tool, but also to reinforce the notion that recovery is a process, and not merely a completed tool.

33. Core Elements of Recovery • According to supervisors & case managers, the most important element of recovery planning was an engaged conversation in the context of a trusting relationship. • Conversations should occur in the context of planning that includes identification of: • Priority goal areas (of 10) • Personal goals within areas • Knowledge, skills, community resources to support goals • Specific plans/action steps to reach goals

34. Core Elements of Recovery I want to make sure that we keep the fact that it’s a conversation, that everything is about the conversation and finding a way to capture that. I think the core of recovery is that it is not the tool, it’s the conversations and it’s relationships. To have the conversations we have to have the relationships and that’s vital to recovery. I do think that it gives us all as service providers some common language. - Supervisor

35. I’ve always hated the “recovery package”, and said that it was written by social workers for social workers….I hated the idea of being told that my recovery was in my control and then being told exactly how to do it. My worker told me the package is gone, they no longer use it. After talking to my worker, I realized that they actually listened to the feedback they received. My opinions actually mattered. Now I am curious and wanting to learn more – to take a fresh look at recovery , to get a new start. - Service user

36. Service User Evaluation – Demographics • 123 service users participated in at least one of the surveys: • 86 in Phase I; 101 in Phase II; 62 in both • 58% were male, 42% female • Average CM use in the region: 5.7 years • 70% of had developed recovery plans; 30% had not. • 58% had completed H.S.; 21% had college/university; 21% had primary school. • 63% were single. • All service users received income from Ontario Works/ODSP (79%) and/or CPP/Disability (33%). 16% received income from employment

37. Service User Evaluation – Comparing Baseline to Follow Up • Our surveys assessed: • Access to Resources • Community Integration • Social Support • Knowledge of Recovery Concept and Own Goals • Relationships with Staff • Independence and Interdependence • Empowerment (control and choice) • Mental Health Recovery (elements of Hope and Dreaming, Risk-taking, Action and Confidence)

38. Analysis of Outcomes We compared baseline to follow up on all our measures across all service users. We compared baseline to follow up for service users who have a recovery plan versus those that do not. We also compared outcomes based on demographic variables.

39. Access to Resources • Service users who had recovery plans were more likely to use other community mental health services (65%) than those who did not have recovery plans (31%). This difference was not present at baseline. Community Integration • Service users who were working on the community involvement goal area had higher ratings of community integration at follow up; this difference was not present at baseline.

40. Social Support • Social support did not increase over time in general. Social support was greater for those who had recovery plans, but this did not reach significance (p < .056). • In qualitative findings, many service users linked their levels of social support to case management. Peer support was viewed as important to people.

41. Identifying Recovery Goals

42. Empowerment • Service users with recovery plans had higher scores on a measure of empowerment. This difference was not present at baseline. Independence & Interdependence • Independence increased for all service users from baseline to follow up. It was not related to recovery planning nor was it correlated with the MHR measure. • Interdependence was higher for service users with a recovery plan, and was correlated with MHR. • Interpretation: Independence is important for daily living but is not associated with the more fundamental values of recovery. Shows difference between PSR and recovery approaches.

43. Mental Health Recovery • Service users with recovery plans had higher scores on a measure of mental health recovery. This is our main outcome measure to which all other outcomes are thought to contribute (and are all correlated). This difference was not present at baseline. • Service users with recovery plans had higher ratings of perceived impact of CM on their recovery.

44. Staff Relationships • No significant differences from baseline to follow up. However, scores were quite high at baseline, and although they increased, there was not much “room to move” – this looks like a ceiling effect. Staff relationships have been very strong from the outset. • Qualitative data backs this up – very positive comments across the survey and interviews.

45. Questions? Comments?