Jean Campbell, Ph.D. Program in Consumer Studies & Training Missouri Institute of Mental Health

1. Transforming Mental Health Services Through the Use of Evidence-Based and Emerging Best PracticesColumbia River Doubletree June 3-4, 2004Peer Support & Peer-Run Programs Jean Campbell, Ph.D. Program in Consumer Studies & Training Missouri Institute of Mental Health

2. New Studies & Tools "Science itself is just a tool for achieving human ends; what the political community decides are appropriate ends are not ultimately scientific questions." --Francis Fukuyama, Our Post Human Future, 2002. NY: Farrar, Straus. p 186

3. Peer Support Outcomes ProtocolProject (POPP) Development of an Evaluation Protocol for Community-Based Peer Support Programs (1996-2001)

4. Case for Action • A Need for Measurement • To survive in an era of evidence-based funding, peer support programs need to measure: • cost, • effectiveness, • quality, • utilization, and • appropriateness of the services they provide.

5. Case for Action (cont.) • A need for accountability • To build partnerships between members, program administrators, and external agencies • To improve current programs and tailoring future efforts to • To demonstrate utility and effectiveness to funding partners • To advocate for programmatic efforts and to guide policy • To recruit members and develop community support

6. Case for Action (cont.) • Why a Peer Developed Protocol? As peer support programs continue to grow, so does the need for an outcomes protocol with measures derived from mental health consumers’ experiences and points of view • The POP: • was developed by consumers • can be administered by consumers • embodies consumer values • is consistent with peer support philosophy • recognizes and utilizes proven consumer abilities to conduct survey and outcome studies

7. Overview of the POPP • Purpose: • To develop, field-test, and distribute an evaluation protocol that measures outcomes and satisfaction of community-based peer support programs that are operated by mental health consumers/survivors. • Four Phase Project (1996-2001) • Conducted by: • Protocol and support materials developed and piloted by the Program in Consumer Studies and Training at the Missouri Institute of Mental Health in St. Louis. • Funded by: • The National Research and Training Center on Psychiatric Disability at the University of Illinois-Chicago

8. Utility of the POPP • Assess programoutcomes for consumer self-help field • Present service outcomes to public funding authorities and manage-care organizations • Provide feedback to consumer-run organizations, enabling consumers with information to improve the organization and delivery of peer support programs

9. POPP Outcome Domains Specific outcome domains organized into individual modules: Demographics Service Use Employment Housing/Community Life Quality of Life Well-Being (Recovery, Empowerment & Personhood) Program Satisfaction

10. Characteristics of Protocol • Measures 7 domains • Designed to be done face-to-face • Independent domains can be separated • Avoids burdening respondents • Flexibility to tailor to program needs and goals

11. Psychometric Refinement Criterion-related validity (POPP & Criterion Scales) • Criterion Social Acceptance Scale with Recovery (.55) • Criterion (Rosenburg) Self-Esteem Scale with Personhood (.76) • Criterion Recovery Scale with Recovery (.63) • Criterion Empowerment Scale with Empowerment (.46) • Criterion (QS-8) Satisfaction Scale with Program Satisfaction (.55)

12. Final Protocol • Seven Modules (with 14 scales) • Demographics • Service Use • Employment • Community Life • Final Factor Structure accounted for 60% of the variance • Test-Retest • ranged from .46-.82 • Quality of Life • Well-Being • Program Satisfaction

13. Barriers to Using the POP • Requires information system to effectively manage and utilize information • Requires support and openness to feedback from members of Peer Support Programs • Potential lack of experience and/or training resources to collect, analyze and feedback information

14. Overcoming Barriers • POPP Supporting Resources • Interviewer Training Manual • Question by Question Guide for Interviewers • Tool Kit • Report on POP Psychometrics

15. Consumer Operated Service Program (COSP) Consumer-OperatedService Program (COSP)Multisite Research Initiative 1998-2004 COSP Baseline Findings: Participant Characteristics

16. State of the Evidence • Prior studies of consumer-run programs suggest that they improve symptoms, promote larger social networks, and enhance quality of life. However, the evidence is limited: -uncontrolled studies -demonstrations of feasibility -preliminary findings

17. Importance of Multisite Study • Determine cost-effectiveness of consumer-operated programs -What works for whom at what cost? • Such evidence is necessary for consumer-run programs that seek to be partners in the community continuum of care -Funding & employment opportunities • Study results can promote new programs, improve quality of existing programs , expand services for people not easily engaged in traditional services, & reduce costs

18. Goals of the Study • Establish the extent to which consumer-operated services when offered as an adjunct to traditional mental health services are effective in improving selected outcomes for people with severe mental illness • Create strong and productive partnerships among consumers, service providers and service researchers • Disseminate the knowledge gained

19. Program Models • Drop-in Centers • Educational & Advocacy Training Programs • Peer or Mutual Support Services

20. Connecticut Florida/California Illinois Maine Missouri Pennsylvania Tennessee Participating Study Sites

21. Research Design Four Year Study Rigorous Methodology • Experimental Multisite Design • Randomization • Baseline, 4, 8 & 12 month follow-ups • Common Protocol Intervention: Consumer-Operated Program + Traditional Mental Health Services Control: Traditional Mental Health Services Only

22. Selected Outcomes • Employment • Empowerment • Housing • Service Satisfaction • Social Inclusion • Costs • Well-being

23. 1,827 Study Participants! Largest Study of Consumer Programs in History

24. Demographics • There were more females (60%) than males (40%) among multisite participants. • Slightly less than half (43%) were minorities or individuals who described themselves using two or more race categories. • The average participant age was 43 years old.

25. Demographics • Only 13% of participants were married at baseline with another 23% having a “significant other” to whom they were not married. • On the other hand, 53% reported having children, averaging one child per parent. • Approximately half of the parents indicated their children were under the age of 18 years.

26. Education & Employment • More than half of the participants had achieved at least a high school diploma, with 42% going beyond high school. • Although nearly all participants (97%) had been employed at some point during their life, and 77% said that having a paying job was important to them, only about one-third (29%) were working either for pay or as a volunteer at the time of the baseline interview.

27. Employment & Benefits • Whereas 19% received income from paid employment (including a sheltered workshop), a substantial proportion of participants received income from non-employment sources, including: • Social Security (84%) income • Other social welfare benefits (40%), • Rent supplements (24%).

28. Benefits • Most participants (79%) were receiving benefits that covered their psychiatric care, although only 59% reported that their benefits covered all the services they needed.

29. Housing • Although about half of the participants had been homeless at some time in their lives (51%), most participants’ living situations at baseline were fairly stable • 85% reporting that there was no time limit on how long they could stay at their current place of residence. Note that this percentage does not indicate how many individuals may have wished to move from their current housing situation but were unable to do so; data were not collected on desire to change current housing.

30. Housing More than half of the participants lived in their own residences at the time of the baseline interview (58%), 16% lived in someone else’s residence, 19% lived in temporary housing; and only 2% were currently homeless. About one-third of participants were living alone (36%).

31. Housing Of the two-thirds of participants who lived with someone else, • 41% lived with another mental health consumer • 29% lived with a spouse or other live-in partner • 15% lived with their parents • 28% lived with their children • 14% lived with other family members • 27% lived with a non-related person

32. Housing • Overall, the housing situation of participants was positive, especially when compared to the fairly high rate of past homelessness reported by these same individuals.

33. Diagnosis • Nearly half of the participants were diagnosed with Schizophrenia and Schizoaffective Disorder (47%) • 31% Schizophrenia, 16% Schizoaffective Disorder • Depression was diagnosed for 25% of the participants • Bipolar Disorder for 18%. • Other major diagnostic categories represented among COSP participants included Anxiety Disorders, Dysthymia, and Psychotic Disorders other than Schizophrenia.

34. Diagnosis • Secondary diagnoses on Axis I were found in a small number (11%) of participants with 76% of those with more than one diagnosis having substance-related disorders. • The majority of these substance-related, secondary diagnoses were reported for participants in one study site that specifically provides services to a dually-diagnosed population.

35. Psychiatric Treatment History • Most participants had been hospitalized for psychiatric/emotional problems at some point in their lives (82%) with 25.8 years old being the average age at first psychiatric hospitalization. • Although 85% of these participants had been hospitalized more than once, and 62% had from 2-10 hospitalizations, relatively few participants reported any hospitalization for only a psychiatric reason within the four months prior to baseline (16%).

36. Psychiatric Treatment History • The average first age of any psychiatric contact – whether that was hospitalization or outpatient treatment – was 23.2. • Almost all participants reported that they had been taking prescribed psychiatric medications within the past four months (96%), and/or had seen a psychiatrist in the past four months (89%), with 91% engaging in both treatment activities. • 95% reported experiencing side effects from psychiatric medications.

37. Psychiatric Treatment History • These percentages reflect a high level of participant involvement in the traditional mental health service delivery system.

38. Program Model Differences • Most of the characteristics of participants were significantly different across the program models. • The percentage of men in the study was lower for the education/advocacy programs than for the drop-in centers and mutual support programs. • The percentage of white participants was lower for the mutual support programs, as was the average age. • The percentage of study participants who were married was greater for those in the education/advocacy programs.

39. Program Model Differences • A higher percentage of drop-in center participants had received some Social Security income in the 30 days previous to the baseline interview. However, these participants did not appear to have the most severe illness as indicated by age at first psychiatric contact and lifetime history of hospitalization. • The study participants in the mutual support programs had the youngest age of first psychiatric contact, and were more likely than participants at other sites to report 5 or more lifetime hospitalizations for psychiatric/emotional problems.

40. Program Model Differences • However, the Hopkins Symptom Checklist (HSCL) indicated that participants of drop-in centers reported the lowest degree of symptoms, as did the Colorado Symptom Index Psychosis Subscale (CSIP). • Study participants in the education/advocacy programs were more likely to live in their own residence at baseline, and less likely ever to have been homeless. • On the other hand, study participants in the mutual support programs felt they were more likely to have to move from their current housing within the near future.

41. Program Model Differences • Diagnoses of participants in both the mutual support and education/advocacy clusters were nearly evenly divided between schizophrenia and psychotic disorders and mood and anxiety disorders. • At the drop-in center sites, on the other hand, more study participants were diagnosed with Psychotic Disorders and fewer with Mood Disorders.