A Mixed Methods Approach to Assessing How System Change Impacts Clinics and Consumers in Public Mental Health

1. A Mixed Methods Approach to Assessing How System Change Impacts Clinics and Consumers in Public Mental Health  Joel T. Braslow, M.D., Ph.D. Semel Neuropsychiatric Institute UCLA John S. Brekke, Ph.D. Frances Larson Professor of Social Work Research School of Social Work University of Southern California Kathleen Daly, M.D., MPH Los Angeles County Department of Mental Health Mapping Progress in Mental Health Disparities, May 20, 2009

2. The California MHS Act: Impact on Practice & Organizational Culture in Public ClinicsNIMH: R-01 MH08067 (Joint PIs: Braslow, Brekke: Co-PI: Kathleen Daly)

3. Study Goals What changes occur as a consequence of the MHSA? Why does change occur (or not)?

4. MHSA in Historical Perspective Apparent discontinuity with the past Continuities with the past Intended and unintended consequences Much of my previous research has focused on therapeutic practices in California state hospitals over the lastMuch of my previous research has focused on therapeutic practices in California state hospitals over the last

6. Average Daily Antipsychotic Drug Dosage, California State Hospitals, 1955-2001

8. Continuities

9. Two Centuries of Recovery

10. UNINTENDED CONSEQUENCES OF POLICIES

11. California State Hospital Population, 1850-1995

12. Elpers JR (1989), Public Mental Health Funding in California, 1959 to 1989, Psychiatric Services

14. Will the MHSA actually �transform� care in Los Angeles County? How do we make sense of the continuities with the past? How do we assess both intended and unintended consequences?

15. California and Los Angeles California population of 36.8 million 12 million live in households below 200% of the FPL 8.8% of these low-income residents meet the MHSA funding allocation criteria of a �serious emotional disturbance (SED) [or] serious mental illness (SMI) Los Angeles County population of 10.25 million 3.95 million individuals living in households below 200% of the FPL 8.7% of these low-income residents meet the MHSA funding allocation criteria of a SED LAC will receive ~30% of MHSA funds

17. The California MHS Act: Impact on Practice & Organizational Culture in Public ClinicsNIMH: R-01 MH08067 (Joint PIs: Braslow, Brekke; Co-PI: Kathleen Daly)

18. Senior Study Team Joel Braslow, MD/PhD Psychiatry/History John Brekke, PhD Social Work Kathleen Daly, MD/MPH Psychiatry Beth Bromley, MD/PhD Psychiatry/ Anthropology Project Director, Amanda Nelligan, MA Psychology

19. Research Staff 4 PhD-level Ethnographers 4 Consumer Specialists 2 PhD Graduate Students

20. Collaborations Public-Academic Partnership: LAC DMH UCLA USC Interdisciplinary Collaborations: Health Services Research Health economics Biostatistics Anthropology Ethnocultural Mental Health

21. Specific Aims Aim 1 County level: Assess how LAC DMH develops and implements MHSA clinical policy over time examining fiscal, clinical, political, historical determinants. Aim 2 Clinic Level: Assess how clinics implement LAC DMH�s interpretation of the MHSA, and how this transforms clinical culture, structure, and providers� understandings of illness and treatment. Aim 3 Client Level: Assess the impact of local clinical transformations on clients� objective outcomes and subjective experiences of their mental health treatment.

22. Mixed-Methods Approach Goal What effects the MHSA has on clients Why the observed effects occurred Quantitative methods using: Provider questionnaires Client questionnaires LAC DMH IS Database Qualitative methods using: Semi-structured interviews Ethnographic observations Focus groups

26. Summary The four studies targeted system factors and then examined their impact on client-level outcomes. None of them examined clinic level factors, yet all involved a clinic level context of service delivery. The clinic and its context was a black box in these studies, despite being critical in mediating between system-level and client level outcomes.

27. Aim 1 Research Questions(Assess how the LAC DMH develops and implements MHSA clinical policy over time.) What clinical, fiscal, and political concerns enter into LAC DMH MHSA policy implementation and policy change over time? What ways in which past policy decisions (e.g., AB 2034) shape current policy decisions?

28. Aim 1 Design Administrative/historical documents Semi-structured interviews (n=18) three levels (executive management, district chiefs and staff) baseline and at the end of years 1, 2, & 3 Five focus groups (1) Union of American Physicians and Dentists; (2) Services Employee International Union (SEIU local 660); (3) National Alliance for the Mentally Ill (Los Angeles Chapter); (4) Los Angeles County Client Coalition; and (5) California Mental Health Directors Association

30. Aim 2 Clinic Level: Assess how clinics implement LAC DMH�s interpretation of the MHSA, and how this transforms clinical culture, structure, and providers� understandings of illness and treatment. 

31. Aim 2 Research Questions How and why do providers� beliefs, attitudes and practices change over time? Do FSP (MHSA) providers differ in their recovery orientation and competencies relative to non-FSP providers? Does MHSA clinic funding affect recovery competencies/orientation for providers not in FSPs?

32. Los Angeles County 469.1 square miles 8 Service Planning Areas 321 clinics 33 directly operated by DMH and treat approximately 60% of unique DMH adult outpatients; 288 contract clinics

33. Design For Aims 2 & 3 Intensive Case Study with Quasi-Experimental Cross-site Comparisons 3 MHSA Clinics (2 DO, 1 Contact) 2 Non-MHSA Clinics (1 DO, 1 Contract)

34. Clinic Site Selection

35. Collaborative Sites 2 Contract Clinics 1 with MHSA funding with Full Service Partnership 1 with no MHSA Funding 3 Directly-Operated Clinics 2 with MHSA funding with Full Service Partnership 1 with no MHSA Funding

36. Aim 2 Methods Ethnographic observation Semi-structured provider interviews (n=50; 6 waves) Observation of clinician-client visits Provider self-administered surveys (n=377; 3 waves) Administrative data from LAC DMH IS database

37. Provider Assessments Survey Universal Sample of All clinicians Frequency: every 12 months Semi-structured interviews Sample based on purposive criteria Frequency: 6 months

38. Provider Survey Recovery Self-Assessment (RSA)-Provider Version Gauges providers� perceptions of how well programs implement practices consistent with principles of recovery. Attitudes Towards Illness Beliefs about: A) impact of illness, B) chronicity, C) client understanding and control of illness, D) causes of illness, E) impact of treatment Clinic Culture Clinic culture, structure, climate, work attitudes (Glisson)

39. Provider Semi-Structured Interviews Open-ended interviews focusing on beliefs about recovery, illness and therapeutics Frequency: 6 months

40. Provider snapshots of clinic life 15 minute face-to-face or phone interactions every two weeks. Cohort followed over time.

41. Aim 3 Client Level: Assess the impact of local clinical transformations on clients� objective outcomes and subjective experiences of their mental health treatment.

42. Aim 3 Research Questions How and why do clients� experiences of care change? How effective are FSPs relative to usual care in MHSA-funded clinics? Does MHSA funding result in worse outcomes for usual-care clients in MHSA- funded clinics?

43. Aim 3 Methods Ethnographic observation Semi-structured client interviews (n=50; 6 waves) Client self-administered surveys (n=616; every six months over 3 years) Administrative data from LAC DMH IS system

44. Client Sampling Design

45. Matching Criteria high service utilization or homeless or jail diagnosis age gender ethnicity

46. Client Assessment Client assessment surveys Semi-structured interviews Observation of client-provider interactions

47. Client Assessment Continued The BASIS-32 is a brief, widely-used behavioral health assessment instrument that provides a comprehensive measure of functioning. Recovery Self-Assessment: Person in Recovery version gauges perceptions of how well programs implement practices consistent with principles of recovery Working Alliance Inventory; Attitudes Towards Illness; Living situation and work functioning Quality of life measures: SWL, AQOL, Acculturation and Ethnic identity Frequency: 6 months

48. Semi-Structured Interviews 5 Clients per cell (10-15/clinic) Frequency: 6 months Aim: probe clients� subjective experiences of treatment (its process and impact on them) and of the clinic.

52. How are we mixing our methods? By type: quantitative measures, focus groups, ethnographic observation, semi-structured interviews By utilization: a) quantitative results are used to select Ss for qualitative interviews; b) qualitative data will be used to interpret pooled and site-specific findings (confirmations and disconfirmations); c) qualitative data will be used to set context for quantitative findings (e.g., leadership, relationship with County); By linking constructs: especially around a) beliefs and behaviors concerning illness and treatment; b)

53. Data collection Clients enrolled: 454 Providers enrolled: 293 All surveyed 99 semi-structured interviews Clients enrolled: 454 All surveyed 68 semi-structured interviews

56. Preliminary data on usual care clients� living situation (site a)