1 / 50

Michael Costa, Abt Barbara J. Silver, CMHS Maria Madison, Abt Tandiwe Njobe, Abt

Constructing a Cross-Site Evaluation of Ethnic Minority HIV Mental Health Services The Mental Health HIV Services Collaborative (MHHSC) Program. Michael Costa, Abt Barbara J. Silver, CMHS Maria Madison, Abt Tandiwe Njobe, Abt Gabriela Garcia, Abt. Presentation Goals.

wright
Download Presentation

Michael Costa, Abt Barbara J. Silver, CMHS Maria Madison, Abt Tandiwe Njobe, Abt

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Constructing a Cross-Site Evaluation of Ethnic Minority HIV Mental Health ServicesThe Mental Health HIV Services Collaborative (MHHSC) Program Michael Costa, Abt Barbara J. Silver, CMHS Maria Madison, Abt Tandiwe Njobe, Abt Gabriela Garcia, Abt

  2. Presentation Goals • Describe the MHHSC Program • Components • Context • Relevance • Convey: • The process of this collaborative. utilization-focused evaluation • Accomplishments to date (products) • Uses of the outputs of products

  3. Substance Abuse Mental Health Services Administration(SAMHSA) • Center for Mental Health Services (CMHS) • Center for Substance Abuse Prevention (CSAP) • Center for Substance Abuse Treatment (CSAT)

  4. SAMHSA HIV/AIDS HISTORY • Prior to 2001: CSAT funds targeted capacity/HIV/AIDS Substance Abuse treatment programs for African American, Hispanic/Latino, and other racial/ethnic minorities. • 2001: CMHS funds a similar targeted/expanded capacity program for community based organizations (CBOs) serving African American, Hispanic/Latino, and other racial/ethnic minorities.

  5. HIV Infection among People with Severe Mental Illness Across all published studies, the rate of HIV infection among psychiatric patients is 10%, 25 times higher than that of the general population. Cournos & McKinnon, 1997;Krakow et al., 1998;Rosenberg et al., 2001

  6. Sexual Risk Behavior Among People With Severe Mental Illness: COMPARED TO GENERAL POPULATION, PATIENTS HAVE FEWER EPISODES OF SEX WITH A PARTNER, BUT THEY HAVE: •  Number of partners •  Number of risky or anonymous partners •  Frequency of sex trading •  Rates of coerced sex • McKinnon et.al., 1996, 1999

  7. Psychiatric Disorders and Risk for HIV Infection • Elevated risk for HIV infection in psychiatric • Patients • Risk factors: • Alcohol and other drug use • Unsafe sex • Environmental circumstances (poverty, institutionalization, etc.) • Substance use is associated with both psychiatric symptoms and HIV risk

  8. MHHSC Program • 21 Mental Health Service Sites – CBOs • at least 2 years experience in behavioral health care services • MH Centers, Substance Abuse facilities, • Primary Health Care &/or HIV/AIDS clinics • Abt Associates, Inc. = Coordinating Center

  9. MHHSC Program • Congressional requirement (CBC & CHC): provide these new HIV/AIDS-related mental health services in both traditional and non-traditional settings. • Funding for mental health treatment services and related case management only. • However, grantees are required to develop comprehensive integrated individual treatment plans and monitor primary and substance use treatment.

  10. WHO ARE THE SERVICE SITES? • New HIV/AIDS-Related Services • New services (no prior HIV/AIDS-related MH services)– 5 sites • Expanded services – 16 sites • Service Delivery Settings • Traditional (primarily clinic-based) – 13 sites • Non-traditional (e.g., mobile treatment, ) – 1 site • Both settings – 8 sites • Target Populations • African American -19 sites • Hispanic/Latino – 14 sites • Haitian – 1 site • Native American – 1 site

  11. Demographics – Gender

  12. Demographics – Race / Ethnicity Hispanic Latino/a = 30.65%

  13. Demographics - Age

  14. DSM IV Diagnoses

  15. DSM IV Diagnoses Categories

  16. MHHSC Program Goals • Expand • Effective • Culturally Competent • Mental Health Services • For PLWHIV • In Minority Communities

  17. MHHSC Capacity Building The Coordinating Center provides technical assistance to grantees: • Local and regional trainings (e.g., ethics, neuropsychology, cultural competence) • Expert speakers at national meetings on topics of mental health and psychiatry, gender issues, consumer involvement, etc. • Assistance with data collection and management

  18. CULTURAL COMPETENCE • Strategies to Promote Culturally-Competence Service Delivery • Demographically similar staffing as target population(s) • Treatment facilities reflect cultural interests of target group(s) (e.g., artwork/decor, waiting room music, etc.) • Diversity/cultural competence training • Consumer feedback • This is being addressed in great detail by the MHHSC Cultural Competence Subcommittee

  19. MHHSC Utilization-Focused Cross-site Evaluation • Clinically/Programmatically Relevant Evaluation • meet the needs of the clinical and other program staff in their efforts to better serve their clients • The MHHSC cross-site evaluation is voluntary, except for submission of required aggregate data • critical that the cross-site evaluation be clinically and programmatically relevant and valuable, otherwise no site would agree to participate

  20. Evaluation as a Pain in the Neck • Mental health/social service providers’ historical experience with evaluation • Intrusive reporting requirement • Necessary to satisfy Local, State and Federal funding requirements. • Often data are never reported back to programs • Concerns that data will used to make them look bad or draw inappropriate comparisons between sites and providers

  21. MHHSC X-Site Evaluation Collaborative Process • Engaging the key stakeholders • Local site-specific evaluators • Local site clinicians • Program Administrators • Consumer Advisory Boards (CABS)

  22. Site-Specific Evaluation Summaries Site Program Logic Models Eval/Clinician Communication Workgroup Determination of Evaluation Foci Across Sites Logic Model Workgroup MHHSC X-Site Evaluation Collaborative Process Evaluation Subcommittee (ESC)

  23. X-Site Candidate Evaluation Foci Face-to-Face Meeting w/MHHSC Stakeholders MHHSC X-Site Evaluation Collaborative Process

  24. X-Site Candidate Evaluation Foci

  25. Quantitative Methods Used Across Sites

  26. Qualitative Methods Used Across Sites

  27. MHHSC National Meeting X-Site Candidate Evaluation Foci Evaluation Design Workgroup Breakout Groups (BOGs) Consumer Network Committee Face-to-Face Meeting w/MHHSC Stakeholders MHHSC X-Site Evaluation Collaborative Process

  28. Field X-Site Evaluation X-Site Design Site Review Instruments/Protocols Final Draft Pilot Test Pray Everything Goes As Planned Instruments/Protocols Beta Version MHHSC X-Site Evaluation Collaborative ProcessNext Steps

  29. Final Cross Site Foci • Four main domains* of interest across sites: • Mental Health • Client Satisfaction • Client Retention and Service Utilization • Quality of Life & Medical Health *Cultural Competence appears in all domains.

  30. Common Questions of Interest • Who is being served? • What are the barriers to care and to services for the target population? How do programs overcome these barriers? • What services are being used by the target population? • Are the services being provided in a culturally competent manner?

  31. Mental Health • Given the target population being served, what are the • Prevailing mental health diagnoses? • Co-occurring disorders? (e.g., substance use) • Changes in mental health symptoms? • Disorders that may be more prevalent with HIV positive status? • Physiological HIV disease factors that contribute to mental health symptoms?

  32. Client Satisfaction • Questions relating to care received by clients. • What care is being received? • What is the frequency of care? • What is the level of client involvement in care? • What is the setting in which care is received? • Is the client satisfied with the care?

  33. Client Retention & Service Utilization • Client Retention • How many cases become active/inactive in a given time period? • What is the site definition for active/ inactive clients? • What are the site policies for case closing? What reasons are given for case closings? • What retention strategies have been effective with the target population?

  34. Client Retention & Service Utilization - continued. • Service Utilization • What is the type, frequency and duration of services used? • How are services provided? • Referrals in and out of program • Agency linkages - in-house and external - to other providers • Service setting – traditional/ non-traditional

  35. Quality of Life • How does the quality of life status and general medical health of a client impact upon • Treatment compliance (medications, ITP)? • Response to treatment? • Client retention? • Service utilization? • How does quality of life status and general medical health change over time with treatment?

  36. Methods & Indicators • CLIENT LEVEL INSTRUMENT • confidential – unique client id • Demographics • Social supports • Acculturation • Substance use and risk behavior • Trauma • Medical health • Medication adherence • Clinicians report

  37. Methods & Indicators • CLIENT SATISFACTION SURVEY • anonymous • Client characteristics • Service utilization • Care • Client involvement in care • Access/ barriers to care • Cultural competence in care • Overall satisfaction with care

  38. Methods & Indicators • FOCUS GROUPS • with clients on site • Focus groups will provide qualitative backdrop to analyze quantitative data from client satisfaction survey and client instrument • Types of services used • Satisfaction with services • Barriers to care • Cultural competency

  39. Methods & Indicators • SITE VISIT DATA COLLECTION • MHHSC Coordinating Center Staff • Continuum of services • Location of site • Geographic setting • Site community • Service setting (traditional / non traditional) • Client retention strategies • Site activities to overcome identified barriers to mental health care

  40. Utility of Cross-Site Analyses From the Sites’ Perspective • Site buy-in has been an on-going process • Clinicians and local evaluators participate in the evaluation design work group • They took ownership of the evaluation design • Made decisions on utility of collecting certain types of data across sites

  41. Client Focused Domain • Client Characteristics • Clients’ Presenting Diagnosis • Barriers/Access to Care (enhance/maintain Client’s Quality of Life/Health)

  42. Program Context Domains • Program Structure • CBO VS Large System of care • Staffing patterns (FT, PT) • Sustainability Efforts • Plan in place

  43. Client Focused Domains

  44. Client Focused Domains (cont.)

  45. Client Focused Domains (cont.)

  46. Program Context Domains

  47. Program Context Domains (cont.)

  48. Program Context Domains (cont.)

  49. Purposes/Uses of Data • Cross-site analysis • Attention to differences across sites, client characteristics • Careful & appropriate • Nuanced, not reductionist • Program context critical • Use of qualitative and program-level data (e.g., differences in resources/capacity available)

  50. Purposes/Uses of Data (Recap) • UTILIZATION-FOCUSED EVALUATION • Data/analysis to be provided to sites • User friendly feedback • A timely manner • For program improvement /development of better intervention strategies • NOT REPORT CARDS • Individual clinicians • Individual sites

More Related