Challenges in Implementing Evidence-Based Programs in Routine Care Settings

1. Studying Effectiveness and Implementation of Evidence-Based, Research-Developed Programs in Routine Care Settings: Tradeoffs and Challenges in Study Designs and MethodsAlison B. Hamilton, Ph.D., M.P.H.Brian Mittman, Ph.D.Gail E. Wyatt, Ph.D. UCLA C-MORE Seminar Series May 21, 2013

2. Eban II Research Team UCLA Gail E. Wyatt, Ph.D., Principal Investigator Alison B. Hamilton, Ph.D., M.P.H., Co-Investigator John K. Williams, M.D., Co-Investigator Hector F. Myers, Ph.D., Co-Investigator Honghu Liu, Ph.D., Co-Investigator Alicia Eccles, M.P.H., Southern CA Project Coordinator Louise Datu, Administrative Analyst Alameda County, Department of Public Health Ron Kabir Hypolite, J.D., Co-Principal Investigator Craig Hutchinson, M.P.H., Northern CA Project Coordinator Consultants Brian Mittman, Ph.D., Implementation Expert, Veterans Administration Thomas Coates, Ph.D., Statewide Sustainability Consultant, UCLA Nabila El-Bassel, D.S.W., Dissemination Consultant, Columbia University, NY David Holtgrave, Ph.D., Cost Effectiveness Expert, John Hopkins Expert C. Hendricks Brown, Ph.D., Statistician, University of Miami Nan Laird, Ph.D., Biostatistician, Harvard University

3. Background • “Implementation imperative” • pressure/need for research to facilitate implementation (and hence greater societal benefit) of research-developed practices • Multiple challenges to uptake of research-developed practices • barriers to implementation • limited interest among researchers and research funders • limited guidance for study design and methods • limitations in the practices themselves (e.g., efficacy vs. effectiveness evidence)

4. Objective of Today's Presentation • To provide an example of a NIMH-funded “hybrid” effectiveness/implementation study to illustrate challenges in study design, methods and conduct, including • sampling, power and outcome selection • measurement • complexities of community-based research in current fiscal and policy environment (e.g., recruitment, retention, partnership)

5. Clinical Background: Epidemiology • HIV/AIDS epidemic disproportionately impacts African American communities • Rates of new infections among African Americans are seven times higher than among Whites • High rates of sexually transmitted infections among African Americans • HIV infection rates in California rank 3rd in the U.S. • Los Angeles and Alameda Counties have the largest concentrations of African Americans and highest proportion of African American HIV/AIDS cases • City of Oakland has the highest and most rapidly growing incidence rate of diagnosed and undiagnosed HIV in the country

6. Clinical Background: Interventions • Few couples-based interventions focused on reducing risky sexual behaviors, increasing condom use, reducing STI/HIV transmission, • Interventions have not focused specifically on heterosexual African Americans and their disproportionate HIV risk • Uneven sustainability of evidence-based practices in CBOs • NIMH-funded Eban (Yoruba for “fence”) risk reduction intervention designed to fill gap • Guided by social cognitive and culturally-grounded theories • Focused on building risk-reduction skills • RCT with 535 couples demonstrated efficacy in reducing rates of unprotected sex & and increased rates of condom use at post-test, 6- and 12-month follow-ups

7. Eban Intervention • 8-week, standardized, manualized intervention facilitated by a male-female team • Components: • strategies that address individual, interpersonal, social, and cultural factors influencing risk behavior • dyadic and group processes that take advantage of relationship and group dynamics • educational and culturally appropriate sessions • discussions of the realities of urban African American couples and focus on positive self-evaluations; cultural, gender, and ethnic pride; and risk avoidance • culturally congruent video clips, games, brainstorming, role-playing, and skill-building activities, and small group discussions

8. Moving Eban into Practice (Eban II) R01 MH093230 • Given established efficacy, appropriate to move Eban into practice in community-based organizations (CBOs) • Shift requires reciprocal, multidirectional information and technology exchange between the research team and the collaborating CBOs • Long-term goal to facilitate large-scale implementation of Eban II in CBOs that serve HIV-positive and at-risk African Americans • need to understand barriers and facilitators to adoption and implementation of Eban with high fidelity • need to obtain preliminary evidence of the effectiveness of specific strategies to facilitate adoption

9. Our goal: To reduce HIV and STI transmission among African American HIV sero-discordant couples.

10. "Hybrid" Study Design • Goal: clinically meaningful, not just statistically significant, evidence-based practice • Collect effectiveness data • Understand and overcome implementation barriers • Understand black box of implementation • Develop a sustainable intervention and implementation strategy • Challenges: • Rapid timeline • Real-time focus on potential and actual influences on the progress and effectiveness of implementation efforts • Activities during the study to refine implementation efforts, resolve mutable barriers, and enhance available facilitators From Curran et al.

11. Types of Hybrid Designs Clinical Effectiveness Research Implementation Research Eban II Hybrid Type I Hybrid Type II Hybrid Type III Hybrid Type I: test clinical intervention, observe/gather information on implementation Hybrid Type II: test clinical intervention, study implementation intervention Hybrid Type III: test implementation intervention, observe/gather information on clinical intervention and outcomes From Curran et al.

12. Eban II Implementation Aims • To facilitate implementation of Eban II in 10 CBOs in California; specifically, to employ a theory-guided strategy to partner with the CBOs to expose providers to the intervention, facilitate its adoption and delivery with high fidelity, and sustain its use for nine months following the active implementation phase. • Using mixed quantitative and qualitative methods, to document the implementation process and identify barriers and facilitators to adoption, fidelity, and sustainability.

13. Effectiveness Aim • To evaluate the effect of Eban II on the following behavioral and biological outcomes among 180 couples: • incidents of protected sex • proportion of condom use • incident sexually transmitted infections (syphilis, gonorrhea & Chlamydia) • Secondary Aim • To determine the cost-effectiveness of implementation of the Eban II intervention based on implementation costs and potential cost savings.

14. Design Overview • Protocol-based implementation approach • Mixed methods process evaluation • Conceptual guidance from the TCU Program Change Model (Simpson et al.) • model of phased organizational change from exposure to adoption, implementation, and sustainability

15. Design Overview: Implementation Approach • Multiple strategies and tools

16. Design Overview: Effectiveness • Effectiveness assessed using a randomized delayed enrollment (waitlist) control design to evaluate the impact of treatment on outcomes at posttest and 3-month follow-up

17. Implementation Aim: Mixed Methods Evaluation • Staff Survey • Web-based, individualized link to SurveyMonkey • Completed by staff (target n=200) who provide direct client care • One time only, at baseline; approx 30 min to complete • Assesses • Organizational climate • Attitudes toward evidence-based practices • Burnout • Familiarity with treatment of couples • Helps to understand organizational settings where Eban II will be delivered to couples • Semi-Structured Interviews • Pre- & post-implementation, and post-sustainability • Key stakeholders (n=50) at participating CBOs

18. Couples screened as eligible if they: • Self-identify as heterosexual • One partner is HIV-positive, the other is HIV-negative & they know each other’s status • At least one partner identifies as African American • Age 18 – 60 • Have been a couple for at least 3 months & intend to stay together • Had unprotected sex within the last 3 months • No plans to relocate beyond a reasonable distance • Willing to complete the study even if relationship ends Effectiveness Aim: Eligibility Criteria

19. Effectiveness Aim: Sustainability • Sustainability phase begins after the active implementation phase is completed • Reliance on grant funds ends and sites will be encouraged to integrate Eban II into their usual services • Pre-sustainability workshop will be provided • Technical assistance (including quality assurance) will be provided • retraining in the intervention • sharing resources • offering suggestions on lessons learned • review of session tapes to assess fidelity • Sustainability=two eight-week cycles of the intervention with 3-5 couples in each cycle, fidelity to the intervention core elements • Pre-post couples-level measures will also be collected in order to examine outcomes • Post-sustainability qualitative interviews with key stakeholders

20. Key milestones met to date • Held project kick-offs to generate enthusiasm, foster buy-in • Trained ~25 staff members as facilitators and site coordinators • Collected org survey from 88 staff members across 9 agencies (64% response rate as of 5/21/13) • Initiated project at first 2 agencies (one in Oakland, one in Los Angeles) • Held first State of California Implementation Network call in April

21. Key challenges of design/methods • Sampling, randomization, unit of analysis • organization/site for implementation; couple for effectiveness • power limitations • Balancing fidelity vs. adaptation to achieve feasibility • Measurement burden: effectiveness outcomes, couple characteristics and adherence, staff characteristics, agency characteristics

22. Key challenges in study conduct • Addressing barriers at agency level • Staffing limitations • Funding limitations • Time limitations (burden) • Phasing in agencies rather than starting all at same time • Maintaining enthusiasm at non-active agencies • Fostering spirit of collaboration in competitive economic times

23. Next Steps: Eban II • Submit protocol manuscript to Implementation Science • Continue process evaluation at active sites • Phase in next pair of sites in Year 2 (July 2013) • Analyze organizational readiness data

24. Next Steps: Supplementary to Eban II • Pursue mechanisms for dissemination of intervention • Seek support (from CTSI?) for packaging/branding of Eban intervention materials (e.g., training videos)  • Talk & Listen • Pursue funding for additional clinical outcomes of interest

25. For Discussion Today • Implications for: • Researchers: how to design and obtain funding for this type of work; how to properly prepare for it • Research institutions, programs, centers, academic leaders • Funding agencies • Other policy, practice leaders and entities

26. For More Information, Contact: Gail E. Wyatt, PhD (PI) email: gwyatt@mednet.ucla.edu phone: 310-825-0193