Improving Implementation of Evidence-Based Innovations for Substance Use

1. Improving Implementation of Evidence-Based Innovations for Substance Use Dr. Bryan R. Garner Senior Implementation Research Scientist RTI International

2. Learning Objectives • Overview of key implementation-related frameworks and models. • Research on implementing the Adolescent Community Reinforcement Approach (A-CRA). • Research on implementing a motivational interviewing-based brief intervention for substance use within HIV service settings.

3. Key implementation-related frameworks and models: Proctor et al (2009) Conceptual Model of Implementation Research

4. Key implementation-related frameworks and models: Aarons et al (2011) EPIS Framework

5. Key implementation-related frameworks and models: Curran et al (2012) Effectiveness-Implementation Hybrid Trial Framework

6. Implementing the Adolescent Community Reinforcement Approach (A-CRA): The Manual

7. Implementing the Adolescent Community Reinforcement Approach (A-CRA): Description of the dissemination & implementation initiative Godley, S. H., Garner, B. R., Smith, J. E., Meyers, R. J., & Godley, M. D. (2011). A Large-Scale Dissemination and Implementation Model for Evidence-Based Treatment and Continuing Care. Clinical Psychology: Science and Practice, 18, 67-83.

8. Implementing the Adolescent Community Reinforcement Approach (A-CRA): Visual of the dissemination & implementation initiative

9. Implementing the Adolescent Community Reinforcement Approach (A-CRA): Impetus for the Reinforcing Therapist Performance (RTP) Experiment Recommended Pay-for-Performance (P4P) as a promising method to improve the delivery of high quality care. “The literature evaluating the effectiveness of pay for performance consists of fewer than 20 studies, yielding mixed conclusions on overall impact.”

10. Implementing the Adolescent Community Reinforcement Approach (A-CRA): The Reinforcing Therapist Performance (RTP) Experiment Garner, B. R., Godley, S. H., Dennis, M. L., Godley, M. D., & Shepard, D. S. (2010). The Reinforcing Therapist Performance (RTP) experiment: Study protocol for a cluster randomized trial. Implementation Science, 5:5.

11. Implementing the Adolescent Community Reinforcement Approach (A-CRA): Overview of the RTP Experiment’s Aims, Design, and Targets

12. Implementing the Adolescent Community Reinforcement Approach (A-CRA): RTP Experiment’s Sample Size and Data Structure

13. Implementing the Adolescent Community Reinforcement Approach (A-CRA): The RTP Experiment’s Effectiveness Results Take Home Message: P4P directly improved treatment implementation and indirectly improved treatment effectiveness.

14. Implementing the Adolescent Community Reinforcement Approach (A-CRA): The RTP Experiment’s Effectiveness Results Publication Garner, B.R., Godley, S. H., Dennis, M. L., Hunter, B. D., Bair, C. M. L., & Godley, M. D. (2012). Using pay-for-performance to improve treatment implementation for adolescent substance use disorders: Results from a cluster randomized experiment. Archives of Pediatrics & Adolescent Medicine, 166, 938-944.

15. Implementing the Adolescent Community Reinforcement Approach (A-CRA): The RTP Experiment’s Cost-Effectiveness Results Publication Garner, B. R., Godley, Lwin, A. K., Strickler, G. K., Hunter. B. D., & Shepard, D. S. (2018). Pay-for-performance as a cost-effective implementation strategy: Results from a cluster randomized trial. Implementation Science, 13(92).

16. Implementing the Adolescent Community Reinforcement Approach (A-CRA): The RTP Experiment’s Cost-Effectiveness Results (Table 2)

17. Implementing the Adolescent Community Reinforcement Approach (A-CRA): The RTP Experiment’s Cost-Effectiveness Results (Table 3) Take Home Message: An average increase of 5% resulted in… A 116% increase in the average number of months therapists demonstrated competence in treatment delivery (ICER = $333). A 325% increase in the average number of patients who received the targeted dosage of treatment (ICER = $453). A 325% increase in the number of days of abstinence per patient in treatment(ICER = $8.134).

18. Implementing a motivational interviewing-based brief intervention for substance use within HIV service settings:The SAT2HIV Project

19. Implementing a motivational interviewing-based brief intervention for substance use within HIV service settings:The SAT2HIV Project’s MIBI Experiment

20. Implementing a motivational interviewing-based brief intervention for substance use within HIV service settings:Conceptual Overview of MIBI Experiment

21. Implementing a motivational interviewing-based brief intervention for substance use within HIV service settings:The SAT2HIV Project’s ISF Experiment

22. Implementing a motivational interviewing-based brief intervention for substance use within HIV service settings:Conceptual Overview of ISF Experiment

23. Implementing a motivational interviewing-based brief intervention for substance use within HIV service settings:ISF Experiment’s Flow Chart

24. Implementing a motivational interviewing-based brief intervention for substance use within HIV service settings:Powell et al (2012) compilation of strategies Powell, B. J., McMillen, J. C., Proctor, E. K., Carpenter, C. R., Griffey, R. T., Bunger, A. C., ... & York, J. L. (2012). A compilation of strategies for implementing clinical innovations in health and mental health. Medical care research and review, 69(2), 123-157

25. Implementing a motivational interviewing-based brief intervention for substance use within HIV service settings:Proctor et al (2013) recommendations Proctor, E. K., Powell, B. J., & McMillen, J. C. (2013). Implementation strategies: recommendations for specifying and reporting. Implementation Science, 8(1), 139 Name it Define it Specify it The actor The action Action target Temporality Dose Implementation outcome affected Justification

26. Implementing a motivational interviewing-based brief intervention for substance use within HIV service settings:The ATTC Strategy • Centralized technical assistance • Develop educational materials • Develop and organize quality monitoring systems • Develop tools for quality monitoring • Distribute educational materials • Conduct educational meetings • Make training dynamic • Audit & Provide feedback • Provide ongoing consultation • Create a learning collaborative

27. Implementing a motivational interviewing-based brief intervention for substance use within HIV service settings:The ISF Strategy • Use an improvement and implementation advisor • Develop tools for quality improvement • Organize implementation team meetings • Identify and prepare champions • Assess for readiness and identify barriers • Conduct local consensus discussions • Conduct cyclical small tests of change

28. Implementing a motivational interviewing-based brief intervention for substance use within HIV service settings:Focus of this presentation

29. The Implementation & Sustainment Facilitation (ISF) Experiment:Implementation Effectiveness as a primary outcome “Implementation effectiveness refers to the consistency and quality of targeted organizational members’ use of a specific innovation.” Klein & Sorra, 1996

30. The Implementation & Sustainment Facilitation (ISF) Experiment:Implementation Effectiveness from Klein, Conn, & Sorra (2001)

31. Implementing a motivational interviewing-based brief intervention for substance use within HIV service settings:Relevant methods for this presentation’s results Sample • 50 staff from 25 AIDS Service Organizations (ASOs; 2 staff per ASO). • 24 staff and 12 ASOs randomized to ATTC condition. • 26 staff from 13 ASOs randomized to ATTC+ISF condition. Outcome • Implementation Effectiveness – A continuous staff measure representing the sum of the standardized cumulative number of MIBIs delivered by BI staff (i.e., consistency) and the cumulative fidelity score of their delivery of the MIBI (i.e., quality). Analysis • Mplus was used to conduct an adjusted multilevel regression analysis focused on the relationship between condition assignment (ATTC = 0; ATTC+ISF = 1) and implementation effectiveness.

32. Implementing a motivational interviewing-based brief intervention for substance use within HIV service settings:Hypothesized Model

33. Implementing a motivational interviewing-based brief intervention for substance use within HIV service settings:Results of Hypothesized Model

34. Implementing a motivational interviewing-based brief intervention for substance use within HIV service settings:Key Takeaways from current results • Preliminary results provide some evidence supporting the effectiveness of the Implementation & Sustainment Facilitation (ISF) intervention as an adjunct to the ATTC strategy, at least for implementation effectiveness. • The SAT2HIV Project is still ongoing, with the ISF Experiment scheduled to be complete in the Fall 2018. • Should the ISF intervention’s effectiveness be supported, future research will need to examine the extent to which our findings can be replicated, improved upon, and generalized to other contexts and EBPs.

35. Thank you! Bryan R. Garner, PhD Senior Implementation Research Scientist RTI International 3040 E. Cornwallis Rd. Research Triangle Park, NC 27709-2194 Phone: (919) 597-5159 Email: bgarner@rti.org