The Promise and Challenge of Dissemination and Implementation Science (DIS)

1. The Promise and Challenge of Dissemination and Implementation Science (DIS) Russell E. Glasgow, PhD Director, ACCORDS DISSEMINATION AND IMPLEMETATION SCIENCE Program University of Colorado School of Medicine https://goo.gl/fTUyrM Acknowledgments: Ross Brownson, David Chambers, PAUL ESTABROOKS, BRYAN FORD, Amy Kilbourne, Borsika Rabin and the great researchers cited throughout this talk

2. Everything I know (about DIS) Dissemination and Implementation Science (DIS) is about: • Multi-level, contextual issues, and external validity • Relevant, pragmatic models, research methods and measures • Real world implementation and adaptation • Designing for dissemination, sustainability and equity (Normal science (T1– T2) is necessary but not sufficient)

3. Objectives • Why do we need DIS? How can DIS inform your work? • What might you expect if you engage in DIS? ....better outcomes: better fit to local context, better uptake, etc. • Current and Evolving Issues in DIS • Overview some current DIS research areas and future opportunities

4. Balas & Boren, 2000 Original research 18% variable Negative results Dickersin, 1987 Submission 46% 0.5 year Kumar, 1992 Koren, 1989 Acceptance Negative results 0.6 year Kumar, 1992 Publication Expert opinion 35% 0.3 year Poyer, 1982 Balas, 1995 Lack of numbers Bibliographic databases Reviews, guidelines, textbook 9.3 years Inconsistent indexing Implementation “PUBLICATION PATHWAY”

5. Balas & Boren, 2000 Original research 18% variable Negative results Dickersin, 1987 Submission 46% 0.5 year Kumar, 1992 Koren, 1989 Acceptance It takes 17 years to transform 14% of original research into the benefit of patient care Negative results 0.6 year Kumar, 1992 Publication Expert opinion 35% 0.3 year Poyer, 1982 Balas, 1995 Lack of numbers Bibliographic databases Reviews, guidelines, textbook 9.3 years Inconsistent indexing Implementation “PUBLICATION PATHWAY”

6. Need for Pragmatic DIS Research • Traditional biomedical RCTs study the effectiveness of treatments delivered to carefully selected populations under ideal conditions • Even when we do implement a tested intervention into everyday clinical practice, we often see a “voltage drop”…a dramatic decrease in effectiveness • Most common reason evidence- based programs are not adopted…they are not seen as relevant “If we want more evidence-based practice, we need more practice-based evidence.” Green LW Am J Pub Health 2006 Rothwell PM. External validity of randomised controlled trials. Lancet 2005;365:82-93.

7. Key Differences Between Traditional RCTs and Pragmatic DIS Controlled Trials (PCTs)

8. T1-T4 ResearchWhich type (T) is most common?

9. Bench to Bookshelf

10. Some Remedies and Key Terms • Implementation science is the study of methods to promote the integration of research findings and evidence into healthcare policy and practice. • Dissemination researchis the scientific study of targeted distribution of information and intervention materials to a specific public health or clinical practice audience. The intent is to understand how best to spread and sustain knowledge and the associated evidence-based interventions. • Implementation researchis the scientific study of the use of strategies to adopt and integrate evidence-based health interventions into clinical and community settings to improve patient outcomes and benefit population health. • Pragmatic research is the use of real-world tests in real-world populations and situations. *Brownson, Colditz & Proctor. Dissemination and Implementation Research in Health 2018

11. A Big Tent of Terms (and ovals) Population Health (and Community) Services Health Services Research Implementation Science Implementation Research Dissemination Research Pragmatic Research Health Communication Research Quality Improvement Science QI . Adapted from Mitchell S, Chambers, D. https://doi.org/10.1200/JOP. 2017.024729.

12. Key Characteristics of D&I Science Glasgow RE, Chambers D. Developing robust, sustainable, implementation systems using rigorous, rapid and relevant science. ClinTransl Sci. 2012;5(1):48

13. Too often, we have assumed, “If you build it…and if you have evidence”

14. An Evidence-Based Cancer Prevention... or Weight Loss... or Mental Health.....or (fill in blank) Story Even if 100% effective...is only so good as how and whether: • it is adopted • practitioners are trained to deliver it • trained practitioners choose to deliver it • eligible populations receive it • it can be sustained If we assume 50% threshold for each step… (even with perfect access/adherence/dosage/maintenance) Impact: .5x .5x .5x .5 x .5 = 3% benefit www.Re-aim.org Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: The RE-AIM framework. Am J Public Health. 1999;89(9):1322.

15. Pragmatic DIS Use of RE-AIM Glasgow R & Estabrooks P. Pragmatic applications of RE-AIM for health care initiatives in community and clinical settings. Preventing Chronic Disease. 2018; 15(1): E02.

16. The 5 Rs to Enhance Pragmatism, DIS and Likelihood of Translation Research that is: • Relevant • Rapid and Recursive • Redefines Rigor • Reports Resources Required • Replicable Peek, C.J, et al. (2014). The 5 Rs: An Emerging Bold Standard for Conducting Relevant Research in a Changing World. Annals Of Family Medicine, 12(5), 447-55. doi:10.1370/afm.1688 deGruy, F.V, et al. (2015). A plan for useful and timely family medicine and primary care research. Family Medicine, 47(8), 636-42.

17. Reporting Resources Required • Reporting on cost and other resources in a standardized manner is useful in: • Demonstrating value • Promoting rigor, transparency and relevance to stakeholders • Present from perspective of stakeholders and decision makers • Simple is fine – sophisticated economic analyses are not needed • Report costs of conducting or replicating interventions • Beyond money, costs can include clinician and staff time, training, infrastructure, startup costs, opportunity costs Ritzwoller, D P, et al. (2009). Costing behavioral interventions: A practical guide to enhance translation. Annals of Behavioral Medicine, 37(2), 218-27.

18. How well do research and practice connect? Dissemination

19. Dissemination Disconnects

20. Dissemination: What we know • Dissemination generally does not occur spontaneously and naturally; • Passive approaches to dissemination (diffusion) are usually ineffective; • Single-source prevention messages are generally less effective than comprehensive, multi-level approaches; • Stakeholder involvement in the research or evaluation process is likely to enhance dissemination; • Theory and frameworks for dissemination are beneficial; and • The process of dissemination needs to be tailored to various audiences Brownson RC, et al. Designing for dissemination among public health researchers...... Am J Public Health. 2013;103(9):1693-1699.

21. DIS Models + Frameworks are available! • Outlined in Tabak, et al., proliferation of models: 61 reviewed, now more than 150!! • Context is critical • Focus on external validity • Begin with stakeholders—take their perspective(s) • Find balance between fidelity to EB program and adaptation to local setting • Unlikely you need to create a new model www.dissemination-implementation.org website by Rabin et al. Tabak RG, et al. Bridging research and practice. Am J Prev Med. Sep 2012;43(3):337-350 Mitchell S & Chambers, D. Leveraging Implementation Science...J Oncology Practice, 2017.024729

22. http://www.dissemination-implementation.org/ Mode

23. Types of Outcomes in DISResearch (Proctor, et al., 2010) Proctor E, Silmere H, Hensley M, et al. Outcomes for implementation research: Administration and Policy in Mental Health [serial online]. March 2011;38(2):65-76.

24. Key Outcomes Related to Health Equity • Penetration (Reach)...It is ‘about the denominator’ • Representativeness- levels of community, setting, staff, patient • Costs and Feasibility- for setting; staff; and patients/citizens/users • Other

25. Evidence-based...but on what? Sorely needed on external validity and pragmatic criteria…often ignored, including: • ParticipantRepresentativeness • SettingRepresentativeness • Contextand Setting • Community/Setting Engagement • Adaptation/Change • Sustainability • Costs/Feasibilityof Treatment • Comparison Conditions

26. Reporting for D&I vs. Usual HSR

27. Questions? “I am all ears!”

28. Future Opportunities Examples of how DIS Research can help addresscurrent and future prevention, healthcare, health equity and population health

29. Example One: Learning Healthcare Systems (and Communities) • Big Data, e-Health/m-Health, geo-coding • Consumer-driven Healthcare policy reform • Exchanges/ACOs • Medicaid expansion • Learning Healthcare Systems • Aligning science with clinical priority goals • Emphasis on costs and value • Conducting more rapid and efficient studies • Leveraging existing data to deploy and evaluate innovations and best practices

30. The Long Road to Learning Healthcare Systems and Communities (and why we need DIS and pragmatic application) • New research takes too long for adoption (if at all) • Research is often not aligned to address critical health and health care problems or feasible • Research is often not designed with stakeholders • Providers lack practical tools/technical assistance and strategies to implement EB treatments (data are not enough) • Large programs are often rolled out without adequate planning to maximize effectiveness, learning and sustainability • Variationand patient-centered care • Treatments work differently for different people and different settings

31. Example Two: Precision Medicine (Health) • How does clinical practice incorporate PMI findings? • How do you implement evidence that will evolve? • What does it cost and what are opportunity costs? • How do you train and support the workforce? • How do you ensure that PM findings reduce, not exacerbate, health inequities?

32. Convergence of Precision Health, DIS & Learning Health Care Systems and Communities Chambers DA, Feero WG, Khoury MJ. Conversion of implementation science, precision medicine, and the learning health care system. JAMA. 2016, 315: 1941-1942.

33. RE-AIM Precision Medicine (PM- or Precision Health) Questions Determine • What percent and types of patients are Reached (equity); • For whom among them is the PM intervention Effective, in improving what outcomes, with what unanticipated consequences (is health equity improved or decreased )? • In what percent and types of settings is this approach Adopted; • How consistently are different PMImplementedat what cost to different parties; • And how well are the intervention components and their effects Maintained at both setting and individual levels? • Gaglio B, Glasgow RE. Evaluation approaches…In: Brownson R, Colditz G, Proctor E, (Eds). Dissemination and implementation research in health. New York: Oxford University Press; 2018. 317-334.

34. Current DIS Funding Opportunities NIH: PAR-18-017; 18-007; (18 ICs & Offices) Purpose: To support innovative approaches to identifying strategies for the adoption, adaptation, integration, scale-up and sustainability ofevidence-based interventions, tools, policies, and guidelines. Also benefit in studying how to “de-implement”or reduce the use of strategies and interventions that are not evidence-based, yield sub-optimal benefits for patients, and are harmful or wasteful. PCORI: Dissemination and Communication and Large Pragmatic Trials NHLBI: Center on Translational Research and Implementation Science (training and research); upcoming discussion of new RFA

35. Key Science Questions- New and Old • Traditional science and evidence question: (necessary but not sufficient): “What intervention produces the largest average effect in tightly controlled trials on the major (clinical) outcome? • Pragmatic DIS question: (contextual) “What program/policy components are most effective for producing what outcomes for which populations/recipients when implemented by what type of persons using what strategies under what conditions, with how many resources and how/whydo these results occur?”

36. Everything I know (about DIS) Dissemination and Implementation Science (DIS) is about: • Multi-level, contextual issues, and external validity • Relevant, pragmatic models, research methods and measures • Real world implementation and adaptation • Designing for dissemination, sustainability and equity (Normal science (T1– T2) is necessary but not sufficient)

38. russell.glasgow@ucdenver.edu University of Colorado School of Medicine ACCORDS D&I Program

39. Future Opportunities (and biased speculations) • *Getting serious about CONTEXT: Measures; designs; international collab. • *Real integration of health policy, public health, and biomedicine; clinical and community; impact on policy • Focus on social determinants of health, health equity, generalizability of results to low resource settings, individuals and LMIC (and vice versa) • Focus on COSTS, resources, de-implementation, and comparative effectiveness research • Precision medicine, big data, AI and learning health systems

40. Expanded CONSORT Figure Existing CONSORT Figures for RCTs focus almost exclusively on internal validity Use of CONSORT- required by most medical journals, has improved quality and reporting on internal validity Until now, no comparable resource has addressed external validity. URL: https://goo.gl/8X2gzL Glasgow, Huebschmann, Brownson. Amer J Prev Med, 2018, DOI: https://doi.org/10.1016/j.amepre.2018.04.044

41. General Resources • Brownson, RC, Colditz, GA, & Proctor, EK (2018). Dissemination and implementation research in health: translating science to practice. Oxford University Press. • re-aim.org • https://rtips.cancer.gov/rtips/index.do • www.ucdenver.edu/accords/implementation • www.Dissemination-Implementation.org

42. ACCORDS Dissemination and Implementation Science Program, University of Colorado Denver School of Medicine Our goal is to: Provide local consultation on D&I related research to increase funding and publication success Create collaborative learning partnerships with embedded research settings to translate research into practice more quickly and successfully Conduct cutting edge T3-T4 research on: pragmatic research and measures, adaptation of interventions, designing for dissemination, shared decision making, planning for and evaluation of reach, implementation and sustainability Use interactive on-line resources and support for patients, medical and public health students, trainees and faculty researchers Communicate the latest information on DIS related conferences, articles, grant opportunities, events, webinars, talks, and training opportunities www.ucdenver.edu/accords/implementation

44. T1 – T4 Extra