LAYING THE FOUNDATION: RE-AIM OVERVIEW

1. LAYING THE FOUNDATION:RE-AIM OVERVIEW Russell E. Glasgow, Ph.D. Kaiser Permanente Colorado http://research-practice.org See also: www.re-aim.org

2. OVERVIEW • Challenges to Translating Research into Practice • Rationale for and Basics of RE-AIM • Common Questions and Issues in Applying RE-AIM • New Developments and Future Directions • Q and A

3. WHAT’S UNIQUE ABOUT TRANSLATIONAL RESEARCH? • Practical Questions: About costs, implementation feasibility • Contextual: Want to know - If works in particular settings, groups, etc. - Generalizability; how broadly applicable • Issues of Adaptation vs. Fidelity • Multiple perspectives

4. RECONCILING DIVERSE PERSPECTIVES—FINDING COMMON GROUND AMONG: • Researchers: Effect size, fidelity • Practitioners and Educators: Feasibility, local applicability • Policy Makers and Administrators Costs, breadth of application • Patients: Quality of life

5. Organization Clinic As Tested Program Delivery Staff Critical Elements Research Design Simplified Systems Model for Translational Research Delivery Site(s) Program or Policy Fit Appropriate For Question Partnership Broader Health Policy and Cultural Context Estabrooks PA, Glasgow RE. Am J Prev Med 2006;31(4S):S45-S56

6. “The law of halves” … a story

7. ULTIMATE IMPACT OF BREAKTHROUGH DEPRESSION PROGRAM Dissemination StepConcept% Impacted 50% of Clinical Settings Use Adoption 50% 50% of Clinicians Trained Adoption 25% 50% of Patients Accept Reach 12.5% 50% Receive Entire Program Implementation 6.2% 50% of Those Receiving Entire Program Remiss Effectiveness 3.2% 50% Continue In Remission after 12 Months Maintenance 1.6%

8. RE-AIM TO HELP PLAN, EVALUATE, AND REPORT STUDIES R Increase Reach E Increase Effectiveness A Increase Adoption I Increase Implementation M Increase Maintenance Glasgow, et al. Ann Behav Med 2004;27(1):3-12

9. REACHIndividual Level Definition:The number, percent of those invited and eligible who participate, and their representativeness. Example:Sixty-five percent of chronic illness patients invited to a group medical visit attended an initial session. Those declining more likely to be male and Latino. Key Issues:Does program reach those at highest risk? What is the “denominator” of intended audience? Are different recruitment strategies or program options needed?

10. EFFECTIVENESS Individual Level Definition:Amount of change in temporally appropriate outcomes and impact on quality of life or any adverse (iatrogenic) effects. Example:Telephone outreach program increased colon cancer screening rates by 20%, with no adverse effects on quality of life or cardiovascular screening rates. Key Issues:Logic model helps to clarify effects; QOL provides common metric for overall impact; important to anticipate unintended consequences.

11. ADOPTION Setting/Staff Level Definition:Number, percent and representativeness of settings and staff invited who participate. Example:(Setting Level): Fifty-two percent of county health departments invited to a training program on evidence-based decision making sent a representative. Departments participating were larger, more urban, and rated chronic illness as a higher priority. Key Issues:Need to focus on “denominator” invited and barriers among non-users. Do initial adoptees include peer opinion leaders?

12. IMPLEMENTATION Setting/Staff Level Definition:Extent to which a program or policy is deliveredconsistently, and the time and costs of the program. Example:Caregivers received at least two follow-up phone calls 75% of the time; took an average of 5 minutes, but not all staff called consistently. Key Issues:Consistency of delivery across staff, program components, and time. Balance between fidelity and local customization.

13. MAINTENANCE Individual Level Definition:Long-term effects on key outcomes and quality-of-life impact. Example:At one-year follow-up, was 58% attrition from Internet weight loss program; those present maintained 95% of an initial 8 lb. weight loss. Key Issues:Length of follow-up; does attrition bias results?

14. MAINTENANCE Setting Level Definition:Extent to which a program or policy is sustained, modified, or discontinued following initial trial or study period. Example:Of 24 settings initially participating, 6 continued the program largely unchanged, 12 made substantial adaptations, and 6 dropped the program. Key Issues:How is program adapted over time; funding strategies; qualitative approaches to understanding why these actions taken.

15. RE-AIM BUILDING BLOCKS THAT TOGETHER PRODUCE PUBLIC HEALTH IMPACT Adoption Building Programs and Policies with a Large Public Health Impact Efficacy Effectiveness Implementation Reach Maintenance

16. RECOMMENDED PURPOSE OF TRANSLATION/EFFECTIVENESS RESEARCH • Reach large numbers of people, especially those who can most benefit • Be widely adopted by different settings • Be consistently implemented by staff members with moderate levels of training and expertise • Produce replicable and long-lasting effects (and minimal negative impacts) at reasonable cost To determine the characteristics of interventions that can:

17. CHALLENGES IN APPLYING RE-AIM

18. CHALLENGES IN APPLYING RE-AIM(cont.)

19. CHALLENGES IN APPLYING RE-AIM(cont.)

20. NEW DEVELOPMENTS AND FUTURE DIRECTIONS • Program Planning and Quality Improvement • RE-AIM Summary Metrics • Combining Qualitative and Quantitative Approaches • Applications to Policies

21. PROGRAM PLANNING Applications: Primary care health behavior counseling; physical activity enhancement in older adults; state obesity programs. Lessons Learned: - RE-AIM helps to understand the trade-offs involved. - Define success up front and have interim measures. - Never too early to plan for maintenance and sustainability. - Adaptation of initial intervention often key to success.

22. WHICH PROGRAM IS BETTER? See www.re-aim.orgfor displays and evaluation questions Glasgow et al. Am J Prev Med 2006;30(1):67-73

23. NEW RE-AIM SUMMARY METRICS THAT ADDRESS: • Health disparities – e.g., who participates and who benefits • Costs and cost-effectiveness • Effects of different interventionists • Combining different factors to produce composite outcomes Glasgow et al. Using RE-AIM Metrics to Evaluate Diabetes…AJPM 2006;30(1):67-73 Glasgow et al. Health Education Research 2006;21(3):688-694

24. NEW RE-AIM METRICS • Individual Level Impact (RE) = Reach x Composite Effectiveness a) Reach = [Participation rate – Median ESdifferential characteristics] b) Composite Effectiveness = [Median ESkey outcomes - Median ESnegative outcomes/QOL – Median ESdifferential impact] Glasgow, et al. Evaluating the Overall Impact of Health Promotion Programs… Health Education Research, 2006;21(3):688-694.

25. COMBINING APPROACHES:WISEWOMAN PROGRAM • CDCP lifestyle program to prevent heart disease and improve health among low-income women. • Used “objective” RE-AIM measures to ID sites high and low on different dimensions. • Follow-up qualitative interviews, observations, and focus groups to understand how results came about. Besculides et al. Prev Chron Dis 2006;3:A07, Epub

26. POLICY LESSONS • REACHgenerally a strength of policies, but cannot assume universal penetration. • EFFECTIVENESS needs to be assessed broadly and different outcomes over time. • Important to assess unintended consequences. • Consistency of enforcement is critical.

27. THE “3 RS” OF INTEGRATING RESEARCH INTO PRACTICE • Representativeness (Reach, Adoption) • Robustness (Effectiveness across subgroups—especially re: disparities) Cronbach’s generalization across person, time, measures • Replicability (Implementation) in representative settings Cronbach LH, et al. The dependability of behavioral measurements: Theory of generalizability for scores and profiles. New York, John Wiley & Sons, 1972 Shadish WR, et al. Experimental and quasi-experimental design for generalized causal inference. Boston: Houghton Mifflin, 2002

28. RE-AIM BUILDING BLOCKS THAT TOGETHER PRODUCE PUBLIC HEALTH IMPACT Adoption Building Programs and Policies with a Large Public Health Impact Efficacy Effectiveness Implementation Reach Maintenance

29. QUESTIONS?