Using RE-AIM as a tool for Program Evaluation

1. Using RE-AIM as a tool for Program Evaluation From Research to Practice

2. What is RE-AIM • RE-AIM is an acronym that consists of five elements, or dimensions, that relate health behavior interventions: • Reachthe target population • Efficacy or effectiveness • Adoption by target settings or institutions • Implementation- consistency of delivery of intervention • Maintenanceof intervention effects in individuals and populations over time • Commonly used for evaluating efficacy trials (Phase III Research)

3. Defining Elements Reach — The absolute number, proportion, and representativeness of individuals who participate in a given program. Representativeness refers to whether participants have characteristics that reflect the target population's characteristics. For example, if your intent is to increase physical activity in sedentary people between the ages of 35 and 70, you wouldn't test your program on triathletes.

4. Defining Elements Efficacy/Effectiveness — The impact of an intervention on important outcomes. This includes potential negative effects, quality of life, and costs.

5. Defining Elements Adoption — The absolute number, proportion, and representativeness of settings and staff who are willing to offer a program.

6. Defining Elements Implementation — At the setting level, implementation refers to how closely staff members follow the program that the developers provide. This includes consistency of delivery as intended and the time and cost of the program.

7. Defining Elements Maintenance — The extent to which a program or policy becomes part of the routine organizational practices and policies. Within the RE-AIM framework, maintenance also applies at the individual level. At the individual level, maintenance refers to the long-term effects of a program on outcomes after 6 or more months after the most recent intervention contact.

8. How do elements relate to planning? • As you design, plan, or evaluate a health behavior intervention, there are questions that you should ask yourself. • Reach: HOW DO I REACH THE TARGETED POPULATION FOR INTERVENTION? • Efficacy or effectiveness: HOW DO I KNOW THAT MY INTERVENTION IS EFFECTIVE? • Adoption HOW DO I DEVELOP ORGANIZATIONAL SUPPORT TO DELIVERY THE INTERVENTION?

9. How do elements relate to planning? • As you design, plan, or evaluate a health behavior intervention, there are questions that you should ask yourself. • Implementation: HOW DO I ENSURE THE INTERVENTION IS DELIVERED PROPRLY? • Maintenance: HOW DO I INCOPROATE THE INTERVENTION SO IT IS DELIVERED OVER THE LONG TERM?

10. How is RE-AIM different from other evaluation approaches? RE-AIM draws upon previous work in several areas including: Diffusion of innovations, multi-level models, and Precede-Proceed. The primary ways that it is different is that it (a) is intended specifically to facilitate translation of research to practice, (b) it places equal emphasis on internal and external validity issues and emphasizes representativeness, and (c) it provides specific and standard ways of measuring key factors involved in evaluating potential for public health impact and widespread application.

11. Is RE-AIM used to design programs, or just to evaluate them? • It is both. Although used more commonly at present to report results or compare interventions, it is also intended as a planning tool.

12. An Example • Physical Activity Promotion in Primary Care: Bridging the Gap Between Research and Practice • Eakin, Brown, Marshall et al. (2004).

13. An Example of Application Translating and Disseminating Evidence-based Falls Prevention Programs into Community Li et al. (2008). American Journal of Public Health

14. Primary Aim: (a) To translate an evidence-based Tai Chi exercise fall intervention into a community-based program for implementation with older adults; and (b) Using the RE-AIM framework (Glasgow et al., 1999), to pilot implement the program with a primary focus on reach, uptake (adoption), and implementation.

15. Secondary Aim: • To evaluate the effectiveness of the program with respect to improvements in physical performance measures germane to falling. • Tertiary Aim: • To evaluate program maintenance with respect to the extent to which older adults would continue to practice Tai Chi beyond the period of the initial evaluation.

16. Translation • Translate an evidence-based Tai Chi exercise fall intervention into a community-based program for implementation with older adults • Identify training objectives and elements • Identify end users and dissemination partners • Develop a dissemination package • Expert evaluation • Pilot testing

17. Program Evaluation Reach: A total of individuals/providers responded to the program promotion Effectiveness: defined as change in physical performance outcome measures taken at baseline and again at 12 weeks termination. Adoption (or uptake): defined as the proportion of local community (senior activity) providers that agreed to participate and implement the program.

18. Implementation: defined as the extent to which providers' implemented a Tai Chi class to participating older adults and the ability to conduct the various elements of program protocols, including the use of implementation plan, a 2-times weekly program schedule, distribution of program supplements (i.e., videotape and a guidebook), a class attendance rate of 75% or better over the 12-week class period.

19. Maintenance: defined at both the service provider level and participant level. At the provider’s level, it was defined as providers’ willingness to consider the program to be part of routine organizational provisions (assuming adequate financial resources). At the participant level, it was defined as the extent to which improvements in participants’ physical performance were sustained, and their continued practice of Tai Chi, 8-weeks after completion of the class.

20. Dissemination Outcome REACH: 87% (by study criteria); 45% by client attendance ADOPTION: Six senior activity centers from five communities: 100% adoption IMPLEMENTATION: 75% completed; >85% class attendance; average 32 min. of home practice

21. Dissemination Outcome • Effectiveness: Improved physical • performance and quality-of life measures • Functional Reach; (b) Up and Go; • (c) Chair Stands, (d) 50-foot speed walk, • and SF-12. Maintenances: Five centers continued; 87% participants continued Impact Adopted by the State of Oregon: being implemented in four counties; three more starting this year

26. Conclusion-Tai Chi The evidence-based Tai Chi program is practical to disseminate and can be effectively implemented and maintained in community settings