RE-AIM for Public Health Impact

1. RE-AIM for Public Health Impact Erika Kirby, MBA, RD, South Carolina DHEC Janice Sommers, MPH, UNC Center for Health Promotion and Disease Prevention ASTPHND, Madison WI June 10, 2008

2. RE-AIM REACH EFFECTIVENESS ADOPTION IMPLEMENTATION MAINTENANCE The purpose of this presentation is to provide an overview and an sample application of the RE-AIM framework for planning and evaluating health promotion interventions at multiple levels of the SE model

3. Our Plan for Today…. Janice provides the framework for thinking about public health impact Erika provides the example of how South Carolina used the framework in practice I know how to re-aim for public health impact !

4. Define the five key dimensions of re-aim Identify factors that contribute to the public health impact of an intervention Understand how to apply re-aim: to plan or develop new interventions, to adapt existing interventions to enhance their impact, and to assess the potential impact of interventions on the publics’ health RE-AIM Objectives

5. Expanded Color Me Healthy…The South Carolina Experience

6. SC Pilot Intervention Goal: Promote healthy eating and activity in young children • Add components to CMH in SC • Test feasibility of the components • Look at outcomes from expansion • Make decisions about how to approach wider dissemination

7. SC Pilot Intervention • Background on Expanded CMH • Three-pronged approach • Individual: Child and Parent • Interpersonal: Parent and Teachers • Organizational: Child Care Center

8. SC Pilot and the Socioecologic Model • Individual: • Color Me Healthy • Activity Bags • Interpersonal: • Activity Bags • Cooking with a Chef • Open House • Families Eating Smart Moving More • Organizational: • Open House • Nutrition and Physical Activity Self-Assessment for Child Care Centers (NAP SACC)

9. RE-AIM for Public Health Impact

10. Are we making an impact or howling at the moon?

11. Striking a Balance Between.. • What works (is effective) • Who we intend to reach • What is reasonable and feasible to implement and maintain given available resources

12. How can RE-AIM help? Reach large numbers of people, especially those who can most benefit Be widely adopted by different settings Be consistently implemented Produce replicable and long-lasting effects and be maintained at reasonable cost

13. RE-AIM DIMENSIONS AND DEFINITIONS

14. RE-AIM DIMENSIONS AND DEFINITIONS

15. RE-AIM DIMENSIONS AND DEFINITIONS (cont.)

16. RE-AIM DIMENSIONS AND DEFINITIONS (cont.)

17. RE-AIM DIMENSIONS AND DEFINITIONS (cont.)

18. RE-AIM and Socioecologic Model • Behavior Change • Individual • Interpersonal • Environmental Change • Organizational • Community • Policy Change • Organizational • Societal

19. How can we use re-aim in practice? Developing a new intervention Adapting an existing intervention Choosing between alternative interventions Assessing an intervention as part of quality improvement Framing questions for evaluation purposes

20. Application of RE-AIM to Health Policy • When planning and writing new policies • When comparing policy alternatives • Considering policies relative to other alternatives • Evaluating the impact of specific policies • Reviewing the literature on policy impact

21. RA-EIM Reach and Adoption work together Effectiveness depends on Implementation Maintenance happens last

22. REACH: Who is intended to benefit from the intervention? Number or percent of those whose health is to be improved as a result of the policy /intervention Representativeness of those affected

23. REACH: Questions to consider What group is affected by or benefits from the policy/intervention? Are they representative of the population? Are the individuals most at risk the ones that are affected?

24. Calculating Reach: Identifying the Denominator How many people are eligible to participate? Age? Gender? Location? How many people would ideally be impacted by a policy? All citizens? All students? All employees?

25. Calculating Reach: Identifying the Numerator How many people actually participate in the intervention? Segment of the eligible? All eligible? How many people are actually affected by the policy? Everyone in the setting? Select group in the setting?

26. South Carolina Example

27. Assessing Representation Representativeness of those who participate Characteristics of participants vs. general population in that area Characteristics of participants vs. those declining Characteristics of those who benefit from the policy vs those who do not

28. South Carolina Example • How do the demographic characteristics of children in SC compare to the characteristics of the children exposed to Expanded Color Me Healthy? • Middle-income • Suburban

29. ADOPTION: Where are the people you want to reach? Number or percent of organizations or governing bodies that enact a policy or an intervention Representativeness of organizations or governing bodies

30. ADOPTION: Questions to consider • How many organizations are eligible to participate? (denominator) • How many actually participate? (numerator) • Are there differences between those that adopt the policy or intervention versus those that do not?

31. ADOPTION: Representativeness How representative are the organizations that participated compared to other organizations of the same type? Is the size, capacity, or type similar or different? Do they have previous experience with this type of policy or intervention? Are the demographics of the people within the organization comparable?

32. South Carolina Example • Two faith-based child care settings • Suburban and middle-income • Half-day programs • No meals, snacks only

33. How Reach and Adoption Contribute To Each Other • Many settings adopt the intervention or policy but they have few people in them • Few settings adopt the policy or intervention but they are highly populated More settings  Greater reach

34. EFFECTIVENESS: Change in the temporally appropriate outcome(s). Effects on primary outcome(s) of interest Assessment of unintended and adverse negative outcomes Impact on Quality of Life LOGIC MODEL HELPS

35. EFFECTIVENESS: Questions to consider • How will you know if your intervention achieved the outcomes you intended? • Are the outcomes consistent across different sub-groups of the population? • Are there any unanticipated consequences? • How confident are you that the benefits outweigh any adverse consequences?

36. Evaluation of Effectiveness Plan for evaluation upfront Identify the outcomes of interest Consider multiple outcomes Use valid and reliable measures of change Measure participation in the intervention Track implementation of the intervention

37. IMPLEMENTATION: Applying the policy as planned or implementing the intervention as intended Time and costs of intervention

38. IMPLEMENTATION: For policies: • Applying the policy as planned • Adequately enforcing the policy • Ensuring ongoing and consistent compliance

39. IMPLEMENTATION: Questions to consider Which elements of the intervention or policy are most critical? Design process measures for each core element Designing an evaluation that does not describe implementation means that you won’t really know what you are evaluating.

40. IMPLEMENTATION: Questions to consider • What are the costs of implementing the intervention/policy? • Staff time; training; materials etc • How complex is implementation?

41. Relationship of Effectiveness and Implementation Effectiveness depends on the fidelity of implementation: ALL of the core elements must be implemented in order for you to expect similar results

42. South Carolina Example Outcome Evaluation • Fruit and vegetable consumption • Sugar-sweetened beverage consumption • Physical activity • Screen time • Home meals • Child care policy Process Evaluation • Implementation of each component of the intervention • Complete and acceptable delivery

43. Maintenance Setting Level: The degree to which the intervention is continued or modified over time. Individual Level: The degree to which effects resulting from the intervention are maintained over time.

44. Maintenance:Questions to Consider • How can you incorporate the intervention or policy into an organization so that it is sustained over time? • How can you help people stay engaged and sustain positive behavior changes over time?

45. RE-AIM Dimensions • REACH – participation among eligible individuals • EFFECTIVENESS – effects on primary outcome of interest • ADOPTION – participation rate among settings • IMPLEMENTATION – intervention was implemented as intended • MAINTENANCE – • Long-term effects of intervention on outcomes • Long-term maintenance of intervention

46. The South Carolina Experience • How SC applied the RE-AIM framework to evaluate an intervention • What SC learned about the intervention as a result • How the intervention was adapted for future implementation • Advice for practitioners

47. I Don’t Know … Process Evaluation HELP !!!! HEADACHE ! Logic Model Summative Information Outcomes Pressure…. Confusion

48. The SC “process” of Process Evaluation • Logic Model… that really was used ! • Complete and Acceptable Delivery • Evaluation Questions and Methods

49. Lessons learned:Reach • What we learned: • Need to expand reach , especially to higher risk families • Parent participation is difficult…. • What we will do in future: • Enhance CMH in Head Start Centers • Modify parent component

50. Lessons learned:Effectiveness • Children more willing to try new fruits / veggies • Increased consumption of fruits / veggies • Healthier snacks served • Increase in % of parents that were active for at least 30 minutes/daily • Increase in # meals eaten at home during week