Using the Community Guide to Move the Research Agenda Forward

1. Using the Community Guide to Move the Research Agenda Forward Peter A. Briss, MD, MPH February 03, 2005

2. Why Evidence-Based Public Health? • Resources are tight … and getting tighter • Public health is more visible—therefore our decisions are more carefully examined • Increasing pressure to be accountable • Gaps between scientists and decision-makers—priorities, language, and approaches • Increasing pressure to embrace “evidence” methods

3. Evidence and Public Health Decision Making • Good news: • Major efforts underway to assess the body of evidence for a wide range of public health interventions • More and more high quality reviews available • But capacity not what it might be • Strong evidence on the effect of many policies/programs aimed to improve public health

4. But…Awareness and Use Are Not What They Might Be • Bad news: • Many public health professionals are unaware of this evidence • Some who are aware of it don’t use it • Many existing disease control programs use interventions with insufficient evidence, while better-documented alternatives are available • Failing to use an effective intervention is a missed opportunity that can adversely affect fulfilling mission and getting public support

6. The Community Guide Seeks to Answer Many Important Questions: • Under what circumstances is an intervention appropriate? • Does it work? • How well? • For whom? • What does it cost? • Does it provide value? • Are there other barriers I need to know about?

7. So What Does One Do with This Kind of Information? • Know what to expect • Know which programs are more likely to be successful • Support decisions about research • What programs need additional research to support decisions? • What research is needed, (e.g., formative, effectiveness, replication, or dissemination)? • Advise program planners and evaluators

8. Essential Information, But Only One Piece Of The Puzzle • Community assessment • Priority setting • Objective setting • Intervention selection • Implementation • Evaluation • Repeating the cycle

9. What’s Been Accomplished So Far? • 171 findings to date • More in the pipeline… • Book publication in Jan 2005: Oxford University Press • People are beginning to use the Community Guide as a starting place to access evidence-based prevention advice • Beginning to see effects on practice, policy, research

10. What’s Been Published Relevant to Cancer Prevention? • Primary Prevention • Tobacco Use (2000, 2001, in preparation) • Physical Activity (2001, 2002, in preparation) • Skin Cancer Prevention (2003, 2004) • Improving Vaccination Coverage (1999, 2000, in press)

11. What’s Been Published Relevant to Cancer Prevention? (cont’d) • Improving processes of health care • Promoting Informed Decision Making (2004) • Culturally-competent health care (2003, in preparation) • Population-based interventions for the detection of oral cancer (2001, 2002)

12. More on the Way • Early phase • Alcohol • Worksite Health Promotion • Midcourse • HIV • Sexual Behavior • Nutrition • Late course • Obesity • Promoting Cancer Screening

13. There Are Only Two Outcomes of a Guide Review • Move practice forward • Move research forward

14. We Know Less About Moving Research Forward Than Practice • Collaboration between the Community Guide and the network is an evolving work in progress • Need for ongoing dialogue: • What does the network need? • formats? • detail? • additional information? • We’d also like to get feedback from you that might influence our more general reviews or communications

15. Still Building the Airplane . . .While We’re Flying • We have only about 4 years of experience in trying to use the Community Guide to move research forward in a variety of areas, but I’ll talk generally about some potential uses • I recommend you also read chapter 12 in the book

16. Effort Required to Establish a Community Guide Recommendation

17. Effort Required to Implement a Community Guide Recommendation

18. Research Phases: Health Promotion Programs (After NCI And NHLBI) • Basic research — • Hypothesis development — • Pilot applied studies Very small scale • Prototype studies Experimental or Q-Exp Small scale • Efficacy trials Experimental Numbers sufficient for behavioral evaluation • Treatment effectiveness trials Exp or Q-Exp With outcomes STD delivery Large scale, real world • Implementation effectiveness As above (#6)trials Several types of delivery • Demonstration studies As above (#6) Unrestricted population(s)

19. This Research-to-Practice View Is Useful But Incomplete • The world is not linear-sequential • No place to put synthesis steps • More consistent with “programs that work” models than with synthesis • can’t say much about characteristics that contribute to success or failure • Based primarily on science push and little on user pull • No place to put research that might follow demonstration of effectiveness

20. Perform Research Appropriate to the Stage of Progress of the Field • Define the problem • Identify targets of intervention • Develop theory-based interventions and taxonomy • Evaluate effectiveness

21. Perform Research Appropriate to the Stage of Progress of the Field (cont’d) • Consider: • Targeted replication research that answers important new questions • Whether applicability can be broadened and, if so, what is required • Targeted dissemination research • Other “post-effectiveness” research questions • Research and support for improving fit • Cost and cost effectiveness • Identification and reduction of implementation barriers • What else? • Testing/production/dissemination of “how to” materials

22. How Can Reviews Help to Inform Additional Research • Identify what is already known and where are the remaining gaps: • Object is to move a field downstream • Hope is to help identify “low hanging fruit” • better complement work that has already been done • Identify opportunities to kill multiple birds with one stone • For example, replication research might be paired with work on economics or identification and reduction of barriers

23. A Case Story • There are now many examples of implementation of Community Guide and follow-up evaluative or research efforts • Designing new studies to add to what’s already known is harder than it appears

24. A Case Story • In 2000, the TF recommended client reminders as one of several client-oriented interventions to improve coverage with vaccines that are recommended for everyone in a particular age group (i.e., universally-recommended vaccines)

25. What Was The Evidence? • 31 intervention arms of reminders used alone produced a median improvement in coverage of 8 pct pts (range –7 to 31 pct pts) • Intervention characteristics, populations, settings were diverse

26. What Else Did the Task Force Say? • Should be applicable to most adults and children in the US for whom universally recommended vaccines are applicable and in whom improvements in coverage are needed

27. What Else Did the Task Force Say? • Suggested a 4-step process for implementing recommended interventions • Assess current intervention activities and needs • Assess barriers to vaccination • Select interventions that address local barriers • “Using additional interventions when coverage is already high or using additional interventions that are poorly matched to local problems are unlikely to result in important benefits” • Monitor progress and effects • Adequate implementation? • Periodically reassess and adjust

28. Client Reminders for Adult Flu Shots: Methods Site Sample and design Data collection 3 Health Plans in CT ~9500 high-risk adults, 18–64 yrs 55% response rate Mail survey

29. “Reminder” vs. Small Media

30. Challenges: Implementation • Little formal or informal a priori assessment of locally-important barriers to vaccination due to time and other constraints • The fit of this intervention to locally-important problems was largely unknown

31. Client Reminders for Adult Flu Shots: Crude Results

32. Client Reminders for Adult Flu Shots:Additional Information • Most (55%) of the people who did not get vaccinated this time had never been vaccinated • Might require additional strategies • Previously vaccinated people who were not vaccinated most commonly reported access barriers for which a reminder might not be expected to provide substantial help

33. If This Was An Effectiveness Study • Change in coverage below the median but well within the reported range

34. If This Was A Replication Study • Were these results importantly different from what was expected? • If so, why? • Population (barriers, coverage) • Setting (IPA) • Intervention (type, implementation, something else?) • What we learn from this addition is harder to interpret than I might have expected

35. If This Was A Dissemination Study • Identification of several important implementation barriers • Ensuring fit • Implementing a reminder in the way it was defined in the guide • We learned less about how to address the barriers

36. Opportunities for Improvement • Improved communications between guideline developers, scientists, implementers, and decision makers • Better positioning of recommendations as part of a portfolio of resources to support decision making • Better positioning of intervention selection as part but not all of comprehensive program planning • Probably broaden the range of questions that are addressed by “replication research”

37. Introduction to the Matrix

38. This Network Will Have A Balanced Portfolio of 4 Main Areas Of Study • “Nearly sufficient” • Replication • Dissemination • Evaluation

39. “Nearly Sufficient Evidence” • One or two well done studies could provide sufficient evidence for a recommendation

40. Translating to Recommendations

41. Examples “Nearly Sufficient” • Small numbers of studies trending positive • Few existing studies • Coded yellow

42. Likely Have More “Nearly Sufficient” Examples Than Can Be Immediately Funded • Likely to need additional priority setting criteria, e.g., • Commonly done by programs (DCPC survey) • Already in the PLANET • “Hot topics” • “High stakes” • Controversial

43. Replication Research • Replicate recommended interventions in populations or community settings in which they have not been previously evaluated, • Underserved populations • Health departments and other cancer control partners. • Consider whether you also want to evaluate particular intervention subtypes

44. Examples (Replication Research) • Some fundamental questions have been addressed rarely • B+C • Effectiveness among never-screened • CRC • Effectiveness in promoting screening other than FOBT

45. We Could Use Some Feedback • What applicability information would be most useful to you? • Types of information • Population, Setting, Intervention • Level of detail • We’re willing to pull more info if needed

46. Other Ways To Set Priorities • Commonly done by programs (DCPC survey) • Not yet in the PLANET • Data set missing a characteristic of setting or population that is essential from the perspective of the B+C program

47. How To Effectively Disseminate • Research on how to effectively disseminate or implement within health departments or with community groups or other cancer control partners Guide-recommended community interventions

48. Examples Relevant To Dissemination • Research that identifies and addresses barriers to implementation • Identification and sharing (e.g., on the PLANET) of useful “tools” • Other related research, e.g., on cost or cost effectiveness • Very little economics thus far except for reminders

49. Evaluations Of Recommended Interventions Already Implemented • Evaluate fidelity to recommended interventions • Determine, as much as possible, if they are as effective as might be expected

50. Examples (already implemented) • Surveys of programs about what they say they’re currently doing (or not doing) • Audits of what they’re actually doing (or not doing) • Checks of whether programs match what was recommended • Identification and sharing (e.g., on the PLANET) of useful “tools” (i.e., “how to” advice)