Using Evidence To Improve Public Health Infrastructure: Let the evidence guide our actions

2. Using Evidence To Improve Public Health Infrastructure: Let the evidence guide our actions January 7, 2004 Jonathan E. Fielding, M.D., M.P.H., M.B.A Director of Public Health and Health Officer L.A. County Department of Health Services Chair, CDC Task Force of Community Preventive Services Professor of Public Health and Pediatrics University of California, Los Angeles

3. If we did not respect the evidence, we would have very little leverage in our quest for truth Carl Sagan

4. Quality of the Evidence • We hear about it often • TV Networks • Usually related to high profile trials • Public Health evidence is different than legal evidence • It is the available information on a particular question • We want the best available evidence in making decisions

5. Decisions and Evidence • Evidence takes many forms • Opinion of leaders • Opinion of “experts” • Studies without controls • Studies with controls • Studies of variable quality in design and execution

6. Decisions and Evidence • Our commitment: • Improve public health • Health problems well defined • Our job: • Make a difference through policies and programs • Inaction is not an option • Hard to identify best evidence to inform decision making

7. Decisions and Evidence • Decisions on policies and programs are often made based on: • Personal experience • What we learned in formal training • What we heard at a conference • What a funding agency required/ suggested • What others are doing

8. Evidence and Public Health Decision Making • Good news • Strong evidence on the effect of many policies/ programs aimed to improve public health • Major efforts underway to assess the body of evidence for wide range of public health interventions

9. What works to improve the public’s health? • Bad news • Many public health professionals are unaware of this evidence • Some who are aware don’t use it • Many existing disease control programs have interventions with insufficient evidence –while others use interventions with strong evidence of effectiveness • Lack of use of effective interventions can adversely affect fulfilling mission and getting public support

10. How do we know what works in improving the health of populations? Background • Many community health improvement efforts have not achieved desired results • Interventions often chosen based on opinions and personal preferences • Evidence based medicine---Clinical Preventive Services Task Force –mid 80s • Evidence based population health --- Community Preventive Services Task Force– mid 90s

11. Systematic Search for the Best Evidence • U.S. Community Preventive Services Task Force Appointed by CDC Director in 1996 • Non-Federal independent task force of experts in multiple relevant disciplines • Epidemiology • Public Health Practice • Behavioral Sciences • Evidence based medicine/ public health • Other relevant areas of expertise

12. Goals • Conduct careful analytic reviews of acceptable evidence for population health interventions and make related recommendations • Use peer reviewed literature • Standard rules of evidence • Standard rules for translating evidence into recommendations for interventions

13. Systematic Reviews of Public Health Interventions are Useful • Methods first developed by social scientists (e.g., Glass, ‘76) • Distill and summarize large and diverse bodies of evidence • Reduce errors and biases in interpretation • Make assumptions explicit

14. Systematic Reviews Are Not: • Limited to randomized controlled trials • Limited to healthcare interventions • Restricted to a “biomedical model” of health - Petticrew, 2001

15. Task Force on Community Preventive Services

16. Who Is the Audience? • People who plan, fund, or implement public health services and policies for communities and healthcare systems • Public health departments • Healthcare systems and providers • Purchasers • Government agencies • Community organizations

17. Community Guide Topics

18. Methods for Systematic Reviews of Effectiveness Evaluations • Develop conceptual framework • Search for and retrieve evidence • Rate quality of evidence • Summarize evidence • Translate strength of evidence into finding • Strongly recommended • Recommended • Insufficient evidence

19. Increasing Treatment Provider- Enhancing Community of Vaccine- Based Access to Demand for Preventable Interventions Vaccinations Vaccinations Diseases Reduced Disease Instance Attendance in Vaccine- Morbidity Vaccination Public, Private, or Preventable and Population Coverage Joint Healthcare Disease Mortality Systems Intervention Types Determinants Exposure to Intermediate Outcomes Vaccine- Environment Preventable Disease Public Health Outcomes Reviewed Reducing Not Reviewed Exposure Logic Framework: Vaccine Preventable Disease

20. Standardized Analysis Process • Systematic review of literature • Abstracting of relevant studies • Grading of evidence • Study design • Execution • Translating from quality of evidence to recommendations • Economic analysis • Other benefits and harms

21. How Does the Task Force Define Suitability of Study Design? • Greatest • Prospective with concurrent comparison • Moderate • Multiple before-and-after measurements but no concurrent comparison OR • Retrospective • Least • Single group before-and-after • Cross-sectional

22. What Factors Determine Quality of Execution? • Description of intervention and study population • Sampling procedures • Exposure and outcome measurements • Approach to data analysis • Interpretation of results • Follow-up • Confounding • Other bias • Other issues

23. How Does the Task Force Draw an Overall Conclusion About the Strength of a Body of Evidence? • Number of studies • Design suitability • Quality of execution • Consistency • Effect size

25. Physical Activity:Review of One Intervention • Goal: increase the amount of time students spend doing moderate or vigorous activity in PE class through curricular change • Interventions reviewed included changing the activities taught (e.g., substituting soccer for softball) or modifying the rules of the game so that students are more active (e.g., in softball, have the entire team run the bases together when the batter makes a base hit). Many interventions also included health education.

26. School Curricular Interventions to Improve Physical Fitness • 14 acceptable studies; in all students’ physical fitness improved. • 5 studies measured activity levels during PE class; all found increases in • amount or percentage of time moderately/ vigorously active and/or • intensity level of physical activity during class. • Median estimates--modifying school PE curricula as recommended will result in an 8% increase in aerobic fitness

27. School Curricula to Improve Physical Activity • Modifying school P.E. curricula was effective across diverse racial, ethnic, and socioeconomic groups, among boys and girls, elementary- and high-school students, and in urban and rural settings. • In a separate literature review, having students attend school PE classes was not found to harm academic performance. • Economic analysis pending.

28. Different Ways to Improve Public Health • Focus on diseases/ injuries e.g. sexually transmitted diseases, intentional injury, diabetes • Focus on risk factors e.g. tobacco use, nutrition, physical activity • Focus on underlying factors that impact multiple dimensions of health e.g. poverty, social isolation

29. Social Environment Physical Environment Genetic Environment • Individual • Response • Behavior • Biology Health and Function Disease and Injury Health and Medical Care Well-Being Prosperity Underlying Health Determinants

30. Impact of Early Childhood Home Visitation Programs • Program can prevent child maltreatment in high-risk families. • In studies reviewed, home visiting resulted in a 40% reduction in child maltreatment episodes. • Longer duration programs produce larger effects; programs of less than 2 years duration did not appear to be effective. • Professional home visitors may be more effective than trained paraprofessionals • but longer-duration programs with trained paraprofessionals can also be effective.

31. Impact of Early Childhood Home Visitation Programs • All programs reviewed were directed at families considered to be at high risk of child maltreatment, (e.g., single or young mothers, low-income households, families with low birth weight infants). • Other benefits • Health benefits for premature, low birth weight infants and for disabled and chronically ill children • Improved maternal educational attainment, reduced public support, improved child educational performance, reduction in drug use and contact with juvenile justice etc.

32. Tenant-based Rental Voucher Programs • Background • Tenant-based vouchers allow very low income families to rent safe, decent, and affordable privately owned housing in neighborhoods of their choice. • Rental voucher programs, known as “housing mobility programs,” work with landlords and tenants to find rental property outside of neighborhoods of concentrated poverty and relocate families to neighborhoods of greater prosperity.

33. Tenant-based Rental Voucher Programs • Findings from the Systematic Review • 6 studies: rental voucher programs resulted in decreases in victimization of tenants or their property • Families enrolled in rental voucher programs who moved to better areas were • 6% less likely to have a household member victimized • 15% less likely to experience neighborhood social disorder. • Changes in victimization in both urban and suburban settings. • Other benefits: substantially reduced symptoms of maternal depression, boys’ behavioral problems in school, and childhood illnesses and accidents requiring medical attention.

34. General Comments on Evidence Based Reviews • More evidence than sometimes expected, however • Insufficient evidence common outcome • Very resource intensive process • Quality of studies vary widely • Economic data still uncommon • Important to consider harms even though uncommon

35. TASK FORCE REVIEWS AND RECOMMENDATIONS www.thecommunityguide.org • Results of all reviews to date • Frequent updating • Downloadable Slide Sets

36. How Evidence Can Improve Public Health Infrastructure • Explore evidence underlying options to reach each public health goal • e.g. smoking control, reducing disparities in infant mortality, increasing physical activity, increasing immunization rates • Comprehensively review the best sources of evidence reviews • Community Guide (best source when topic of interest has been covered) • Recent review articles in peer reviewed journals • Other meta-analyses funded by responsible federal agencies • Compare results and recommendations of different sources

37. How Evidence Can Improve Public Health Infrastructure Use evidence to decide among possible interventions • How does each possible intervention suit the problem and the population? • E.g. was it tried on particular racial/ ethnic/ age/ gender groups? • Is there reason to belief it would not be as effective for some of these on whom it was not tried? • Is the problem now similar to what it was when the major studies took place?

38. Deciding on Interventions • Single versus multiple component interventions • Single component interventions easier to develop, implement, control and assess, but • Multi-component interventions usually more effective • E.g. Tobacco control in California

39. Deciding on Interventions • Consider both policies and programs • Programs • Greater control over all aspects • Organizational unit has primary responsibility for design, implementation and outcomes • Policies • Control varies: broad policies often made by elected officials • Policies have potential for greater public health impact • Credit needs to be shared e.g. LAUSD Nutrition Policies, increase in tobacco tax

40. Deciding on Interventions • Consider effect size • Median • Consistency • Consider breadth of target population • Together effect size and target population define the overall population effect

41. How Evidence Can Improve Public Health Infrastructure • Use evidence to determine realistic goals by estimate effect size (i.e.. how much you move the needle!) Relative percentage changes in exposure to environmental tobacco smoke attributable to workplace smoking bans and restrictions from studies that qualified for inclusion in this review (“a” and “b” in Study names refer to first or second study by the same author in that year, included in this review)

42. Deciding on Interventions • What is the slope of the effect curve? • Larger initial effects with significant recidivism • Smaller initial effects with Increasing impact over time • What is the time frame for observed health benefits? • How long were the follow-up periods for the best studies? • For equal benefit, shorter is better, but • Long term benefit is primary interest

43. Deciding on Interventions • What is the cost of the intervention? • Personnel • Dollars i.e. contracts • Time to implement • Likelihood of funding for sufficient period to get effect • Potential for dedicated or incremental funding • Opportunity cost (i.e. cost of not doing other things)

44. Deciding on Interventions • Relative cost-effectiveness • Cost effectiveness is dollar cost per health outcome (including clear intermediate outcomes) e.g. smoker prevented lead poisoning prevented STD cured drug treatment completed Note: some interventions have multiple health benefits e.g. smoking affects CVD, some cancers, respiratory disease etc.

45. Deciding on Interventions • Who else needs to be involved to be successful? • Within public health • Within personal health services • Voluntary agencies • Health care organizations • Health plans • Employers • How difficult is it to get agreement on: • Roles and responsibilities • Interventions? • Time cost versus partnership benefit

46. How Evidence Can Improve Public Health Infrastructure • Use evidence to help decide on construction of intervention • Interventions with same name can be very different • Follow the design used in most successful interventions • Talking to those who did the studies is very helpful in refining intervention

47. Using the Evidence • Use evidence to frame objectives • Use evidence to develop evaluation plan and related evaluation • Approach • Measures • Data collection plan • Develop internal evidence through performance measurement system • Frequent monitoring essential

48. What more is needed? • More research on public health practice; for many interventions---insufficient evidence • Increased funding for evidence based reviews using consistent methodologies • More training on appropriate sources and uses of evidence • in schools of public health and others training public health professionals • in public health practice settings e.g. state and local health departments • Political leaders and others who influence the decision making process to improve health

49. DHS Public Health Opportunities • Become sophisticated user of evidence based information and recommendations • Make use of best evidence key aspect of performance of program directors and key managers • Contribute to the literature on what works in public health practice

50. Presentation available on DHS Intranet • Also see Evidence-Based Public Health, Ed. Ross C. Brownson, Elizabeth Baker, Terry L. Leet etc. Oxford University Press, 2003