US Preventive Services Task Force

1. US Preventive Services Task Force Association of Community Health Nurse Educators Lucy Marion, PhD, RN Dean, MCG School of Nursing June 6, 2008

2. History of the Task Forces • 1976 – Canadian Task Force on Periodic Health Exam • 1984 – USPSTF established by PHS • 1996 – Task Force on Community Preventive Services (Community Guide) established by CDC • 1998 - 3rd USPSTF reconvened by AHRQ • 2001 – Present – Standing USPSTF

3. Who is the USPSTF? • Experts in primary care, prevention, research methods • Government supported, by AHRQ, but independent • Family medicine, internal medicine, pediatrics, obstetrics/gynecology, nursing, behavioral health • Scientific support from Evidence-Based Practice Centers (EPC) • Non-member liaisons from primary care clinician associations, Federal agencies

4. Thomas G. DeWitt, MD Allen Dietrich, MD, MPH Kimberly D. Gregory, MD, MPH David Grossman, MD, MPH George Isham, MD, M.P.H. Michael LeFevre, MD, MSPH Rosanne Leipzig, MD, PhD Bernadette Melnyk, PhD, RN, CPNP/NPP Lucy N. Marion, PhD, RN Virginia A. Moyer, MD, MPH Judith K. Ockene, PhD, MSEd George F. Sawaya, MD J. Sanford Schwartz, MD, AB Timothy Wilt, MD, MPH Current Members of the USPSTF Bruce N. (Ned) Calonge, M.D., M.P.H. (Chair) Diana B. Petitti, M.D., M.P.H. (Vice Chair)

5. Carolyn Williams Nola Pender Janet Allan Carol Loveland-Cherry Lucy Marion Bernadette Melnyk Nurse Members of the USPSTF

6. Task Force Activities • Systematically reviews the evidence of effectiveness and develops recommendations for clinical preventive services • Age- and risk-factor specific • For routine use in primary care practice • Recommendations include: • Screening tests • Counseling • Preventive medications

7. Why Evidence-Based? • Need transparent, systematic process to obtain and distill best available (or best feasible) evidence to support decision making • Identifying, evaluating and summarizing scientific evidence about outcomes or interventions or policies • Translating evidence into practice recommendations

8. Topic Selection and Prioritization

9. Review of Criteria for Selecting and Prioritization of Topics • The Task Force solicits new topics for consideration from public, professional orgs, and TF members. 2.  The USPSTF first considers whether newly nominated topics are within scope of primary/secondary prevention, primary care relevant, and with substantial health burden. 3. The USPSTF prioritizes the topics according to public health importance, potential for impact on clinical practice, and addressing diverse populations.

10. Process for Prioritization of Topics • USPSTF prioritizes topics on a 3 point scale (low, moderate, high) based on: • Impact • Burden • Intensity of resources • Helps in determining the order of reviews • Helps in allocating limited resources

11. Methodology for Developing Task Force Recommendations

12. Steps in the Process of Developing Recommendations • Define questions and outcomes of interest • Define and retrieve relevant evidence • Evaluate QUALITY of individual studies • Synthesize and judge STRENGTH of available evidence • Determine balance of benefits and harms • Link recommendation to judgment about net benefits

13. Step 1: Analytic Framework on Screening for a Disease

14. Step 1: Example Analytic Framework - Prostate Cancer 1 Treat radiation, prostatectomy 3 Screen: PSA, DRE Reduced prostate cancer morbidity, mortality Early Prostate Cancer Asymptomatic Men 2 5 4 Adverse effects of screening: false positive, false negative, inconvenience, labeling Adverse effects of Rx: Impotence, incontinence, death, overtreatment

15. 1 Correct use of age- and weight- appropriate restraints (safety seats, booster seats, seat belts) Decreased morbidity (injuries, severity of injuries, length of hospitalizations, short- and long-term disability) and/or mortality from MVOI 2 Behavioral counseling interventions • Clinical Populations • Infant/Child • Adolescent • Young Adults • Adult 3 Reduced driving or riding when driver is under the influence of alcohol 4 Adverse effects Step 1: Example Analytic Framework – Motor Vehicle Occupant Injuries

16. Steps in the Process for Development of Recommendations • Define questions and outcomes of interest • Define and retrieve relevant evidence • Evaluate QUALITY of individual studies • Synthesize and judge STRENGTH of available evidence • Determine balance of benefits and harms • Link recommendation to judgment about net benefits

17. Define and Retrieve Relevant Evidence • For each Key Question developed from AF: • Create inclusion/exclusion criteria based on the key questions defined from the analytic framework • PubMed, Cochrane, and Other database search (CINAHL, etc) • References from key articles, editorials, review articles • Expert consultation (others, TF members)

18. Steps in the Process for Development of Recommendations • Define questions and outcomes of interest • Define and retrieve relevant evidence • Evaluate quality of individual studies • Synthesize and judge STRENGTH of available evidence • Determine balance of benefits and harms • Link recommendation to judgment about net benefits

19. Assess Quality of Evidence What do we mean by quality of evidence? “Extent to which a study’s design, conduct, and analysis has minimized selection, measurement, and confounding biases.” • Lohr, J Qual Improvement, 1999 “Extent to which one can be confident that an estimate of effect is correct” • GRADE , BMJ 2004

20. Evaluate Quality of Individual Studies • Good: • Uses a credible reference standard • Reliability of test assessed • Includes large number of subjects

21. Evaluate Quality of Individual Studies • Fair: • Uses reasonable although not best standard • Interprets reference standard independent of screening test • Moderate sample size

22. Evaluate Quality of Individual Studies • Poor: Has fatal flaw such as: • Uses inappropriate reference standards • Biased ascertainment of reference standard • Very small sample size or very selected patients.

23. Steps in the Process for Development of Recommendations • Define questions and outcomes of interest • Define and retrieve relevant evidence • Evaluate quality of individual studies • Synthesize and judge STRENGTH of overall evidence • Determine balance of benefits and harms • Link recommendation to judgment about net benefits

24. Critical Appraisal Questions • Do the studies have the appropriate research design to answer the key questions? • To what extent are the existing studies high quality? • To what extent are the results of the studies generalizable (or “applicable”) to the general US primary care population and situation?

25. Critical Appraisal Questions • How many studies have been conducted that address each key question? How large are the samples in the studies? • How consistent/coherent are the results of the studies? • Are there additional factors that assist us in drawing conclusions about the certainty of the evidence? (e.g., presence or absence of dose-response effects; fit within a biologic model)

26. Step 4: Synthesize and Judge Strength of Overall Evidence • Evidence reports • Evidence tables summarizing studies • Narrative discussing overall strength of evidence • Meta-analysis • Modeling • Systematic reviews from others – Cochrane, etc.

27. Step 4: Synthesize & Judge Strength of Key Question Evidence Convincing: Well-designed, well-conducted studies in representative populations that directly assess effects on health outcomes

28. Step 4: Synthesize & Judge Strength of Key Question Evidence Adequate: Evidence sufficient to determine effects on health outcomes, but limited by number, quality, or consistency of studies, generalizability to routine practice, or indirect nature of the evidence

29. Step 4: Synthesize & Judge Strength of Key Question Evidence Inadequate : Insufficient evidence to determine effect on health outcomes due to limited number or power of studies, important flaws in their design or conduct, gaps in the chain of evidence, or lack of information on important health outcomes

30. USPSTF Defines Certainty • Likelihood that the assessment of the net benefit of a preventive service is correct. • The net benefit is defined as benefit minus harm of the preventive service as implemented in a primary care population. • The USPSTF assigns a certainty level based on the nature of the overall evidence available to assess the net benefit of a preventive service.

31. Levels of Certainty – HIGH • The available evidence usually includes consistent results from well-designed, well-conducted studies in representative primary care populations. • These studies assess the effects of the preventive service on health outcomes. • This conclusion is therefore unlikely to be strongly affected by the results of future studies.

32. Levels of Certainty – MODERATE • The available evidence is sufficient to determine the effects of the preventive service on health outcomes, but confidence in the estimate is constrained by factors such as: • the number, size, or quality of individual studies • inconsistency of findings across individual studies • limited generalizability of findings to routine primary care practice • or lack of coherence in the chain of evidence. • As more information becomes available, the magnitude or direction of the observed effect could change, and this change may be large enough to alter the conclusion.

33. Levels of Certainty – LOW • Low: The available evidence is insufficient to assess effects on health outcomes. • Evidence is insufficient because of: the limited number or size of studies; important flaws in study design or methods; inconsistency of findings across individual studies gaps in the chain of evidence; findings not generalizable to routine primary care practice; or a lack of information on important health outcomes. • More information may allow an estimation of effects on health outcomes.

34. Steps in USPSTF Process for Development of Recommendations • Define questions and outcomes of interest • Define and retrieve relevant evidence • Evaluate quality of individual studies • Synthesize and judge STRENGTH of overall evidence • Determine balance of benefits and harms • Link recommendation to net benefits

35. Step 5: Determine Balance of Benefits and Harms: Assessing Harms • Potential harms are real, but hard to quantify • Include psychological and physical consequences of false-positives, false-negatives, “labeling”, over treatment

36. Step 5: Determine Balance of Benefits and Harms: Assessing Harms • Opportunity costs • Magnitude and duration of harm subjective, hard to compare to benefits • May translate into QALYs to compare • NNH

37. Step 5: Determine Balance of Benefits & Harms: Assessing Magnitude of Net Benefit • No explicit criteria for magnitude of net benefit • Substantialbenefit: impact on high burden or major effect on uncommon outcome • Problems: requires evidence on harms and common metric for benefit and harms

38. Step 5: Determine Balance of Benefits and Harms Estimate Magnitude of Net Benefit Benefits – Harms = Net Benefit

39. Steps in the Process for Development of Recommendations • Define questions and outcomes of interest • Define and retrieve relevant evidence • Evaluate quality of individual studies • Synthesize and judge STRENGTH of overall evidence • Determine balance of benefits and harms • Link recommendation to net benefits

40. Step 6: Link recommendation to net benefits:USPSTF Grades Recommendations

41. RS: USPSTF Conclusion about Evidence and Net Benefit

43. Step 6: Link Recommendation to Net Benefits: Insufficient Evidence • Lack of evidence on harms or benefits • Poor quality of existing studies • Good quality studies with conflicting results

44. Communicating the Task Force Recommendations: The Recommendation Statement

45. Template for USPSTF Recommendation Statement (RS) • Preamble • Summary of Recommendation & Evidence • Structured Rationale • Clinical Considerations • Discussion

46. RS: Preamble • The USPSTF makes recommendations about preventive care services for patients without recognized signs or symptoms of the target condition. • It bases its recommendations on a systematic review of the evidence of the benefits and harms and an assessment of the net benefit of the service. • The USPSTF recognizes that clinical or policy decisions involve more considerations than this body of evidence alone. Clinicians and policy-makers should understand the evidence but individualize decision-making to the specific patient or situation.

47. RS: Summary of Recommendation & Evidence DO The USPSTF recommends X service for Y population. (A recommendation) The USPSTF recommends X service for Y population. (B recommendation)

48. RS: Summary of Recommendation & Evidence DON’T DO The USPTF recommends against routinely (providing) X service for Y population. There may be considerations that support (providing) the service in an individual patient. (C recommendation) The USPSTF recommends against X service for Y population. (D recommendation)

49. RS: Summary of Recommendation & Evidence WE DON’T KNOW The USPSTF concludes that the current evidence is insufficient to assess the balance of benefits and harms of X service in Y population. (I statement) See Clinical Considerations for suggestions for practice for I recommendations and a discussion of known risk factors, etc.

50. RS: Structured Rationale Importance: Detection: Benefits of detection and early intervention: • Bullets for different populations Harms of detection and early intervetion: • Bullets for different populations The USPSTF concludes that for : • Statement about certainty • Bullets for each population