Standards for Developing Trustworthy Clinical Practice Guidelines

1. Standards for Developing Trustworthy Clinical Practice Guidelines Institute of Medicine January 11, 2010 Sandra Zelman Lewis, PhD Asst VP, Health & Science Policy American College of Chest Physicians

2. Most Important Challenge: Definition • Need for a universally accepted definition of evidence-based guidelines (EBGs) • The problem • Consensus statements and “even less structured” documents are often designated as “guidelines” • Physicians are misled into thinking that these are evidence-based and methodologically rigorous • Patient care can be adversely impacted

3. Most Important Challenge: Definition • What action should this committee take? • Set the bar high for methodological rigor • Require a minimal threshold of rigor or at least provide a rating scale that EBG users can understand • Do not allow consensus statements (and less evidence-based documents) to be titled “guidelines”

4. Most Important Challenge: Definition • Challenges • Within EBGs, should consensus-based recs be allowed? • Reduce to areas of identified need for guidance • Allow only when the evidence is weak, inconsistent, or nonexistent • Downgrade appropriately • Ensure panel: expertise, multidisciplinary, vetted for COIs • Appropriate review and balanced viewpoints • Some guideline developers must adjust their processes • Set a timetable to meet the new definition and standards • Provide instruction on new standards and methods

5. Another Important Challenge: Funding • Funding guidelines is the biggest challenge that guidelines developers face. • The problem • EBGs are very expensive if done correctly • Little funding available, difficult to obtain • Charge to this committee • Identify sources for funding guidelines produced according to the established standards

6. When Evidence is Nonexistent or Poor Quality • What does ACCP do when the scientific evidence is absent or poor? • Sets minimal threshold for evidence: Must be published in a peer-reviewed journal • Downgrade recs when poor quality evidence • Allow consideration of indirect evidence if justification (described in text) acceptable but downgrade appropriately • If evidence is not sufficient, then discuss in text but do not provide recommendation

7. Disagreements in Interpretations of Evidence • How do you reconcile disagreements in evidence interpretation among guidelines? • Assess rigor of methodology/adherence to the evidence • Request invited associations to review and comment on our guideline recommendations • Offer competing guideline organizations opportunity to appoint a member to our next edition or update panel (providing he/she passes COI vetting and approval process)

8. Accommodating Subgroups • How do guidelines accommodate subgroups (ex. older populations or persons with multi-morbidities) whose treatment outcomes may differ from the average patient? • If data exist for subgroups, use it. • If not, use indirect evidence from similar groups and downgrade appropriately • Always describe patient population in each recommendation based on the patient population in the original studies • A major challenge is multiple co-morbidities

9. Setting Standards • What topics and/or processes do you think the committee should consider in deriving quality standards for clinical practice guidelines?  • Establish a definition of evidence-based guidelines, possibly with rating scales • Standard setting topics and criteria • PCPI (Physician Consortium for Performance Improvement) criteria for guidelines permitted as basis of performance measures • AGREE instrument (AGREE III is in development) • COGS (Conference on Guideline Standardization) • GLIA – for implementation purposes

10. Panel Composition • What should the composition of CPG development panels, in particular the balance of methodologists, topical experts, and consumers, look like? • Depends on structure of panel but should have methodologists reviewing evidence and developing evidence tables or profiles • Consumers must be educated in EBM • Content experts provide credibility but all must go through rigorous review, including COIs • Other considerations: health economists, frontline clinicians, patient preference consultants

11. Grading System • Is there an available assessment tool that adequately rates both the level of the scientific evidence and strength of clinical recommendations that should be used as standard practice in guideline development? • The ACCP Grading System • A modification of GRADE (major difference: restricts evidence to approved threshold) • Based on (1) assessment of the quality of the original studies and (2) a balance of the risks to benefits • Has been adopted by other guideline developers

12. Grading System

13. Recommendations for PMs • What methods might be developed for determining which recommendations among those in a guideline should be applied to quality measures or electronic medical record decision prompts?     • Evidence should dictate direction and strength of recs • Suggest some 1A and 1B recs for PMs (although not all) • 1C, 2A, 2B, and 2C recs should generally not be used • However, all should be evaluated based on feasibility, usability, scientific importance, practicality, and applicability

14. QI and Harmonization • What administrative (eg, accreditation) or legal approaches might improve the quality of clinical practice guidelines? • Published guideline quality rating scale • Listed on NGC Web site • Currency rating also listed in NGC • PMs & CMS policies should be based on highly rated EBGs • What explicit approaches might harmonize guideline developers and increase guidelines convergence? • Funding requires multiple societies to collaborate • Require compliance with evidence-based standards, rating scale

15. EBG Promotion • What types of strategies might promote greater utilization of guidelines? • Requirements for implementation into: • EMRs • Registries • PMs, including PQRI • Education to allow developers to learn about new techniques and processes (see last slide) • Healthcare providers need to know how to find good guidelines and good guidelines should address the needs of providers (ask frontline clinicians what they need!)

16. Other Considerations • Are there other characteristics of guideline standards you think are important for the committee to consider? • Most important: address funding • Also should move the field toward incorporating resource considerations into the EBG recs

17. Promotion of IOM Standards • The IOM report and new standards should be presented to appropriate audiences: • Guideline developers will be attending the Guidelines International Network conference and guideline methodology course

18. Guidelines International Network (G-I-N) 2010 Conference - Chicago August 26-28, 2010 – Conference Dates August 25, 2010 – Pre-meeting Course in Guideline Methodology Chicago, Illinois, USA www.GIN2010.org Host: American College of Chest Physicians