New York, December 10, 2012 How to develop pragmatic and trustworthy guidelines

1. Holger Schünemann Professor and Chair, Dept. of Clinical Epidemiology & Biostatistics Professor of Medicine Michael Gent Chair in Healthcare Research McMaster University, Hamilton, Canada New York, December 10, 2012How to develop pragmatic and trustworthy guidelines

2. Disclosure • Co-chair GRADE Working Group • World Health Organization: Advisory Committee for Health Research, Expert Advisory Panel on Clinical Practice Guidelines and Clinical Research Methods and Ethics & Chair of various guideline panels; funding for guideline development/SR • Work with various other guideline groups using GRADE • American College of Physicians (ACP) Clinical Practice Guidelines Committee • American College of Chest Physicians (ACCP) • No direct/personal payments from drug or device industry

3. “Pragmatic” (rapid?) systematic reviews • How to? • Challenges • Solutions • Integration in guidelines • Developing recommendations (example and small group work) ontent C

4. Institute of Medicine Report on Trustworthy guidelines • Be based on a systematic review of the existing evidence; • Be developed by a knowledgeable, multidisciplinary panel of experts and representatives from key affected groups; • Consider important patient subgroups and patient preferences as appropriate; • Be based on an explicit and transparent process that minimizes distortions, biases, and conflicts of interest; • Provide a clear explanation of the logical relationships between alternative care options and health outcomes, and provide ratings of both the quality of evidence and the strength of recommendations; and • Be reconsidered and revised as appropriate when important new evidence warrants modifications of recommendations.

5. Guideline International Network

6. Guideline development Process

7. Working with evidence at WHO • For key recommendations: • Search for and retrieve all available evidence • Identify relevant SRs • Formally assess quality of evidence • GRADE (systematic and transparent approach) • Ideally based on systematic reviews

8. AGREE II

9. Rapid & pragmatic but systematic • Why? • Resources or Time (or both) • Transparency is key • Methods do not have to differ if more capacity in the team

12. Team with expertise • Manage bias/COI (team and others) • Stakeholder input • Formulate topic • Develop protocol • PICO • Protocol for peer review & make public • Comprehensive search • Address biased reporting • Screen and select • Manage data selection • Critically appraise studies • Evaluate body of evidence (GRADE) • Qualitative and quantitative summary • Structured report • Peer review • Public access R apid & pragmatic IOM

13. Team with expertise • Manage bias/COI (team and others) • Stakeholder input • Formulate topic • Develop protocol • PICO • Protocol for peer review & make public • Comprehensive search • Address biased reporting • Screen and select • Manage data selection • Critically appraise studies • Evaluate body of evidence (GRADE) • Qualitative and quantitative summary • Structured report • Peer review • Public access R apid & pragmatic IOM

14. Use existing SR?

15. Allergic Rhinitis in Asthma (ARIA) Guidelines • Model applied to many other guidelines (e.g. for WHO, WAO, ACCP)

16. Create evidence profile with GRADEpro Summary of findings & estimate of effect for each outcome Guideline development Gradeoverall quality of evidence across outcomes based on lowest quality of critical outcomes Rate quality of evidence for each outcome Outcomes across studies Formulate question Rate importance Select outcomes Risk of bias Inconsistency Indirectness Imprecision Publication bias P I C O Outcome Critical High  Outcome Critical Moderate O Grade down Low OO Outcome Important Very low OOO Outcome Not Large effect Dose response Opposing bias & Confounders important Grade up Input? Panel • Grade recommendations • For or against (direction)  • Strong or conditional/weak (strength) • By considering balance of: • Quality of evidence • Balance benefits/harms • Values and preferences • Revise if necessary by considering: • Resource use (cost) Systematic review Guideline • Formulate Recommendations ( | …) • “We recommend using…” | “Clinicians should…” • “We suggest using…” | “Clinicians might…” • “We suggest not using…”| “Clinicians … not…” • “We recommend not using…”| “Clinicians should not…”

17. Steps

18. Steps

19. Steps • e.g. missing outcomes • Use same search strategy • Do meta-analysis or assess similarity of results

20. Steps

21. 48 recommendations + recommendations for subgroups 84 evidence profiles

24. Understanding recommendations and the underlying evidence

26. R apid guidelines

28. Budget • Usually $5,000 to 10,000 per evidence profile • ARIA ~ $300,000 plus meeting cost

29. Summary • Pragmatism can retain transparency • Following SR methods, including evidence profiles enhances transparency • Progress on moving from evidence to recommendations

30. Team with expertise • Manage bias/COI (team and others) • Stakeholder input • Formulate topic • Develop protocol • PICO • Protocol for peer review & make public • Comprehensive search • Address biased reporting • Screen and select • Manage data selection • Critically appraise studies • Evaluate body of evidence (GRADE) • Qualitative and quantitative summary • Structured report • Peer review • Public access R apid & pragmatic IOM