Unsatisfactory quality

1. Failure to meet standards T r u s t Reform required Unsatisfactory quality Cannot trust

2. Failure to meet standards T r u s t Reform required Unsatisfactory quality Cannot trust

3. Failure to meet standards T r u s t Reform required Unsatisfactory quality Cannot trust

4. Failure to meet standards T r u s t Reform required Unsatisfactory quality Cannot trust

5. Failure to meet standards T r u s t Reform required Unsatisfactory quality Cannot trust

6. Arch Int Med T r u s t JAMA Lancet BMJ

7. Arch Int Med T r u s t JAMA Lancet BMJ

8. How to produce top quality national Guidelines Thomy Tonia, MSc Guidelines Methodologist

9. Today’s schedule • Why do we need evidence-based guidelines • Step by step introduction to the GRADE approach • How to adapt guidelines for use on a national level

11. 1990 “Appropriateness Guidelines describe accepted indications for using particular medical interventions and technologies, ranging from surgical procedures to diagnostic tests” 2011 “Systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances" 2013 “Clinical practice guidelines are statements that include recommendations intended to optimize patient care. They are informed by a systematic reviewof evidence and an assessment of the benefits and harms of alternative care options”

12. Grading of Recommendations Assessment Development and Evaluation The GRADE working group began in the year 2000 as an informal collaboration of people with an interest in addressing the shortcomings of present grading systems in health care. Our aim is to develop a common, sensible approach to grading quality of evidence and strength of recommendation

13. Why should *I* bother?

15. Quality rating outcomes across studies Clinical question Rate importance Select outcomes High P I C O Outcome Critical Moderate Outcome Critical Grade down or up Outcome Important Overall quality of evidence Low Outcome Important Not Outcome Very low important Panel • Formulate recommendations: • For or against (direction) • Strong or weak (strength) • By considering: • Quality of evidence • Balance benefits/harms • Values and preferences • Revise if necessary by considering: • Resource use (cost)

16. Main characteristics • Separates quality of evidence (high, moderate, low, very low) and strength of recommendation (strong, weak/conditional) • Systematic, explicit, transparent • Considers patients’ values and preferences very important for local contexts!

18. Theory Hints Setting the scope manageable Choosing the Guideline panel (content experts; methodologist; patients; primary care physicians) and Chair! Guideline Panel

19. Theory Hints Population Intervention Comparison Outcome Population Intervention Comparison Outcome Are IV corticosteroids effective for treating COPD exacerbations? In patients hospitalized for COPD exacerbations is initial treatment with IV corticosteroids compared to oral corticosteroids better (reduction in length of hospital stay)?

20. Theory Hints 1 PICO question/ recommendation 7-10PICO questions/ Guideline No PICO question, no recommendation!

21. Theory Hints • Importance- NOT evidence driven • Guided by patients’ needs and values • What do younger doctors need guidance for?

22. Theory Hints

23. Theory Hints Systematic literature review All relevant electronic databases Handsearching of journals Two assessors

24. Theory Hints Pragmatic GRADE approach Search for recent systematic reviews and build up on them Search main database(s) only One assessor

25. Theory Hints Select studies according to predefined criteria Extract outcomes of interest Meta-analyse, when applicable

26. Theory Hints RevMan

27. Theory Hints

28. Theory Hints Grading per outcome and not per study! Quality in GRADE means more than risk of bias Expert opinion is not a type of evidence A particular quality of evidence does not necessarily imply a particular strength of recommendation

29. Theory Hints • Quality  theextend of ourconfidencethattheestimates of theeffectare • correct adequatetosupport a particulardecision/ recommendation

30. Theory Hints Grading per outcome, not per study! RCTs: high quality Risk of bias Inconsistency Indirectness Imprecision Publication bias

31. Theory Hints Grading per outcome, not per study! Observational studies: low quality Large magnitude of effect Dose-response relation All plausible confounding would result in an overestimate of effect

32. Theory Hints Risk of bias • Lack of allocation concealment • Lack of blinding • Large loss to follow-up • No ITT

33. Theory Hints Inconsistency • Variability/ heterogeneity of results • Possible reasons? (intervention, definition of outcomes, quality of studies etc) • Similarity of point estimates • Overlap of CIs • Statistical criteria (p value for test for heterogeneity, I2)

34. Theory Hints Inconsistency • Differences in direction do not constitute a criterion for rating down! • Sub-group analyses even if statistical heterogeneity is small

35. Theory Hints Inconsistency

36. Theory Hints Indirectness • Generalisability, transferability, applicability • Differences in the components of PICO questions • Indirect comparison • Differences in population (children/adults), intervention (intravenous/oral), outcomes of interest (surrogate outcomes)

37. Theory Hints Imprecision • Confidence interval • Estimate of effect includes both appreciable benefits and not appreciable benefits (or even harms) • If the recommendation would differ if the upper vs the lower boundary of the CI represented the truth, consider rating down

38. Theory Hints Publication bias • Difficult to estimate! • Failure of reporting studies that were undertaken • Delayed reporting of negative trials

39. Theory Hints • Risk higher when only a few small studies that show positive effect are available

40. Theory Theory Hints Hints Guideline Panel

41. Benefits vs Downsides Evidence Quality - + Strong (we recommend…) vs Conditional (we suggest….) Recommendations Values and Preferences Costs

42. - + Strong recommendation more likely as the difference between desirable and undesirable consequences becomes larger.

43. Strong recommendation more likely with higher quality evidence

44. Strong recommendation is more likely as the variability (or uncertainty) about patient values and preferences decreases

45. Theory Hints Values and preferences: SRs? Include patient representatives at the guideline panel or as reviewers Patient surveys If none of the above, describe the values and preferences that the panel placed on each outcome “This recommendation places a relatively high value on the reduction of mortality and a relatively low value on quality of life”

46. A weak recommendation is more likely as the incremental costs of an intervention (more resources consumed) increase

47. Explicit and transparent • Distinguish between quality of the evidence and grade of recommendations • Patient important outcomes • Benefits vs harms • Values and preferences of patients

48. Time and resource demanding • Does not guarantee consistency across graders and does not eliminate the need for judgement • Has been developed mainly for intervention Qs and not for diagnosis

49. Adapting an exıstıng Guideline

50. Definition Process The process provides a systematic approach to adapting guidelines produced in one setting for use in a different cultural and organisational context. The adapted guideline addresses specific health questions relevant to the context of use and is suited to the needs, priorities, legislation, policies and resources in the targeted setting. Flexible, transparent and explicit