Why Grade Recommendations?

1. Why Grade Recommendations? • strong recommendations • strong methods • large precise effect • few down sides of therapy • weak recommendations • weak methods • imprecise estimate • small effect • substantial down sides

2. Why Grade Recommendations? • focus on the evidence • alternate practitioner behavior • apply uniformly or • examine evidence themselves • consider patient circumstances • explore with the patient

3. What Are We Grading? • methodological quality of evidence • likelihood of bias • strength of recommendations • must do to might do

4. Grade of Evidence for Specific Question • too vague: alendronate in osteoporosis • patients – post-menopausal women • intervention • daily alendronate, dose 10 to 20 mg. • outcome – non-vertebral fractures

5. What Influences Grade? • study design • basic • detailed design and execution • consistency • directness • reporting bias

6. Methodological Quality • study design • randomization • quasi-randomization • observational study • detailed design and execution • concealment • balance in known prognostic factors • intention to treat principle observed • blinding • completeness of follow-up

7. Methodologic Quality • consistency of results • if inconsistency, look for explanation • patients, intervention, outcome, methods • no clear threshold • size of effect, confidence intervals, statistical significance

8. Relative Risk of Conversion to Sinus Rhythm Amiodarone vs Placebo or Digoxin or CCB Favours Control Favours Amiodarone ' Cowan 1986 1.11 (0.78 to 1.58) n = 34 ' Noc 1990 18.0 (1.17 to 276) n = 24 ' Capucci 1992 0.77 (0.37 to 1.62) n = 40 ' Cochrane 1994 1.15 (0.91 to 1.44) n = 30 ' Donovan 1995 1.05 (0.69 to 1.60) n = 64 ' Hou 1995 1.29 (0.97 to 1.72) n = 39 ' Kondili 1995 1.33 (0.71 to 2.47) n = 42 ' Galve 1996 1.13 (0.84 to 1.52) n = 100 ' Kontoyannis 1998 1.42 (1.08 to 1.85) n = 42 ' Bellandi 1999 1.41 (1.15 to 1.72) n = 120 ' Cotter 1999 1.43 (1.15 to 1.80) n = 100 ' Kochiadakis 1999 1.46 (1.19 to 1.78) n = 204 ' Bianconi 2000 2.04 (0.19 to 22.00) n = 83 ' Galperin 2000 33.70 (2.08 to 546) n = 95 ' Hohnloser 2000 3.13 (1.50 to 6.70) n = 203 ' Joseph 2000 1.32 (0.95 to 1.80) n = 75 ' Natale 2000 5.12 (2.60 to 10.00) n = 85 ' Peukurinen 2000 2.45 (1.49 to 4.02) n = 62 ' Vardas 2000 2.01 (1.55 to 2.60) n = 208 ' Villani 2000 4.75 (1.60 to 14.00) n = 120 ' Cybulski 2001 1.87 (1.37 to 2.55) n = 160 0.1 1 10 100

9. Relative Risk of Conversion to Sinus Rhythm Amiodarone vs Placebo or Digoxin or CCB Favours Control Favours Amiodarone AF Duration > 48 hrs ' Bianconi 2000 2.04 (0.19 to 22.00) n = 83 ' Galperin 2000 33.70 (2.08 to 546) n = 95 ' Hohnloser 2000 3.13 (1.50 to 6.70) ' n = 203 Natale 2000 5.12 (2.60 to 10.00) n = 85 ' Villani 2000 4.75 (1.60 to 14.00) n = 120 ' Pooled Estimate 4.33 (2.76 to 6.77) AF Duration =/< 48 hrs ' Cowan 1986 1.11 (0.78 to 1.58) n = 34 ' Noc 1990 18.0 (1.17 to 276) n = 24 ' Capucci 1992 0.77 (0.37 to 1.62) n = 40 ' Cochrane 1994 1.15 (0.91 to 1.44) n = 30 ' Donovan 1995 1.05 (0.69 to 1.60) n = 64 ' Hou 1995 1.29 (0.97 to 1.72) n = 39 ' Kondili 1995 1.33 (0.71 to 2.47) n = 42 ' Galve 1996 1.13 (0.84 to 1.52) n = 100 ' Kontoyannis 1998 1.42 (1.08 to 1.85) n = 42 ' Bellandi 1999 1.41 (1.15 to 1.72) n = 120 ' Cotter 1999 1.43 (1.15 to 1.80) n = 100 ' Kochiadakis 1999 1.46 (1.19 to 1.78) n = 204 ' Joseph 2000 1.32 (0.95 to 1.80) n = 75 ' Peukurinen 2000 2.45 (1.49 to 4.02) n = 62 ' Vardas 2000 2.01 (1.55 to 2.60) ' n = 208 Cybulski 2001 1.87 (1.37 to 2.55) n = 160 ' Pooled Estimate 1.40 (1.25 to 1.57) 0.1 1 10 100

10. Directness of Evidence • indirect treatment comparisons • interested in A versus B • have A versus C and B versus C • alendronate vs risedronate • both versus placebo, no head-to-head

11. Four Levels of “Directness” • patients meet trials’ eligibility criteria • not included, but no reason to question • slight age difference, comorbidity, race • some question, bottom line applicable • valvular atrial fibrillation • serious question about biology • heart failure trials applicability to aortic stenosis

12. Levels of Directness • interventions • same drugs and doses • similar drugs and doses • same class and biology • questionable class and biology • outcomes • same outcomes • similar (duration, quality of life) • less breathlessness for role function • laboratory exercise capacity for q of life

13. Magnitude, Precision, Reporting Bias • magnitude not generally part of quality • but very large magnitude can upgrade • precision not generally part of quality • but sparse data can lower quality • reporting bias • high likelihood can lower quality

14. Grading System • high quality well done RCT • intermediate quasi-RCT • low well done observational • insufficient anything else

15. Moving Down • study execution • serious flaws can lower by one level • fatal flaws can lower by two levels • consistency • important inconsistency can lower by one level • directness of evidence • some uncertainty re relevance lower by one level • major uncertainty re relevance lower by two levels • selection bias • strong suspicion lower by 1 level

16. Moving Up • very strong association, up 2 levels • insulin in diabetic ketoacidosis • strong, consistent association with no plausible confounders, up 2 levels • fluoride for preventing cavities • strong association can move up 1 level • ? HRT ?

17. Risk/Benefit tradeoff • aspirin after myocardial infarction • meta-analysis of high quality RCTs • clear positive treatment effect • unequivocal recommendation to treat, uniform practice • accelerated TPA vs streptokinase after MI • very large single trial • clear positive effect • uncertainty whether to treat, varying practice

18. Risk/Benefit tradeoff • Aspirin after myocardial infarction • 25% reduction in relative risk • side effects minimal, cost minimal • benefit obviously much greater than risk/cost • TPA vs streptokinase after MI • 12% reduction in relative risk • increased rate of intracranial hemorrhage • large increase in cost • benefit vs risk/cost a judgment call

19. Strength of Recommendations Aspirin after MI – do it TPA rather than SK in MI -- probably do it -- probably don’t do it

20. Grade of Recommendations • do it • probably do it • toss-up • probably don’t do it • don’t do it

21. Judgment: Benefits vs Risks/Costs • seriousness of outcome • magnitude of effect • precision of treatment effect • risk of target event • risk of adverse events • cost of therapy • values

22. Grades Translations • do it - 90% would choose it • possibly do it – 60% to 90% would choose • toss-up – 40% to 60% would choose • possibly don’t – 60% to 90% would decline • don’t - 90% would decline

23. Conclusion • challenges in grading • balancing simplicity and complexity • judgement always required • consistent grading system required • must consider study design, execution, consistency, directness, reporting bias • magnitude, precision, only when extreme • balance of benefits and risks/cost • magnitude of effects; precision of effects; values and preferences