Evaluating Therapy Articles

1. Evaluating Therapy Articles Lecture 11, 12

2. NotOutcome Outcome Exposed a b a+b a b c d NotExposed c d c+d a+b+c+d b+d a+c Randomized Trial

3. Results of randomized controlled trial • Mean difference (CI) • ARR, RRR, NNT (CI)

4. Most scientific papers are presented in a standard format: Abstract Introduction (why the research was done), Methods (how the study was done and what analysis was used), Results (what was found), and Discussion (what the results mean).

6. Steps of critical appraisal A. Are the results of study valid B. Are the valid results of this study important? C. Can we apply this valid, important evidence in caring for our patient?(usefulness in clinical practice)

7. Example • A 75-year-old man is seen in our office after being discharged from hospital 2 weeks previously. During this admission he underwent a carotid endarterectomy after suffering a transient ischemic attack (TIA) and being diagnosed with significant carotid stenosis. His hospital stay was uncomplicated and his discharge medications included metoprolol 50 mg twice daily for hypertension and aspirin 81 mg daily.

8. Today, he has brought us an article from the Internet describing the benefits of statins for stroke prevention, and he wonders what this medication is and if he should take it. Our note from his last visit showed that his total cholesterol was 200 mg/dl/L, HDL-cholesterol was 65 mg/dL, and LDL-cholesterol was 80 mg/dL. His examination was unremarkable.

9. Based on this scenario, we posed the following question: In a patient with history of TIA, carotid endarterectomy, hypertension, and normal lipid profile, does therapy with a statin decrease risk of stroke? • Using PubMed Clinical Queries we identified the MRC trial which might help us answer this question. • (We can also found the MRC trial in ACP Journal Club Online.) • MRC/BHF Heart Protection Study. Lancet 2002; 360: 7–22.

10. A. Are the Results of Study Valid • The issue of validity speaks to the "truthfulness" of the information. • The evidence that supports the validity or truthfulness of the information is found primarily in the study methodology. Here is where the investigators address the issue of bias, both conscious and unconscious.

12. If the study fails any of the criteria discussed above, we need to decide if the flaw is significant and threatens the validity of the study.

13. 1. Was the assignment of patients to treatment randomized? • Randomization balances the treatment groups for prognostic factors, even if we don’t yet know enough about the target disorder to know what they all are. • We should insist on random allocation to treatment because it comes closer than any other research design to creating groups of patients at the start of the trial who are identical in their risk of the event we are trying to prevent.

14. 2. Was the randomization concealed? • We should look to see if randomization was concealed from the clinicians and study personnel who entered patients into the trial. • If allocation was concealed, the clinicians would be unaware of which treatment the next patient would receive and thus unable, consciously or unconsciously, to distort the balance between the groups being compared. • As with failure to use randomization, inadequate concealment of allocation can distort the apparent effect of treatment in either direction, causing the effect to seem larger or smaller than it really is. • Often, articles don’t state whether the randomization list was concealed, but if randomization occurred by telephone or by some system that was at a distance from where patients were being entered into the trial, we can be assured by this.

15. 3. Were the groups similar at the start of the trial? • We should check to see if the groups were similar in all prognostically important ways (except for receiving the treatment) at the start of the trial. • As noted above, the benefit of randomization is the equal distribution of potential confounders between the study groups. • However, baseline differences between the study groups may be present due to chance. If the groups aren’t similar, we must determine if adjustment for these potentially important prognostic factors was carried out. It is reassuring if the adjusted and unadjusted analyses yield similar results.

18. 4. Was follow-up of patients sufficiently long and 5. complete? • If, for example, patients receiving the experimental treatment dropped out because of adverse outcomes, their absence from the analysis would lead to an over-estimation of the efficacy of the treatment. • Drop out should not be more than 20% • worst case analysis • We should also ensure that the follow-up of patients was sufficiently long to see a clinically important effect.

19. 6. Were all patients analyzed in the groups to which they were randomized? • Because anything that happens after randomization can affect the chance that a study patient has the outcome of interest, it’s important that all patients (even those who fail to take their medicine, or accidentally or intentionally receive the wrong treatment) are analyzed in the groups to which they were allocated;

22. 7. Were patients, clinicians, and study personnel kept blind to treatment? • Blinding is necessary to avoid patients’ reporting of symptoms or their adherence to treatment being affected by hunches about whether the treatment is effective. • Similarly, blinding prevents the report or interpretation of symptoms from being affected by the clinician’s or outcomes assessor’s suspicions about the effectiveness of the study intervention.

23. For example, in the North American Symptomatic Carotid Endarterectomy Trial (this study randomized patients with symptomatic carotid stenosis to either carotid endarterectomy or medical therapy with aspirin), • the patients in the surgical group could obviously not be blinded to the treatment they received. • Outcome events were assessed by four groups: the participating neurologist and surgeon; the neurologist at the study center; “blinded” members of the steering committee; and “blinded” external adjudicators.

24. 8. Were groups treated equally, apart from the experimental therapy? • Blinding of patients, clinicians and study personnel can prevent them from adding any additional treatments (or “co-interventions”), apart from the experimental treatment, to just one of the groups. Usually, we can find information about co-interventions in the methods and/or results sections of an article.

25. 9. Conclusion validity • Sample size, • Standardization and adherence to the protocol

26. 10 control of external factors • (environment and constancy of conditions)

27. 11. outcome • - is it a valid outcome • Is it measured accurately • 12. data analysis

29. B) ARE THE VALID RESULTS OF THIS INDIVIDUAL STUDY IMPORTANT?

30. Chloramphenicol treatment for acute infective conjunctivitis in children in primary care: a randomized double-blind placebo-controlled trial • The mean difference in the time to cure was 0·3 days (log-rank test p=0.025). • Clinical cure by day 7 occurred in 83% of children with placebo compared with 86% of those with chloramphenicol (risk difference 3·8%, 95% CI –4·1% to 11·8%). www.thelancet.com Vol 366 July 2, 2005

31. 1. What is the magnitude of the treatment effect?

33. We can describe the adverse effects of therapy in an analogous fashion, as the number needed to cause harm to one more patient (NNH) from the therapy. • The NNH is calculated as 1/ARI. In the statin study, 0.03% of the control group experienced rhabdomyolysis compared with 0.05% of patients who experienced this in the group that received a statin. This absolute risk increase of |0.03% − 0.05%|=0.02% generates an NNH over 5 years of 5000. • This means that we’d need to treat 5000 patients with a statin for 5 years to cause one additional patient to have rhabdomyolysis. Thus, the NNT and NNH provide us with a nice measure of the effort we and our patients have to expend to prevent or cause one more bad outcome, and their attractiveness as an effort:yield ratio (or “poor clinicians’ cost-effectiveness analysis”) is easily recognized.

34. 2. How precise is this estimate of the treatment effect? • Like any other clinical measure, NNTs are estimates of the truth and we should specify the limits within which we can confidently state that the true NNT lies.

37. C. ARE THE VALID, IMPORTANT RESULTS OF THIS INDIVIDUAL STUDY APPLICABLE TO OUR PATIENT? • To evaluate the relevance of the study’s benefit (=efficacy) to clinical usefulness (=effectiveness), examine carefully the study population.

38. Were the study participants sufficiently different from my patient that this study doesn't help me at all? • Were all clinically important outcomes considered? • Are the likely treatment benefits worth the potential harm and costs?

41. See suggested therapy work sheet

42. Further reading about individual randomized trials • Guyatt G, Rennie D, eds. Users’ Guides to the Medical Literature. A Manual for Evidence-Based Clinical Practice. AMA Press: Chicago, 2008.

43. 1. write a clinical scenario • 2. write an EBM question • 3. Conduct a clinical search step by step • 4. explain in details how did you search the pub med with Search terms

44. Homework • 6. select A RCT, Systematic review, Cohort study or a case control study • 7. summaries the study result in appropriate table and write the author conclusion • 8. critically apprise it using the provided format • 9. what is your conclusion