Lecture 17: Prevention of bias in RCTs

1. Lecture 17: Prevention of bias in RCTs • Statistical/analytic issues in RCTs • Measures of effect • Precision/hypothesis testing • Compliance/intention to treat • RCTs of effectiveness of screening • Effects of study design (Schultz paper) • Strengths and weaknesses of RCTs Lecture 17 (Oct 28,2004)

2. Analysis of RCTs • Planning stage: • Pre-specified hypotheses • Primary and secondary outcomes • Measure of effect • Sample size calculation Lecture 17 (Oct 28,2004)

3. Analysis of RCTs • Analysis stage: • Check on success of randomization • Analyze adherence to interventions • Intention to treat - why? • Should the analyses be blinded? Lecture 17 (Oct 28,2004)

4. MRFIT study(Multiple Risk Factor Intervention Trial) • Prevention of coronary heart disease (CHD) • Followed Framingham and other observational studies • Multi-site RCT • High-risk men age 35-57 (Framingham algorithm) • N = 12,866 • Comparison groups: • Special intervention (SI): • Reduction of serum cholesterol via smoking cessation, hypertension treatment, dietary modification • Usual care (UC): • Notification of physician of results of risk status Lecture 17 (Oct 28,2004)

5. MRFIT study (cont) • Primary outcome: Death from CHD • Method of analysis? • Secondary outcomes: • Death from any cardiovascular disease • Death from any cause • Overall CHD incidence (fatal and non-fatal cases) • Intermediate outcomes: • Risk factor levels Lecture 17 (Oct 28,2004)

6. MRFIT study(Multiple Risk Factor Intervention Trial) • Sample size estimation: • Expected 6-year CHD death rate = 29.0/1,000 • Hypothesized rate in SI group = 21.3/1,000 (26.6% reduction) • P (type 1 error) = 0.05 (one-sided test) • Power = 0.88 • Basis for projection: • 10% reduction of serum cholesterol if >220 mg/dL (vs no change in UC) • Reduction in smoking rate: • 25% for smokers of 40+ cigs/day (vs 5% UC) • 40% for smokers of 20-39 cigs/day (vs 10% UC) • 55% for smokers of <20 cigs/day (vs 15% UC) • Sub-group hypotheses: • Formulated during trial, blind to interim mortality data • Example: SI would be especially effective in men with normal resting ECGs Lecture 17 (Oct 28,2004)

7. Lecture 17 (Oct 28,2004)

8. Lecture 17 (Oct 28,2004)

9. MRFIT: explanations? • Intervention not effective • Intervention is effective, but less than expected because: • Lower than expected mortality in UC group • Risk reduction in UC group • Positive effect in some sub-groups offset by negative effect in others • In subgroup with hypertension and ECG abnormalities, higher death rate in SI vs UC • Possibly unfavorable response to antihypertensive drug therapy? Lecture 17 (Oct 28,2004)

10. MRFIT - lessons • Consider “contamination” and “compensatory” effects in study design • Clear specification a priori of planned sub-group analyses (with sample size calculations) • (Reference: JAMA 1982, 248: 1465-1477) Lecture 17 (Oct 28,2004)

11. Measures of effect • Types of data to be analyzed: • incidence rate of an adverse event (death, etc) • It = incidence rate in treatment group • Ic = incidence rate in control group • Example (mammography and mortality): • It = 2/10,000/year • Ic = 4/10,000/year Lecture 17 (Oct 28,2004)

12. Risk difference and ratio Risk difference = Ic - It/units • usually easier to express as risk reduction • 4 - 2/10,000/year = 1/10,000/year Risk ratio (relative risk) = Ic = 4/2 = 2.0 It Alternatively: = It = 2/4 = 0.50 Ic Lecture 17 (Oct 28,2004)

13. Relative risk reduction • Analogous to attributable risk percent • Sometimes called percent effectiveness = risk difference = Ic - It risk in control group Ic = 2/4 = 50% • Can be computed from the risk ratio: 1 - 1 RR = 1 -1/2 Lecture 17 (Oct 28,2004)

14. Example from GUSTO trial • tissue plasminogen activator (TPA) vs streptokinase (SK) as thrombolytic strategy in treatment of AMI. • 30-day mortality in TPA group = 6.3% • 30-day mortality in SK group = 7.3% Lecture 17 (Oct 28,2004)

15. Measures of effect RATE/RISK RATIO SK rate = 7.3 = 1.16 TPA rate 6.3 RELATIVE RISK REDUCTION SK rate – TPA rate = 7.3 – 6.3 = 14% SK rate 7.3 [also calculated as 1 – (1/rate ratio)] Lecture 17 (Oct 28,2004)

16. Measures of effect (cont) ABSOLUTE RISK REDUCTION (rate/risk difference; attributable risk) SK rate – TPA rate = 7.3% – 6.3% = 1.0% NUMBER NEEDED TO TREAT (NNT) (Reciprocal of risk difference) 1 = 1 = 100 SK rate – TPA rate .01 Lecture 17 (Oct 28,2004)

17. SELECTION OF EFFECT MEASURES Ratio measures assess strength of effect - how effective is the treatment? Difference measures take into account frequency of the outcome – can assess whether it is worthwhile (allocation of time and $$) Both ratio and difference measures are needed All these measures are estimates and are subject to sampling error – need confidence intervals to determine their precision All the measures are limited by the study(ies) that generated them – they may vary by patient characteristics, adherence to treatment, duration of follow-up, etc) Measures consider only beneficial and not adverse effects of treatment. Lecture 17 (Oct 28,2004)

18. Aspirin in prevention of MI among male smokers (data from Physicians’ Health Study) 5-year incidence of MI: aspirin group = 1.2% placebo group = 2.2% • Risk ratio = 1.8 • Relative risk reduction = 45% • Absolute risk reduction = 1.0% in 5 years • NNT = 100 for 5 years (to prevent 1 MI) Lecture 17 (Oct 28,2004)

19. Antihypertensive treatment in 75-year old women with BP of 170/80 (data from SHEP study) • 5-year incidence of stroke: treatment group = 5.2% placebo group = 8.2% • Risk ratio = 1.6 • Relative risk reduction = 37% • Absolute risk reduction = 3.0% in 5 years • NNT = 33 / 5 years (to prevent 1 stroke) Lecture 17 (Oct 28,2004)

20. Measures of effect in RCTs: continuous outcomes • Example: RCT of antidepressant vs placebo: • Measures on depression scale at baseline and at follow-up • Possible measures: • Difference in mean scores at follow-up • Difference in change scores from baseline to follow-up Lecture 17 (Oct 28,2004)

21. Adherence to interventions • Possible outcomes: • Low adherence in one or both study groups • E.g. St John’s wort vs sertaline • Cross-over • E.g., RCTs of medical vs surgical treatment of CHD • How should results be analyzed? • By intervention to which randomized (“intention-to-treat”) • By intervention actually received? Lecture 17 (Oct 28,2004)

22. RCTs of screeningExample: evaluation of the effectiveness of breast cancer screening (HIP study) • 1st RCT of breast cancer screening • Study population: Members of HMO • Intervention: Invitation to receive annual mammography and clinical exam (3 years) • Possible outcomes: • survival rate (1 year, 5 year) • case-fatality rate • mortality rate • Which would you use? Lecture 17 (Oct 28,2004)

23. Lecture 17 (Oct 28,2004)

24. Bias in RCTs of screening • Definition of time zero? • Date of first symptoms? • Date of detection? • Date of diagnosis? • Bias if difference in “time zero”between study groups: • screening/early detection intervention shifts time zero • intervention appears to lengthen time to outcome without real change in prognosis • “lead time” bias • “length” bias Lecture 17 (Oct 28,2004)

25. Other types of bias in RCTs • Hawthorne effect: • Non-specific effect of being in a study • Prevention? • Contamination bias: • Control group receives some component(s) of intervention • Prevention? • Confounding variables • Variables associated with intervention group and outcome, not in causal chain • Prevention? Lecture 17 (Oct 28,2004)

26. Internal vs external validity • Internal validity • Lack of bias in study • External validity • Generalizability • Representativeness of study sample Lecture 17 (Oct 28,2004)