Randomized Trials: the Evidence in “Evidence-Based”

1. Randomized Trials: the Evidence in “Evidence-Based” • Today • Randomized trials: why bother? • Randomization • Selection of participants (Inclusion/exclusion) • Design options for trials • Dennis Black, PhD • Dblack@psg.ucsf.edu • 597-9112

2. Randomized Controlled Trial (RCT) A study design in which subjects are randomized to intervention or control and followed for occurrence of disease • Experimental (as opposed to observational) Definitive test of intervention Confounders are equally distributed across intervention groups • Treated not younger, richer, healthier, better dieters

3. Examples of interventions • Drug vs. placebo • Low fat diet vs. regular diet • Exercise vs. CPP

4. Number of randomized trials published* 8000 7000 6000 5000 4000 3000 2000 1986 1988 1990 1992 1994 1996 1998 * Based on Medline search for “Randomized”

5. Disadvantages of RCTs • Expensive • Time Consuming • Can only answer a single question So, why bother?

6. Alternatives to RCTs(30 second Epi. Course) • Case-control studies • Compare those with and without disease • Cohort studies (prospective) • Identify those with and without risk factor • Follow forward in time to see who gets disease • Cohort and case-control are observational (not experimental)

7. Reasons for doing RCTs • Only study design that can prove causation • Required by FDA (and others) for new drugs and some devices • Most influential to clinical practice

8. Example: Estrogen Replacement Therapyin post-menopausal women • Important therapeutic question • Applies to 30 (?) million women in US • Prempro (estrogen/progestin combo)may be most prescribed drug in US • Potentially huge impact on public health • Complex, ERT effects multiple diseases

9. Estrogen Replacement Therapy (ERT) Disease Effect on Risk* Coronary heart disease Decrease by 40 - 80%Osteoporosis (hip fx) Decrease by 30 - 60%Breast cancer Increase by 10 - 20%Endometrial cancer Increase by 700% Alzheimer’s Decrease by ? Pulmonary embolism & Increase by 200 - 300%deep vein thrombosis * From observational (case-control and cohort) studies

10. Nurses Health Study (NEJM, 9/12/91) • Prospective cohort study, n = 48,470 • 337,000 person years of follow-up Risk of Major Estrogen Use Coronary Disease* Relative Risk** Never Used 1.4 1.0 Current user 0.6 0.56 (0.40-0.80) Former user 1.3 0.83 (0.65-1.05) * Events per 1000 women-years of follow-up** Relative Risk (95% CI) compared to never users

11. Meta-analysis of ERT, Published ~4/10/97 “Benefits (for CHD, osteoporosis) outweigh risks (breast cancer) and side effects…All post-menopausal women should be taking ERT”* * CNN, 4/10/97

12. Virtually all estrogen results arebased on observational data • Women chose to take ERT • Are ERT users different from non-users? • Age • Health status • More exercise • Health behaviors (see Dr.) • SES • Try to adjust in analysis, but may not be possible • Randomized trials alleviate these problems

13. Heart and Estrogen-Progestin Replacement Study (HERS) • Secondary prevention of heart disease • HRT (Prempro) vs. placebo (4-5 years) • ~ 2763 women with established heart disease • Postmenopausal, < 80 years, mean age 67 • 20 clinical centers in U.S./UCSF Coordinating center • Funding by Wyeth-Ayerst (post-NIH refusal) • Expected results???? • Real results: JAMA: 8/98

14. HERS: Summary of results Endpoint Placebo HRT RR P New CHD 176 172 0.99 0.91 Any fracture 138 130 0.95 0.70 Conclusion: Randomized trials can lead to big surprises!

15. Other surprises:Beta Carotene and cancer • Strong suggestions that beta carotene would prevent cancer 1. Observational epi. (diets high in fruits and vegetables with beta carotene lower cancer risk) 2. Pathophysiology • Clinical trials needed to establish cause and effect

16. Beta carotene: Clinical trial #1 The Alpha-Tocopherol, Beta CaroteneCancer Prevention Study RQ: Do vitamin E and beta-carotene prevent lung cancer in smokers? Design: RCT, factorial, 6.1 years Subjects: 29,133 smokers, Finnish men aged 50-69 Intervention: 1. Alpha-tocopherol, 50 mg/day vs. placebo(factorial) 2. Beta-carotene, 20 mg/day vs. placebo Outcome: Lung cancer incidence

17. Beta-carotene: Clinical Trial #1Results Beta-Carotene Control RR* Lung Cancer Cases 56.3 47.5 1.19 Lung Cancer Deaths 35.6 30.8 1.16 * Relative risk: Beta carotene vs. control Incidence per 10,000 person years

18. Beta carotene: Clinical trial #2 The Beta-Carotene and Retinol Efficacy Trial (CARET) RQ: Do vitamin A and beta-carotene prevent lung cancer in smokers? Design: RCT, 4.0 years Subjects: 18,314 men, smokers or asbestos workers Intervention: Retinol (25,000 IU) and beta carotene (15 mg) vs. placebo Outcome: Lung cancer incidence

19. Beta-carotene: Clinical Trial #2Results Lung Cancer* Death (all causes)* All Subjects 1.28 (1.04-1.57) 1.17 (1.03-1.33) Asbestos-exposed 1.40 (0.95-2.07) 1.25 (1.01-1.56) Smokers 1.23 (0.96-1.56) 1.13 (0.96-1.32) * Relative Risk (95% CI), treatment vs. placebo

20. Beta Carotene RCTs • Beta carotene not recommended for cancer prevention • Similar story for beta carotenes and heart disease • RCT’s very useful

21. Examples of major breakthroughs from RCTs • Protease inhibitors and AIDS • Aspirin and heart disease • Lipid lowering (statins) and heart disease

22. Steps in a “Classical” Randomized, Controlled Trail (RCT) 1. Select participants 2. Measure baseline variables 3. Randomize (to 1 or more treatments) 4. Apply intervention 5/6. Follow-up--measure outcomes Most commonly: one treatment vs. control Can be used for various types of outcomes (binary, continuous)

23. Randomization • Key element of RCT’s • Assure equal distribution of both... • measured/known confounders • unmeasured/unknown confounders • Important to do well • True random allocation • Tamper-proof (no peaking, altering order of participants, etc) • Simple randomization • Low tech • High tech

24. Other types of randomization • Blocking*: equal after each n assignments • e.g., block size of 4, treatments a and b abab aabb bbaa baab • Assure relatively equal number of ppts. to each treatment • Disadvantages of blocking • Size of block: 2 treatments--4 or 6 • Very commonly used *Formally: random, permuted blocks

25. Randomization to balance prognostic variables • Stratified permuted blocks • Blocks within strata of prognostic variable • e.g., HRT study of prevention of MI. High LDL at much higher risk--want to avoid more higher LDL in placebo. • Stratum High LDL: aabb baba … Normal LDL: baab abab …. • Limited number of risk factors • Very commonly used in multicenter studies to balance within clinical center • Fancier techniques for assuring balance • Adaptive randomization (not much used)

26. Implementation of randomization • Less challenging for blinded studies • Sealed envelopes in fixed order at clinical sites • Alternatively: list of drug numbers • a b a b b b a a • 1 2 3 4 5 6 7 8 • Clinic receives bottles labeled only by numbers--assign in order • Unblinded studies: important to keep next assignment secret • Problem with blocks within strata

27. Who to Study: Principles for Inclusion/exclusion • Widest possible generalizability • Sufficiently high event rate (for power to be adequate) • Population in whom intervention likely to be effective • Ease of recruitment • Likelihood of compliance with treatment and FU

28. Explicit criteria for inclusion in a trial • Typically written as “inclusion/exclusion” criteria in protocol • The more explicit the better • Want centers or investigators to be consistent • Examples of exclusion decisions • 1. Women with heart disease vs. Women with CABG surgery or documented MI by ecg (criteria) or enzymes (criteria) • 2. Users of estrogen vs Use of ERT for more than 3 months over last 24 mos.

29. Valid reasons to exclude participants (Table 10.1) • Treatment would be unsafe • Adverse experience from active treatment • “Risk” of placebo (SOC) • Active treatment cannot/unlikely to be effective • No risk of outcome • Disease type unlikely to respond • Competing/interfering treatment (history of?) • Unlikely to adhere or follow-up • Practical problems

30. Design-a-trial: Inclusion criteria options for HRT • Study HRT and prevention of heart disease, 4 years (HERS-like) • Women over age 50 years • Women over 60 years • Women over 75 years • Women with existing heart disease • Generalizability? • Feasible sample size? • Population amenable to intervention? • Logistic difficulties (recruitment? cost? adherence)

31. HERS inclusion options • HERS trial options (event rate) • Women over age 50 years (0.1%/year) • Women over 60 years (0.5%/year) • Women over 75 years (1%/year) • Women with existing heart disease (4%/year)

32. HERS inclusion options • HERS trial options (event rate) [n required] • Women over age 50 years (0.1%/year) [55,000] • Women over 60 years (0.5%/year) [45,000] • Women over 75 years (1%/year) [34,000] • Women with existing heart disease (4%/year) [3,000] (Choose last option as most practical: common to generalize from secondary to primary prevention)

33. Exclusions/inclusions examples • Important impact on generalizability of both efficacy and safety • Example: Fracture Intervention Trial (FIT) • Study of alendronate (amino-bisphosphonate) vs. placebo in women with low bone mass • 6459 women randomized to alendronate or placebo • Fracture endpoint • Upper GI and esophagitis concerns with bisphosphonates, esp. aminos • Who to exclude?

34. FIT inclusion/exclusion example • Alendronate studies (pre-FIT) excluded: • Any history of upper GI events • Any (remote) history of ulcer • Esophagial problems, etc. • Reports of upper GI problems in clinical practice: 5 to 20% of patients stop alendronate. Due to: • Use by “real world” patients? • Use in real world? • Psychological--due to warnings about potential problems

35. Inclusion may impact effect of treatment • FIT: reduction in hip fractures only among those with more severe osteoporosis BL BMD T-scoreRR for hip fracture - 2.0 to -1.6 0.98 - 2.5 to -2.0 1.05 < -2.5 0.6 (p<.01) • Similar findings in statin trials: higher lipids, more benefit

36. Alternative RCT designs: Factorial design • Test of more than one treatment (vs. placebo) • Each drug alone and in combination • Allows multiple hypotheses in single trial • Efficient (sort of) • e.g., Physician’s Health Study • Test aspirin ==> MI • beta caratene ==> cancer

37. Factorial design: Physician’s Heath Study Placebo Beta-carotene Aspirin vs. no aspirin (MI) Aspirin plus Beta-carotene Aspirin Beta carotene vs. no beta carotene (cancer)

38. Factorial design assumptions/limitations • Treatments do not interact • Effect of aspirin on MI is same with and without beta-carotene • Difficult to prove, requires large sample • Women’s Health Initiative (MOAS, $600M +) • Estrogen vs. placebo (all outcomes) • Calcium/Vit D vs. placebo (fractures) • Effect of calcium is the same/additive with and without estrogen..very shaky • Best used for unrelated RQ’s (both treatments and outcomes)

39. Cross-over designs • Both treatments are administered sequentially to all subjects • Subject serves as own control, random order • Compare treatment period vs. control period • Diuretic vs. beta blocker for blood pressure • 1/2 get d followed by bb • 1/2 get bb followed by d

40. Cross-over assumptions/limitations • Continuous variables only • No order effects • No carry-over effects • Need quick response and quick resolution • “Wash out” period helpful • More commonly used in phase I/II

41. Other special designs • Matched pairs randomized • One of each pair to each treatment • e.g., two eyes within an individual (one to each treatment) • Diabetic Retinopathy study

42. Other special designs • Cluster or grouped randomization • Randomize groups to treatments • Often useful especially for public health-type interventions

43. Other special designs (clusters) • Cluster or grouped randomization examples • medical practices to stop-smoking interventions • cities to public health risk factor reduction (5 Cities Project) • baseball teams to chewing-tobacco intervention • Analysis complex • Sample size complex: true n is between n clusters and n individuals (closer to clusters)

44. Previews of coming attractions • Blinding, interventions, controls (placebo vs. active) • Follow-up, compliance, analysis • Outcomes (efficacy and adverse effects) • Ethical issues (many!!) • Nuts and bolts • Multiple hypothesis testing • Working with the evil empire (drug cos)