How many patients do I need for my study? Realistic Sample Size Estimates for Clinical Trials

1. How many patientsdo I need for my study? Realistic Sample Size Estimates for Clinical Trials

2. Sample Size Estimation 1.General considerations 2. Continuous response variable • Parallel group comparisons • Comparison of response after a specified period of follow-up • Comparison of changes from baseline • Crossover study 3. Success/failure response variable • Impact of non-compliance, lag • Realistic estimates of control event rate (Pc) and event rate pattern • Use of epidemiological data to obtain realistic estimates of experimental group event rate (Pe) 4. Time to event designs and variable follow-up

3. Useful References Lachin JM, Cont Clin Trials, 2:93-113, 1981 (a general overview) Shih J, Cont Clin Trials, 16:395-407, 1995(time to eventstudies withdropouts,dropins, and lag issues) – see size program on biostatistics network Farrington CP and Manning G, Stat Med, 9:1447-1454, 1990 (sample size for equivalence trials) Whitehead J, Stat Med, 12:2257-2271, 1993 (sample size for ordinal outcomes) Donner A, Amer J Epid, 114:906-914, 1981 (sample size for cluster randomized trials)

4. Key Points • Sample size should be specified in advance (often it is not) • Sample size estimation requires collaboration and some time to do it right (not solely a statistical exercise) • Often sample size is based on uncertain assumptions (estimates should consider a range of values for key parameters and the impact on power for small deviations in final assumptions should be considered) • Parameters that do not involve the treatment difference (e.g., SD) on which sample size was based should be evaluated as part of interim monitoring • It pays to be conservative; however, ultimate size and duration of a study involves compromises, e.g., power, costs, timeliness.

5. Some Evidence that Sample Size is Not Considered Carefully: A Survey of 71 “Negative” Trials(Freiman et al., NEJM, 1978) Authors stated “no difference” P-value > 0.10 (2-sided) Success/failure endpoint Expected number of events >5 in control and experimental groups Using the stated Type I error and control group event rate, power was determined corresponding to: 25% difference between groups 50% difference between groups

6. Frequency Distribution of Power Estimates for 71 “Negative” Trials 25% Reduction 25 20 15 Number 10 5.63% 5 0 0-9 10-19 20-29 30-39 40-49 50-59 60-69 70-79 80-89 90-99 Power (1 - ß) References: Frieman et al, NEJM 1978.

7. Frequency Distribution of Power Estimates for 71 “Negative” Trials 50% Reduction 29.58% 25 20 15 Number 10 5 0 0-9 10-19 20-29 30-39 40-49 50-59 60-69 70-79 80-89 90-99 Power (1 - ß) References: Frieman et al, NEJM 1978.

8. Implications of Review by Frieman et al. Many investigations do not estimate sample size in advance Many studies should never have been initiated; some were stopped too soon “Non-significant” difference does not mean there is not an important difference Design estimates (in Methods) are important to interpret study findings Confidence intervals should be used to summarize treatment differences

9. Studies with Power to Detect 25% and 50% Differences 6 50 25% Difference l l 50% Difference 45 l 40 35 30 l Percent of Studies with at Least 80% Power 25 l 6 20 15 6 6 10 6 5 0 1975 1980 1985 1990 Moher et al, JAMA , 272:122-124,1994

10. These Results Emphasize the Importance of Understanding that the Size of P-Value Depends on: • Magnitude of difference (strength of association); and • Sample size “Absence of evidence is not evidence of absence”, Altman and Bland, BMJ 1995; 311:485.

11. Steps in Planning a Study 1) Specify the precise research question 2) Define target population 3) Assess feasibility of studying question (compute sample size) 4) Decide how to recruit study participants, e.g., single center, multi-center, and make sure you have back-up plans

12. Beginning: A Protocol Stating Null and Alternative Hypotheses Along with Significance Level and Power Null hypothesis (HO) Hypothesis of no difference or no association Alternative hypothesis (HA) Hypothesis that there is a specified difference (Δ) No direction specified (2-tailed) A direction specified (1-tailed) Significance Level(): Type I Error The probability of rejecting H0 given that H0 is true Power = (1 - ): ( = Type II Error) Power is the probability of rejecting H0 when the true difference is Δ

13. End: Test of Significance According to Protocol Statistically Significant? Yes No Reject HO Do not reject HO Sampling variationis an unlikelyexplanation for thediscrepancy Sampling variationis a likelyexplanation for thediscrepancy

14. Normal Distribution If Z is large (lies in yellow area), we assume difference in means is unlikely to have come from a distribution with mean zero.

15. Continuous Outcome Example Observations:Many people have stage 1 (mild) hypertension (SBP 140-159 or DBP 90-99 mmHg) For most, treatment is life-long Many drugs which lower BP produce undesirable symptoms and metabolic effects (new drugs are needed) Research Can new drug T adequately control BP for patients Question: with mild hypertension? Objective: To compare new drug T with diuretic treatment for lowering diastolic blood pressure (DBP)

16. Parallel Group Design Comparing Average Diastolic BP (DBP) After One Year Hypothesis HO: DBP after one year of treatment with new drug T equals the DBP for patients given a diuretic (control) HA: DBP after one year is different for patients given new drug T compared to diuretic treatment (difference is 4 mmHg or more)

17. Parallel Group Design Comparing Average Difference (Year 1 – Baseline) in DBP. Hypothesis HO: DBP change from baseline after one year of treatment with new Drug T equals the DBP change from baseline after one year for patients given a diuretic (control) HA: DBP change from baseline after one year of treatment with new Drug T is different than the DBP change from baseline after one year for patients given a diuretic (control) treatment (difference is 4 mmHg or more)

18. Why Δ= 4 mmHg? An importantdifference on a population-wide basis Clinical trials (Lancet 1990;335:827-38) • 14 randomized trials; 36,908 participants • 5-6 mmHg DBP difference (treatment vs. control) • 28% reduction in fatal/non-fatal CVD • Observational studies (Lancet 2002;360:1903-13) • 58 studies; 958,074 participants • 5 mm Hg lower DBP among those 40-59 years • 41% (30%) lower risk of death from stroke (CHD)

19. Considerations in Specifying Treatments Effect (Delta) • Smallest difference of clinical significance/interest • Stage of research • Realistic and plausible estimates based on: • previous research • expected non-compliance and switchover rates • consideration of type of participants to be studied • Resources (compromise) Delta is a difference that is important NOT to miss if present.

20. Principal Determinants of Sample Size • Size of difference considered important (Delta) • Type I error () or significance level • Type II error (), or power (1- ) • Variability of response/frequency of event Constants

21. Sample Size for Two Groups: Equal AllocationGeneral Formula 2 x Variability x [Constant (,)]2 Delta2 N Per Group = Delta = Δ = clinically relevant and plausible treatment difference

22. Weighing the Errors   Type 2 error: Sponsor’s concern Type 1 error: Regulator’s Concern

23. Typical Values for (Z1-/2 + Z1- )2 Which Is Numerator of Sample Size Type I Error () or Significance Level (Z1-/2) 2-sided test 0.05 (1.96) 0.80 (0.84) 7.84 0.90 (1.28) 10.50 0.95 (1.645) 13.00 0.01 (2.575) 0.80 (0.84) 11.67 0.90 (1.28) 14.86 0.95 (1.645) 17.81 Power (1 - ) (Z1-) (Z1-/2 + Z1-)2

25. HO : 1 = 2 ; 1 - 2 = 0 HA : 1 ≠ 2 ; 1 - 2 = 4 mmHg ExampleHypertension Study HO HA mmHg 0 4   Usually formulated in terms of change from baseline (e.g., Ho = D1 - D2 = 0)

26. Derivation Solve for N using these 2 equations and by noting that Δ = sum of 2 parts from the previous figure .

27. Sources of Variability of BP Measurements Ref: Rose GA. Standardization of Observers in Blood Pressure Measurement. Lancet 1965;1:673-4. Recent physical activity Emotional state Position of subject and arm Room temperature and season of year Known factors True variations in arterial pressure Unknown factors Variability of blood pressure readings Inaccuracy of sphygmomanometer Instrument Cuff width and length Mental concentration Hearing acuity Confusion of auditory and visual Interpretation of sounds Rates of inflation and deflation Reading of moving column Measurement errors Chiefly affecting the mean pressure estimate Observer Distorting the frequency distribution curve (and sometimes affecting the mean) Terminal digit preference Prejudice, e.g., excess of readings at 120/80

28. 2 s Estimates of Variability for Diastolic Blood Pressure Measurements (MRFIT)Estimated Using Random-Zero (R-Z) Readings Estimate Variance Component (mmHg)2 Between Subject 58.4 Within Subjects 36.3   2   e

29. 2 s 2 v 2 e Estimates of Variability for Diastolic Blood Pressure MeasurementsEstimated Using Random-Zero (R-Z) Readingsat Screen 2 and Screen 3 in MRFIT(2 Readings at Each Visit) Estimate Variance Component (mmHg)2 Between Subject 58.4 Between Visits 26.1 Between Readings 10.2  Within subject analyzed further  

30. N per Group No. of No. of visitsreadings/visit∆ = 8∆ = 4 1 1 31 124 1 2 30 118 2 1 25 100 2 2 24 97 Between visit variability = 26.1 (mmHg)2 Within visit variability = 10.2 (mmHg)2 Consequences on Sample Size of Using Multiple Readings for Defining Diastolic BP =0.05, 1-=0.90 Inter-subject variability=58.4 (mmHg)2

31. Parallel Group Design Comparing Average DBP After One Year. Hypothesis HO: DBP after one year of treatment with new Drug T equals the DBP for patients given a diuretic (control) HA: DBP after one year is different for patients given new Drug T compared to diuretic treatment (difference is 4 mmHg or more)

32. Parallel Group StudiesComparing Average DBP After One Year1 measure, 1 visit (a=0.05, b= .10) s2=58.4 + 26.1+10.2=94.7 D=8 mmHg D=4 mmHg

33. Parallel Group Design Comparing Average Difference (Year 1 – Baseline) in DBP. Hypothesis HO: DBP change from baseline after one year of treatment with new Drug T equals the DBP change from baseline after one year for patients given a diuretic (control) HA: DBP change from baseline after one year of treatment with new Drug T is different than the DBP change from baseline after one year for patients given a diuretic (control) treatment (difference is 4 mmHg or more) (2-Tailed)

34. Sample Size for Two Groups: Equal AllocationGeneral Formula 2 x Variability x [Constant (,)]2 Delta2 N Per Group = Delta = Δ = clinically relevant and plausible treatment difference

35. Estimate of Variability for Change Outcome • Prior studies (For MRFIT, SD of DBP change after 12 months = 9.0 mmHg [baseline is one visit, 2 readings; follow-up is one visit, 2 readings]. For comparison, SD of 12 month DBP is 9.5 mmHg) • Use correlation(ρ) of repeat readings for participants to estimate e2. (For MRFIT, correlation of DBP at baseline and 12 months is 0.55; note that SD (diff) can be written as 2σT2 (1-ρ) = 2σe2 = 2(81)(1-0.55) = 72.9 (SD of change ≈ 8.5 mmHg) • Estimate of SD change using analysis of covariance (regression of change on baseline) (For MRFIT, SD = 7.9 mmHg)

37. Crossover Group Design Comparing Average Difference (Diuretic – Drug T) in DBP Hypothesis HO: Average of the two orders average differences between treatment and control is zero. HA: Average of the two orders average difference between treatment and control is not zero. (difference is 4 mmHg or more) (2-Tailed) Order: I Order: II

38. Var(dl) = 2 Var(dll) = 2 – – dl + dll Dl + Dll ∆ = TT - TC = E = 2 2 2 e 2 e Crossover Study Design Period 12Diff. I y1 y2 dl II y1 y2 dll

39. Dl + Dll HO = = 0 2 With parallel group comparison we had: HO : mT = mC or HO : DT = DC where DA and DB refer to the difference between follow-up and baseline levels of outcome With crossover we have: or equivalently: HO = TT – TC = 0

40. Variance forSample Size Formula:

41. Substitution intoSample Size Formula Gives: n| = n|| = number randomly allocated to each sequence - I (AB) or II (BA). This follows because the variance of the pooled treatment difference across the 2 sequences is ¼ (22e + 22 e)

42. Crossover Sample Size Compared to Parallel Design (no baseline)

43. Crossover Sample Size Compared to Parallel Design (no baseline) But the crossover design will require twice the number of measurements. So, ifρ= 0 then number of measurements are equal, but sample size for crossover is ½.

44. Consider an Experimentwith Diastolic BP ResponseType 1 error = 0.05 (2-sided) and Power = 0.95

45. nc n 2 2   s e Examples DBP (mmHg) 58 36 0.62 0.19 Cholesterol (mg/dl) 1200 400 0.75 0.125 Overnight urine 325 625 0.34 0.33excretion Na+(meq/8 hours) 2 overnights 325 312 0.51 0.24 7 overnights 325 90 0.78 0.11 

48.    Sample size for  = .05 (2-sided) and  = .05 0.4 0.6 0.8 1.0 Parallel Number/group 163 72 41 26(no baseline) Parallel Baseline 80 36 20 12number/group(r=0.75) Crossover(Number/seq.)  = 0.00 82 36 21 13 = 0.25 62 27 15 10 = 0.50 41 18 10 7 = 0.75 20 9 5 3

49. Key Points Sample size should be specified in advance Sample size estimation requires collaboration Often sample size is based on uncertain assumptions, therefore estimates should consider a range of values for key parameters (i.e., investigate the impact on power if sample size and treatment effect is not achieved) Parameters on which sample size is based should be evaluated as part of interim monitoring It pays to be conservative; however, ultimate size and duration of a study involves compromises, e.g., power, costs, timeliness.