Outline of Stratification Lectures

1. Outline of Stratification Lectures • Definitions, examples and rationale (credibility) • Implementation • Fixed allocation (permuted blocks) • Adaptive (minimization) • Rationale - variance reduction

2. Stratification • A procedure in which factors known to be associated with the response (prognostic factors) are taken into account in the design (e.g., randomization) • Pre-stratification refers to a stratified design; post-stratification refers to the analysis

3. Pre- versus Post Stratification and Precision (Variance Reduction) • As a general rule, the precision gained with pre- versus post-stratification is less than one might expect • The gain in precision is greatest in small studies (where you need it the most) because the risk of chance imbalance is greater. • Covariate adjustment for prognostic factors is usually carried out with regression (e.g., linear, logistic, or proportional hazards regression.

4. Stratification Can Increase Precision Simple versus stratified random sampling. Snedecor and Cochran note (p. 520): “If we form strata so that a heterogeneous population is divided into parts each of which is fairly homogeneous, we may expect a gain in precision over simple random sampling”. Ref. Snedecor and Cochran, Statistical Methods

5. Stratification Can Increase Precision Randomized block versus completely random design. Snedecor and Cochran note (p. 299): “Knowledge (about predictors or response) can be used to increase the accuracy of experiments. If there are a treatments to be compared,…first arrange the experimental units in groups of a, often called replications. The rule is that units assigned to the same replication should be as similar in responsiveness as possible. Each treatment is then allocated by randomization to one unit in each replication…Replications are therefore usually compact areas of land…This experimental plan is called randomized blocks.”

6. Pre-stratification Does Not Matter. Peto et al note: “As long as good statistical methods,…,are used to analyze data from clinical trials, there is no need for randomization to be stratified by prognostic features.” • Keep it simple so investigators are not discouraged from participating. • Post-stratified analysis is needed with pre-stratification anyway. • Improvement in sensitivity (precision) with pre-stratification compared to letting stratum sizes be determined by chance is small. Peto R et al., Br. J Cancer, pp. 585-612,1976

7. m1A m1B m1 m2A m2B m2 m3A m3B m3 m4A m4B m4 • Typical situation: m1 ≠ m2 ≠ m3 ≠ m4 • Study is designed/powered based on na and nb • Goal: miA = miB for all i. Stratified Design for Comparing Treatments Treatment Stratum A B 1 2 3 4 na nb

8. How much of a price does one pay with respect to precision by trusting randomization to achieve reasonable balance? Consider the relative efficiency of a stratified design to an unstratified design: Var (treatment contrast with stratification) RE = Var (treatment contrast with no stratification in design, but post-stratified analyses)

9. Pooling Estimates Estimates: E , E Var (E ) =  Var (E ) =  w E + w E Pooled Estimate: Best Pooled Est: w =  , w =  w  + w  Variance Pooled Est: 1 2 2 1 2 2 1 2 1 1 2 2 w + w 2 1 1 1 2 1 2 2 2 1 2 2 2 2 2 1 2 1 2 (w + w ) 1 2 1

10. n g n h + n g B A A n n h n n g B B A A n h n g B A Continuous response, equal variance - effect of chance imbalance -1 n h   +     nA = total number randomly assigned to A nB = total number randomly assigned to B g = fraction of those given A with prognostic factor h = fraction of those given B with prognostic factor B RE = 1 - + (1-h) + (1-g) Treatment A B S1 Stratum S2 nB (1-h) nA (1-g) nA nB

11. 2  (y - y ) = A B n g A n n B A RE obtained by noting: ( ) 1 + n g 1 n h 1A 1B A B + 1) Var( y - y ) = ) ( 1 1 2 n n  2) Var(y - y ) = (1-h) 2A (1-g) 2B A B 3) Pooled variance is: 2 2 wi (y - y ) Var iA iB VarPooled 2 wi n g n h B A w1 = n h + B (1-g) (1-h) w2 = + (1-h) n (1-g) n B A

12. g n n B A + n n B A w = 1 (1 - g) n n A B w = 2 + n n A B For Stratified Design, g = h

13. n n = A B Assume e.g., block randomization used       0.10, 0.05 0.99 0.25, 0.125 0.97 0.50, 0.25 0.93 0.75, 0.375 0.86 g + h -1 g + h RE = 1 - 2 g(1-g) + h(1-h) Consider the case of g = 2h: g, h RE

14. Bernouli Response h2 n2 1 - Loss of efficiency = h = lack of balance 2n = number in each stratum Ref: Meier (Controlled Clinical Trials, 1981)

15. B A 1 1 This can be seen by noting: 1) Stratified design, for stratum 1 1 n 1 n (p - p ) = ( ) Var p1q1 + A B 1 1 2 n = ( ) ( ) p1 q1 since n = n = n A1 B1 Note: q1 = 1- p1

16. 2) No stratification in design; post-stratification in analysis The ratio of these variances is proportional to: 1 n-h 1 n+h (p - p ) = ( ) Var p1q1 + A B1 1 2/n h2 n2 = 1 - 1/n+h + 1/n-h

17. A B 1 1 n = 10 1 (11, 9) 0.99 2 (12, 8) 0.96 4 (14, 6) 0.86 5 (15, 5) 0.75 (n , n ) RE h

18. Example: Brown et al. Clinical Trial of Tetanus Anti-toxin in Treatment of Tetanus. Lancet, 227-30;1960 (see also Meier, Cont Clin Trials, 1981; a slightly different approach is taken here). No Anti- Toxin (B) Anti- Toxin (A) 30 9 Alive 21 20 49 Dead 29 79 41 38 ^ 49 79 p = overall death rate = = 0.620

19. ^ p A ^ = 29/38 = 0.763 p B ^ ^ p p - = -0.275 A B ^ ^ ^ ^ ( ) ( ) Var 1 n 1 n = [ ] 1 p p p p - - + A B A B ^ ^ p p - A B = 20/41 = 0.488 [ ] ( ) ( ) 49 79 1 41 1 38 49 79 = + 1 - = 0.01195 ( ) = SE 0.109

20. Time from first symptoms to admission turned out to be an important prognostic factor; therefore, post-stratification was carried out. < 72 Hours ≥ 72 Hours B A B A 11 4 5 10 Alive Alive 18 26 Dead Dead 3 2 30 13 8 28

21. ^ ^ p p = 0.643 = 0.866 1A 1B ^ p = 0.759 1 ^ ^ = - 0.223 p p - 1A 1B ^ ^ p p - 1A 1B ^ p = 0.238 2 ^ ^ = - 0.221 p p - 2A 2B ^ ^ p p - 2A 2B Stratum 1: < 72 hoursStratum 2: ≥ 72 hours SE( ) = 0.112 SE( ) = 0.191

22. Let G = fraction of patients in Stratum 1 = = 58/79 0.734 ˆ ˆ 2 p p G + (1 - ) VAR( - ) = 0.00938 2A 2B ˆ ˆ p - p ) Weighted diff. ( w A B ˆ ˆ ˆ ˆ ˆ p p ˆ ˆ ˆ ˆ ( p - p ) = G ( p - p ) + (1 - G)( - ) 2B 2A w 1B 1A A B = - 0.223 compared to - 0.275 unweighted ˆ 2 ˆ ˆ ˆ ˆ VAR(p - p ) = G VAR(p - p ) 1A w 1B A B ˆ ˆ ˆ SE(p - p - ) = .097 A w B

23. Gain in precision achieved with post-stratification Var(post-stratification) Var(no stratification) RE = 0.00938 0.01195 = = 0.78 22% reduction

24. How much gain in precision would be achieved if stratification was used in the design?

25. Force balance within stratum < 72 Hours ≥ 72 Hours B B A A Alive Alive Dead Dead 29 10 29 11 ˆ p ‘s don’t change Assume ij ˆ SE(p - p ) = 0.109 instead of 0.112 SE(p - p ) = 0.186 instead of 0.191 ˆ 1B 1A ˆ ˆ 2B 2A

26. SE stratified design = 0.096 (same weights are used) Var(stratified design) Var(no stratification) RE1 = 2 (0.096) (0.109) = = 0.77 23% reduction 2 Var(stratified design) Var(post-stratification) RE2 = 2 (0.096) (0.097) = = 0.98 2% reduction 2

27. Gp1.(1-p1.) + (1-G)p2.(1-p2.) [ Gp1. + (1-G)p2. ] [1 – Gp1. – (1-G)p2.] RE = If p1. = p2. Then RE = 1 0.50 0.60 0.30 0.91 0.20 0.60 0.30 0.94 0.10 0.60 0.30 0.96 0.50 0.60 0.20 0.83 0.20 0.60 0.20 0.87 0.10 0.60 0.20 0.92 0.50 0.10 0.05 0.991 0.20 0.10 0.05 0.992 0.10 0.10 0.05 0.996 G P1. P2. RE

28. The reduction in variance achieved with post-stratification depends on: 1) the distribution of the prognostic factor in the population; 2) the relative strength of the prognostic factor; and 3) the expected endpoint rate in the group studied.

29. Scott’s Survey of Trials Published in Lancet and N Eng J Med in 2001 Stratification Permuted block 43/150 Minimization 6/150 Other adaptive 3/150 Other 19/150 Unspecified 79/150 Scott et al. Cont Clin Trials 2002; 23:662-674

30. Kahan’s Survey of 258 Trials Published in Four Major Medical Journals in 2010 Method of Randomization Simple: 4 Permuted blocks, no stratification: 40 Permuted blocks, stratification: 85 Minimization: 29 Other: 4 Unclear: 96 Kahan BC et al. BMJ 2012; 345:e5840

31. Conclusions 1. Usually there is little loss of efficiency with post-stratification as compared to a stratified design. 2. Loss of efficiency results from large chance imbalances for important prognostic factors, which are more likely in small studies. • Stratified designs should be considered in small studies (n < 50) with important prognostic factors. • Strictly speaking, analysis should account for pre-stratification.

32. Recommendation for Multi-Center Trials:Always Consider Stratification on Center 1. Clinic populations differ. 2. Treatment differs from clinic to clinic. 3. Each center represents a replicate of overall trial – can investigate treatment x clinic interactions. 4. In some trials (surgery), it may be better to stratify on surgeon within clinic. 5. If there are a very large number of clinical sites, small block size may have to be used and site combined into a priori defined larger strata (e.g., region or country) for analysis

33. General Recommendations • Large trials • Block randomization with stratification by center • Stratification on other factors not necessary (I am a lumper) • If needed, usually okay to carry out block randomization within each stratum • Small trials • Block randomization with stratification by center • If stratification on other factors is considered, may have to use an adaptive approach These are consistent with Freidman, Furberg and DeMets (see page 111)