Group-Randomized Trials

1. Group-Randomized Trials • Distinguishing characteristics • The unit of assignment is an identifiable group. • Different groups are allocated to each condition. • The units of observation are members of the groups. • The number of groups allocated to each condition is usually limited.

2. Group-Randomized Trials • Primary advantages • Randomization provides the statistical basis for the assumption of independence at the group level. • With proper randomization and enough groups… • Potential sources of bias are fairly distributed across the study conditions. • Inferences based on a valid analysis can be as strong as those obtained from a randomized clinical trial. • For this reason, the GRT is the gold standard for design in public health and medicine when allocation of identifiable groups is necessary.

3. Group-Randomized Trials • Primary disadvantages • There is extra variation attributable to the groups. • This increases the standard error of the intervention effect. • The number of groups is often limited. • This limits error degrees of freedom and reduces the effectiveness of randomization. • Together, these problems can threaten internal validity and power in small trials. • At the same time, logistics and cost can threaten the viability of large trials.

4. The Warning • Randomization by cluster accompanied by an analysis appropriate to randomization by individual is an exercise in self-deception, however, and should be discouraged. Cornfield (1978)

5. Progress Addressing the Major Disadvantages • Extra variation • Regression adjustment for covariates • Modeling time • Limited degrees of freedom • Proper planning • Borrowing information from similar studies

6. Other Recent Methodological Developments • Analysis methods • Small sample corrections for GEE • Methods for survival analysis • Improved power for permutation tests • Mediation models • Missing data • Nonlinear mixed models • Reference • Murray, D. M., Varnell, S. P., & Blitstein, J. L. (2004). Design and analysis of group-randomized trials: A review of recent methodological developments. AJPH, 94(3), 423-432.

7. State of the Practice • How are we doing? • Sample size estimation • Analysis methods • 58 GRTs published in AJPH and Prev Med, 1998-2002 • Reference • Varnell, S., Murray, D. M., Janega, J. B., & Blitstein, J. L. (2004). Design and analysis of group-randomized trials: A review of recent practices. AJPH,94(3), 393-399.

8. Findings • Only 16% reported evidence of using appropriate methods for sample size calculations. • Similar to 19% in Simpson et al., 1995 review • 54% reported only analyses judged to be appropriate. • Similar to 57% in Simpson et al., 1995 review • That figure rises to 68% if we use Simpson et al.'s criteria • 19% reported only analyses deemed invalid. • 26% reported a mix of methods. • Some progress since 1995, but still need for improvement, especially with regard to more recent findings.

9. Summary and Recommendations • Many questions do not require a GRT. • Formative research • Other questions may require a GRT design but not a full-blown GRT analysis. • Feasibility or preliminary evidence of efficacy • Here, estimate effect and/or ICC, but avoid causal inference. • Other questions cannot be addressed using a GRT. • Policy change at large aggregate level • Use quasi- or natural experiments if they will provide useful information efficiently. • Do not ignore the design and analytic challenges inherent in nested designs. • For efficacy studies, use individual-level randomization whenever possible.

10. Summary and Recommendations • Reserve GRTs for situations in which… • Experimental evidence is needed for causal inference, • Individual randomization is not possible, • There is preliminary evidence for feasibility and efficacy, • There is sufficient information available to size the study. • Use the available tools to do the best job possible. • Match the design to the research question. • Use appropriate analysis methods. • Ensure adequate power. • Choose a valid analysis that minimizes the SE for the intervention effect. • Build in sufficient replication at the group level.