1 / 24

The Power of Pairing in Cluster Randomized Block Designs: A Monte Carlo Simulation

The Power of Pairing in Cluster Randomized Block Designs: A Monte Carlo Simulation. Nianbo Dong & Mark Lipsey 03/04/2010. Power Analysis for Group-Randomized Experiments. Two Big Design Families (Kirk, 1995) 1) Hierarchical Design 2) Cluster Randomized Block Design

Mia_John
Download Presentation

The Power of Pairing in Cluster Randomized Block Designs: A Monte Carlo Simulation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Power of Pairing in Cluster Randomized Block Designs: A Monte Carlo Simulation • Nianbo Dong & Mark Lipsey • 03/04/2010

  2. Power Analysis for Group-Randomized Experiments Two Big Design Families (Kirk, 1995) 1) Hierarchical Design 2) Cluster Randomized Block Design Three Ways to do Power Analysis 1) Software, e.g., Optimal Design 2.0 (Spybrook, Raudenbush, Congdon, & Martinez, 2009) 2) MDES formula (Bloom, 2006; Schochet, 2008) 3) Power table using operational effect size (Hedges, 2009; Konstantopoulos, 2009)

  3. Matched-Pair Cluster-Randomized Design (1) 3. But, the gain in predictive power may outweigh the loss of degrees of freedom (Billewicz, 1965; Bloom, 2007; Hedges, 2009; Martin, Diehr, Perrin, & Koepsell, 1993; Raudenbush, Martinez, & Spybrook, 2007) 4. Break-even point using MDES for fixed pair effect model (Bloom, 2007) Advantages 1) Avoiding bad randomization, 2) Face validity 2. Cost: Loss of degree of freedom

  4. Matched-Pair Cluster-Randomized Design (2) • MDES for 2-level hierarchical design, w/o covariance adjustment • (Bloom, 2006) 5. MDES Comparison • MDES for 3-level matched-pair cluster-randomized design, random pair effect model w/o covariance adjustment, VC (SE from Raudenbush & Liu, 2000) : ICC for hierarchical design; : Pair-level ICC for matched-pair cluster-randomized design J: # of clusters; n: average # of individuals

  5. Matched-Pair Cluster-Randomized Design (2) • MDES for 2-level hierarchical design, w/o covariance adjustment • (Bloom, 2006) 5. MDES Comparison • MDES for 3-level matched-pair cluster-randomized design, random pair effect model w/o covariance adjustment, VC (SE from Raudenbush & Liu, 2000) : ICC for hierarchical design; : Pair-level ICC for matched-pair cluster-randomized design J: # of clusters; n: average # of individuals

  6. Conceptual Model: Options & Decisions L1 L2 L3 L4

  7. Conceptual Model: Options & Decisions L1 L2 L3 L4

  8. Research Questions The Overall Question Are there design and analysis options other than increasing the sample size that might keep pre-randomization matching from degrading power relative to the analogous unmatched design?

  9. How much difference does it make to statistical power: • 1. If we are able to make close matches or unable to do so? • 2. If we are treating the pairwise blocks as fixed effects vs. random effects? • 3. If we ignore the pairwise blocking entirely (and does this compromise the Type I error rate)? • 4. If we also use the blocking variable as a covariate in the analysis? Four Sub-Questions

  10. Simulation: Parameter Combinations

  11. Results: Median and Minimum Power on Each Branch Median/Min SE =.009-.016

  12. Results: Median and Minimum Power on Each Branch Median/Min SE =.009-.016

  13. Results: Median and Minimum Power on Each Branch Median/Min SE =.009-.016

  14. Results: Median and Minimum Power on Each Branch Median/Min SE =.009-.016

  15. Results: Median and Minimum Power on Each Branch Median/Min SE =.009-.016

  16. Results: Median and Minimum Power on Each Branch Median/Min SE =.009-.016

  17. Results: Median and Minimum Power on Each Branch Median/Min SE =.009-.016

  18. Results: Median and Minimum Power on Each Branch Median/Min SE =.009-.016

  19. Results: Median and Minimum Power on Each Branch Median/Min SE =.009-.016

  20. Conclusions • The most important technique for maintaining power is to also use the matching variable as a covariate. The advantages of pre-randomization matching do not have to come at the cost of reduced power– even when the matching is not very good. • The random effects model does not necessarily have less power than the fixed effects alternative. • Ignoring the pairwise blocking variable in the analysis, though not faithful to the actual design used, does not appear to cause problems with either the Type I or Type II error rate. (Consistent with Diehr, Martin, Koepsell, & Cheadle, 1995; Lynn & McCulloch, 1992; Proschan, 1996)

  21. Thanks

  22. Appendix 1:

  23. Appendix 2:

  24. Appendix 3: MDES for Matched-Pair Cluster-Randomized Design

More Related