Multiple problems

1. Multiple problems Keith J Barrington CHU Sainte Justine, Montréal

2. Twins and triplets in neonatal research • Neonatal research is unique in often enrolling large numbers of genetically identical or closely related subjects. • Particularly true when entry criteria are not very restrictive, such as by gestational age or birth weight • For some outcomes there is now evidence of a genetic influence, for other outcomes there may well be genetic effects

3. There are situations in other fields that are analogous • Enrolment in maternal treatment trials of mothers carrying twins • But they are always in the same group • Could analyze all data independently, just select the 1st twins outcomes, randomly select 1, analyze ‘by pregnancy’ • Enrolment of individuals with bilateral eye disease • Such as retinopathy, very highly correlated, but for local treatments, can be randomized separately, as a block or into different groups

4. The 2 eye problem • Statistical techniques for addressing the correlation within blocks of 2 have been developed for the eyes in ophthalmologic research • All eye pairs are ‘monozygotic’ • All eye pairs experience near identical environmental conditions.

5. ETROP study • Infants with bilateral eye disease (80%) • One eye randomized to early treatment • Other eye treated conventionally • Equivalent to randomizing twins to opposite groups • No correction for block • 2 consents, one for enhanced screening and another for randomization to treatment • 40 refusals and 401 randomized

6. What outcomes may be genetically influenced? • Lung disease • Hyaline Membrane Disease • Nagourney et al 1996 • Bronchopulmonary Dysplasia • Bhutani et al. Lavoie et al. • Brain Injury • IVH, less evidence • Bhutani et al not significant. • Retinopathy • Bizzarro et al • Patent DuctusArteriosus • Lavoie et al

7. What outcomes may be correlated? • Other outcomes may be correlated in twins because of shared intra-uterine environment • Chorioamnionitis • Pre-Eclampsia • Diabetes • Could all affect both twins (or all 3 triplets)

8. How does this affect research? • If there is a correlation in an outcome between twins, and the twins are in the same group, how does this affect trial design and analysis?

9. Gates and Brocklehurst BJOG 2004 • ‘Inclusion of non-independent data means that the ‘‘effective sample size’’ of the trial is reduced: there are fewer independent outcomes in the trial than the number of babies that took part in it. • Analysingall babies as if they are independent will therefore overestimate the sample size and give confidence intervals that are too narrow’

10. For example: • If you had a trial with 100 twins, who had perfectly concordant results: • You have in reality only 50 ‘subjects’ in the trial • Therefore much less power than you thought • But you have an ‘n’ of 100 for the analyses • Therefore artificially narrowing the confidence intervals

11. How important is this in reality? • In reality: • The magnitude of this effect will depend on: • The proportion of twins, the degree of concordance, the method of analysis, and the impact of the intervention being studied • Even if the intervention has no overall impact, if it changes the concordance between twins, may have a major effect on results • And depends on how many twins are in the same group, or opposite groups

12. Does independent randomization really decrease enrolment? • I have heard individual parents tell me that they only want their babies in the trial if they can get the same treatment. • I have also heard parents say they want their babies in opposite arms, to be sure that at least one gets the better treatment

13. Consort diagrams in neonatal research • Individually randomized versus block randomized multiples • Is there a difference in consent rate?

14. Consent rate by method of multiple randomization • Problem : often not clear from the publication how twins were enrolled. • Limitation: many other reasons for differential consent rates • Methods: calculated number of refusals of consent as a proportion of those approached for consent

15. Large RCTs • EUNO: individual randomization. • 610 refusals of 2227 approached • NOCLD: block randomization • 587 enrolled of 1555 eligible (?number of refusals) • SUPPORT: block randomization • 748 refusals of 2064 approached • CAP: individual randomization • 1628 refusals of 3634 approached • COIN: individual randomization • 233 refusals of 906 approached

16. Randomization Models • If infants are randomized to the same group, i.e. as a cluster, then the analysis should take account of that • If there is little correlation between twins (eg severe IVH) then a strict cluster approach, such as the multiple outputation technique will reduce power, and risk type 2 error • If there is strong correlation between twins (eg BPD) then not correcting for clusters will, in general, make confidence intervals more narrow, and risk type 1 error

17. Actual example • NOCLD trial • Supported by NIH, plan to use a multiple outputation technique to analyze because : • Block randomization (first of multiples to be enrolled was randomized, other(s) entered in the same arm) • Strong evidence of coherence in the primary outcome, BPD.

18. Published data • Survival without BPD • 129/294 with iNO • 105/288 infants with placebo • p = 0.03 RR= 1.26; 95% CI, 1.02 to 1.55 • Attempted Cochrane meta-analysis • If all the data are analyzed as if they were independent • p=0.08 RR = 1.12

19. Randomization models • If twins are randomized independently, then those who are in different groups do not require correction • Adaptive models which control for the degree of correlation among twins in the same group should be used • Such as GEE

20. Another example • If twins are individually randomized, then using stringent cluster techniques for all the pairs will decrease power • Schreiber et al • iNO 51/105 infants death or BPD • Placebo 65/102 • (relative risk, 0.76; 95 percent confidence interval, 0.60 to 0.97; P=0.03)

21. The iNO trials (MAPPINO)

22. Primary endpoint 1 Death or CLD (Best available definition) iNO 40 / 48 (83%) 6 / 16 (38%) 43 / 105 (41%) 42 / 61 (69%) 54 / 64 (84%) 170 / 224 (76%) 292 / 398 (73%) 4 / 20 (20%) 165 / 294 (56%) 134 / 399 (34%) 954 / 1629 (59%) Placebo 27 / 32 (84%) 4 / 18 (22%) 56 / 102 (55%) 51 / 84 (61%) 56 / 62 (90%) 174 / 225 (77%) 294 / 395 (74%) 8 / 20 (40%) 184 / 288 (64%) 137 / 401 (34%) 992 / 1627 (61%) Trial RR (95% CI) Kinsella 1999 0.99 (0.81, 1.21) Srisuparp 2002 1.59 (0.55, 4.62) Schreiber 2003 0.77 (0.57, 1.04) Hascoet 2005 1.11 (0.85, 1.43) INNOVO 2005 0.93 (0.82, 1.07) Van Meurs 2005 0.98 (0.88, 1.09) Kinsella 2006 0.99 (0.91, 1.08) Dani 2006 0.53 (0.19, 1.46) Ballard 2006 0.85 (0.74, 0.98) EUNO 2008 1.01 (0.83, 1.23) OVERALL* 0.96 (0.91, 1.01) p=0.095 0.2 0.5 1 2 5 † Subhedar removed from the analysis as zero cell counts caused model instability. Favours iNO Favours placebo * χ2test for heterogeneity p > 0.05 Estimates derived from N=1000 iterations of log-binomial model using multiple outputation method.

23. Implications? • Do not use multiple outputation for independently enrolled infants • The impact of analyzing multiples in the same group as if they were independent are dependent on how highly correlated the outcomes are, and what proportion of twins.

24. Gates and Brocklehurst • Re-analysis of the antenatal TRH trial (20% multiples) • Monte Carlo simulation of trials with 33% multiples

27. Analysis of binary outcomes from randomised trials including multiple births: when should clustering be taken into account?Yelland et al 2011A= intrapair correlation of 0.1b=intrapair correlation of 0.5c=intrapair correlation of 0.9 Paediatric and Perinatal EpidemiologyVolume 25, Issue 3, pages 283-297, 6 APR 2011 DOI: 10.1111/j.1365-3016.2011.01196.xhttp://onlinelibrary.wiley.com/doi/10.1111/j.1365-3016.2011.01196.x/full#f1

28. Analysis of binary outcomes from randomised trials including multiple births: when should clustering be taken into account? Paediatric and Perinatal EpidemiologyVolume 25, Issue 3, pages 283-297, 6 APR 2011 DOI: 10.1111/j.1365-3016.2011.01196.xhttp://onlinelibrary.wiley.com/doi/10.1111/j.1365-3016.2011.01196.x/full#f2

29. Example of independently randomized newborns Shaffer et al 2009 BMC research methodology

30. What should we do? • Because different methods of analysis can give different results it is important to specify the analysis before the data are available, to prevent ‘creative accounting’ • Subjects randomized as a cluster should be analyzed as a cluster, some techniques seem more conservative and may inflate the risk of type 2 error

31. What should we do? • When subjects are randomized independently, subjects that are in the same group should be analyzed by methods that can adapt to the degree of correlation • Reasonable to plan from the start that if the correlation is less than a certain limit, it will not be adjusted for. EG ≤ 0.2

32. Publication of protocols • Protocols as designed at the start of a trial may have different primary outcomes to those actually published. • A major reason seems to be that the primary outcome was not significant!

33. Chen et al CMAJ 2004

34. What should we do • Develop empirical data concerning the effects of different randomization models on consent rates • Protocols should be published and, in addition to other details such as the primary outcome, should detail analysis methods • Including the methods that will be used for twins • During enrolment it should be clearly identified in the data set if siblings are in the trial • Original data should be available, to enable SR and meta-analysis which replicates the original analysis

35. Which randomization method is best? • In terms of the mathematical consequences randomizing the first eligible infant, and entering the second in the alternate group • Genetic and other risk factors balanced • Maintains or even increases power • Questionable acceptability • What about triplets? Multiple group trials? • For some families randomizing 1st eligible infant and entering siblings in the same group may be only acceptable method • Don’t know how frequent this is, may ensure good representation of multiples • Effectively discounts the contribution of the multiples to the analysis

36. Which randomization method is best? • Independent randomization has less effect on power • Examples I showed have good twin representation • Doesn’t completely eliminate the need to adjust the analysis for those multiples which are in the same group • Preserves the value of the contribution of the multiples

37. How does this affect the families? • Are there consequences of the method of enrollment on the families in the long term? • If one dies or does poorly? • If there is an important treatment effect? • Little information of the long term effects of trial participation, particularly for multiples