Sensitivity Analysis of Randomized Trials with Missing Data

1. Sensitivity Analysis of Randomized Trials with Missing Data Daniel Scharfstein Department of Biostatistics Johns Hopkins University dscharf@jhsph.edu

2. ACTG 175 • ACTG 175 was a randomized, double bind trial designed to evaluate nucleoside monotherapy vs. combination therapy in HIV+ individuals with CD4 counts between 200 and 500. • Participants were randomized to one of four treatments: AZT, AZT+ddI, AZT+ddC, ddI • CD4 counts were scheduled at baseline, week 8, and then every 12 weeks thereafter. • Additional baseline characteristics were also collected.

3. ACTG 175 • One goal of the investigators was to compare the treatment-specific means of CD4 count at week 56 had all subjects remained on their assigned treatment through that week. • The interest is efficacy rather than effectiveness. • We define a completer to be a subject who stays on therapy and is measured at week 56. Otherwise, the subject is called a drop-out. • 33.6% and 26.5% of subjects dropped out in the AZT+ddI and ddI arms, respectively.

4. ACTG 175 • Completers-only analysis

5. ACTG 175 • The completers-only means will be valid estimates if, within treatment groups, the completers and drop-outs are similar on measured and unmeasured characteristics. • Missing at random (MAR), with respect to treatment group. • Without incorporating additional information, the MAR assumption is untestable. • It is well known from other studies that, within treatment groups, drop-outs tend to be very different than completers.

6. Goal • Present a coherent paradigm for the presentation of results of clinical trials in which it is plausible that MAR fails (i.e., NMAR). • Sensitivity Analysis • Bayesian Analysis

7. Sensitivity Analysis Step 1: Models • For each treatment group, specify a set of models for the relationship between the distributions of the outcome for drop-outs and completers. • Index the treatment-specific models by an untestable parameter (alpha), where zero denotes MAR. • alpha is called a selection bias parameter and it indexes deviations from MAR. • Pattern-mixture model

8. Treatment-specific imputed distributions of CD4 count at week 56 for drop-outs

9. Sensitivity Analysis Step 1: Models • Selection model • The parameter alpha is interpreted as the log odds ratio of dropping out when comparing subjects whose log CD4 count at week 56 differs by 1 unit. • alpha>0 (<0) indicates that subjects with higher (lower) CD4 counts are more likely to drop-out. • alpha=0.5 (-0.5) implies that a 2-fold increase in CD4 count yields a 1.4 increase (0.7 decrease) in the odds of dropping out.

10. Sensitivity Analysis Step 2: Estimation • For a plausible range of alpha’s, estimate the treatment-specific means by taking a weighted average of the mean outcomes from the completers and drop-outs.

11. Treatment-specific imputed distributions of CD4 count at week 56 for drop-outs

12. Treatment-specific estimated mean CD4 at week 56 as function of alpha

13. Sensitivity Analysis Step 3: Testing • Test the null hypothesis of no treatment effect as a function of treatment-specific selection bias parameters. • For each combination of the treatment-specific selection bias parameters, form a Z-statistic by taking the difference in the estimated means divided by the estimated standard error of the difference.

14. Sensitivity Analysis Step 3: Testing • If the selection bias parameters are correctly specified, this statistic is normal(0,1) under the null hypothesis. • Reject the null hypothesis at the 0.05 level if the absolute value of the Z-statistic is greater than 1.96.

15. Contour Plot of Z-statistic

16. Contour Plot of Z-statistic

17. Bayesian Analysis • Think of all model parameters as random. • Place prior distributions on these parameters. • Informative prior on alpha (e.g., normal with mean -0.5 and standard deviation 0.25). • Non-informative priors on all other parameters (e.g., the distribution of the outcome). • Results are summarized through posterior distributions.

18. Posterior Distributions 368 (342,391) 348 (330,365)

19. Posterior Distribution of Mean Difference 20 (-11,49); 91%

20. Likelihood-based Inference • A parametric model for the outcome and a parametric for the probability of being a completer given the outcome. • For example, the outcome is log normal. • Inference proceeds by maximum likelihood (ML). • ML inference can be well approximated using Bayesian machinery.

21. Maximum Likelihood Distributions -2.6 (-3.0,-2.1) 303 (278,331) 368 (342,391) -2.8 (-3.3,-2.2) 297 (271,324) 348 (330,365)

22. Treatment-specific imputed distributions of CD4 count at week 56 for drop-outs

23. Maximum Likelihood Distribution of Mean Difference 7 (-31,44) 20 (-11,49)

24. Incorporating Auxiliary Information • MAR (with respect to all observable data) • Sensitivity analysis with respect alpha. • Bayesian methods under development. Longitudinal/Time-to-Event Data • Same underlying principles.

25. LOCF Bad idea • Imputing an unreasonable value. • Results may be conservative or anti-conservative. • Uncertainty is under-estimated.

26. Conjecture • There is information from previously conducted clinical studies to help in the analysis of the current trials. • Data from previous trials may be able to restrict the range of or estimate alpha.

27. Summary • We have presented a paradigm for reporting the results of clinical trials where missingness is plausibly related to outcomes. • We believe this approach provides a more honest characterization of the overall uncertainty, which stems from both sampling variability and lack of knowledge of the missingness mechanism.

28. dscharf@jhsph.edu • Scharfstein, Rotnitzky A, Robins JM, and Scharfstein DO: “Semiparametric Regression for Repeated Outcomes with Non-ignorable Non-response,” Journal of the American Statistical Association, 93, 1321-1339, 1998. • Scharfstein DO, Rotnitzky A, and Robins, JM.: “Adjusting for Non-ignorable Drop-out Using Semiparametric Non-response Models (with discussion),” Journal of the American Statistical Association, 94, 1096-1146, 1999. • Rotnitzky A, Scharfstein DO, Su TL, and Robins JM: “A Sensitivity Analysis Methodology for Randomized Trials with Potentially Non-ignorable Cause-Specific Censoring,” Biometrics, 57:30-113, 2001 • Scharfstein DO, Robin JM, Eddings W and Rotnitzky A: “Inference in Randomized Studies with Informative Censoring and Discrete Time-to-Event Endpoints,” Biometrics, 57: 404-413, 2001. • Scharfstein DO and Robins JM: “Estimation of the Failure Time Distribution in the Presence of Informative Right Censoring,” Biometrika 89:617-635, 2002. • Scharfstein DO, Daniels M, and Robins JM: “Incorporating Prior Beliefs About Selection Bias in the Analysis of Randomized Trials with Missing Data,” Biostatistics, 4: 495-512, 2003.