Sensitivity to MAR Assumption in Missing Data: Case Studies Using Model-Based Multiple Imputation

1. Sensitivity to MAR Assumption in Missing Data: Case Studies Using Model-Based Multiple Imputation Jie Zhang, PhD and José Pinheiro, PhD Biostatistics, Novartis Pharmaceuticals NJ ASA Chapter 28th Spring Symposium – May 31, 2007

2. Case study I: INDDEX trial • A randomized, multicenter, double-blind, placebo-controlled study of the effect of Exelon on the time to clinical diagnosis of Alzheimer’s Disease (AD) in subjects with Mild Cognitive Impairment (MCI) • A total sample size of 1018 patients allocated to two treatment groups: Exelon (510 patients) and placebo (508 patients). • Primary endpoint: time to clinical diagnosis of AD (possibly censored due to drop-out or end of study) • Primary analysis: Cox proportional hazards model including treatment, gender, race, age, and education level • Study duration: approximately 48 months • Drop-out rate: approximately 50% in both treatment groups 2007 NJ ASA Spring Symposium

3. Concerns about impact of drop-outs • Towards the end of the trial, but before unblinding – concerns about: • large drop-out rate • potential different drop-out patterns between treatment arms • Larger number of patients drop-out early due to certain adverse events which are potentially associated with the Exelon • Drop-out time due to AEs believed to be unrelated to time till conversion to AD (independent censoring) • Team wanted to investigate potential impact of different missing patterns on the primary analysis results (more specifically, the p-value for treatment effect) 2007 NJ ASA Spring Symposium

4. Model-based multiple imputation • Key idea: • imputed censored times under different assumptions about distribution of time to conversion to AD (multiple imputation – MI) • model-based MI is different from MI based on posterior predictive distribution of missing data, P(Ymis|Yobs), which requires MAR assumption • explore sensitivity of p-values and estimates of treatment effect • “Residual” time between drop-out and diagnosis of AD imputed using exponential (λ) model – study design used to produce censoring • Different hazard rates λ used for Exelon and placebo arms, to explore various scenarios of drop-outs • Complete, imputed datasets analyzed using same Cox proportional hazards model as in primary analysis • Twenty complete datasets were created, with MI techniques used to produce a p-value and an estimated hazard ratio 2007 NJ ASA Spring Symposium

5. Sensitivity analysis • A grid of imputation hazard rates for Exelon and placebo was used for the sensitivity analysis: 20 hazard rates in each arm, covering expected times to diagnosis of AD between 1 and 20 years (total 400 scenarios) • Each hazard rate combination used with model-based MI algorithm, producing 400 p-values and 400 estimated hazard rates • Goal was to produce a two-dimensional mapping of p-values and estimated hazard rates, identifying region where statistical significance (at 5% level) was attained • Non-significance restricted to implausible regions for imputation hazard rates would strengthen validity of a positive result in the trial • Non-significance under plausible region would have reverse effect 2007 NJ ASA Spring Symposium

8. Case Study I: Conclusions • Trial results were inconclusive with a p-value > 0.05 • Sensitivity analysis was not discussed with HA, but included in study report • P-value and hazard ratio sensitivity mapping provided insights on how much larger difference in mean time to AD diagnosis between Exelon and placebo would need to be for statistical significance to be attained • Clinical team was comfortable with sensitivity analysis approach and confident on its usefulness had the trial results been positive • The MI did not utilize the information from covariates and observed patterns of failure and censoring up to the time of censoring 2007 NJ ASA Spring Symposium

9. Case study II • A 3-year multinational, randomized, double-blind, placebo-controlled clinical trial in ~ 8,000 postmenopausal women with osteoporosis • Primary objective: evaluate the potential of a new drug in reducing the risk of both hip and vertebral fracture • Focus on vertebral fracture endpoint in pre-defined Cohort of ~6000 patients • Primary analysis: logistic regression including treatment and baseline vertebral fracture status 2007 NJ ASA Spring Symposium

10. Case study II - Data structure 2007 NJ ASA Spring Symposium

11. Case study II – Missing data problem • Drop-out rate can be very high due to elderly and fragile patient population (higher than fracture rate) • The concerns are: • The distribution of fracture rate for drop-out patients may be different from that of non drop-out patients. • No available data to examine the assumption of missing mechanism completely. • What can be different between the distribution of drop-out patients and non drop-out patients in this study: • treatment effect • 2-year fracture rate • baseline risk factors can be different (able to be adjusted to a certain degree) • other unknown factors (not able to be adjusted) 2007 NJ ASA Spring Symposium

12. Case study II – Proposed method Instead of assuming the missing data follow a specific distribution in the analysis, perform sensitivity analysis across a range of plausible scenarios using model-based multiple imputation approach: • treatment effects and • 2-year fracture rates adjusted by baseline risk factors Model-based MI differs from MI based on posterior predictive distribution of missing data, P(Ymis|Yobs), which require MAR assumption Parameterizations used to impute the missing data are different from Case I 2007 NJ ASA Spring Symposium

13. Case study II - Proposed method • Fit a logistic regression model with 2-year fracture response on baseline risk factors using patients in the placebo group. In addition, these patient’s whose responses are “no fracture” at 1-year • Use the above logistic regression model to predict the 2-year probability of fracture (Pi2(2)) for the missing data based on each individual patient’s baseline risk factors regardless of treatment group • For the missing data (pattern 2): • Impute 2-year fracture response in the placebo group using logit(Pi2C(2)) = logit(Pi2(2)) + log(rC) • Impute 2-year fracture in the treatment group using logit(Pi2E(2)) = logit(Pi2(2)) + log(rC) + log(rE) • Note: • rC controls the difference of 2-year fracture rate between observed data and missing data after adjusting for baseline fracture risks (~odds ratio) • rE controls the treatment effect for missing data 2007 NJ ASA Spring Symposium

14. Case study II - Proposed method Sensitivity factors: • rC = 1,, implies that 2-year fracture rates are the same for missing data and observed data given the same baseline risk factors for placebo patients • rE = 1,, implies that there is no treatment benefit for the missing data • rE = estimated treatment effect from observed data, it implies that treatment benefit is the same for the missing data and observed data By varying rC and rE , we can assess the robustness of the results under different assumptions Use multiple imputation (MI) methods to compute the inferential statistics • For each imputation, compute the inferential statistics using protocol specified analysis • Combine L imputation results using MI method to obtain p-value and estimated treatment effect 2007 NJ ASA Spring Symposium

15. Case study II - Planning • Data were simulated based on protocol assumptions to assess the proposed methods • One example from simulated data Fracture response by treatment and missing pattern Baseline fracture risks used: age, BMD, previous fractures 2007 NJ ASA Spring Symposium

16. Case study II - Planning (simulated)P-values under different assumptions of rCand rE Sensitivity factor rC Sensitivity factor rE 2007 NJ ASA Spring Symposium

17. Case studies II – Present the results • Present the sensitivity analyses under a large range of scenario (including plausible and implausible scenario), for example, using contour plot. • Pre-specify plausible scenarios with consultation with external experts (KOL …), only present the results for the pre-specified cases 2007 NJ ASA Spring Symposium

18. Case study II – final results At Year 3, there are overall 16% drop-out with 18% in the Cohort. 2007 NJ ASA Spring Symposium

19. Case study II – final results (SAS outputs submitted) 2007 NJ ASA Spring Symposium

20. Conclusions • Two case studies were presented using sensitivity analysis to assess the impact of potential non-ignorable missing data • Both cases used model-based MI. Case II also utilized the baseline information and data pattern up to missing • Fortunately, conclusions can be easily reached for both cases. An example with borderline results would be more interesting for discussion • Sensitivity analyses under variety of assumptions for missing data distribution • provide an unique way to assess the impact of the missing data • provide more balanced and complete assessment of the impact of missing data • much better approach than any single imputation approach, such as LOCF, worst scenario approach … 2007 NJ ASA Spring Symposium

21. Acknowledgements We like to thank the following individuals for their support in the projects • Joseph Hogan Brown University • Gerd Rosenkranz Novartis • Nathalie Ezzet Novartis • Peter Mesenbrink Novartis • Huilin Hu Novartis • Audrey Wong Novartis • Peter Quarg Novartis • Jane Xu Novartis 2007 NJ ASA Spring Symposium