Management of Missing Data in Clinical Trials from a Regulatory Perspective H.M. James Hung

1. Management of Missing Data in Clinical Trialsfrom a Regulatory Perspective H.M. James Hung Div. of Biometrics I, OB/OPaSS/CDER/FDA Presented in FDA/Industry Workshop, Bethesda, Maryland, September 23, 2004 James Hung, 2004 FDA/Industry

2. Collaborators Charles Anello, Yeh-Fong Chen, Kun Jin, Fanhui Kong, Kooros Mahjoob, Robert O’Neill, Ohid Siddiqui Office of Biostatistics, OPaSS, CDER Food and Drug Administration James Hung, 2004 FDA/Industry

3. Disclaimer The views expressed in this presentation are not necessarily of the U.S. Food and Drug Administration. Acknowledgment O’Neill (2003, 2004) Temple (1994-2004) James Hung, 2004 FDA/Industry

4. Outline • Informative dropout • Statistical analysis methods • Methodology consideration • Summary James Hung, 2004 FDA/Industry

5. Clinical trial focuses on intent-to-treat population (including completers and dropouts) Response variables often measured over time (e.g., at multiple clinic or hospital visits) James Hung, 2004 FDA/Industry

6. Often the main clinical hypothesis concerns the effect Kof a test drug r.t. a control at some time K (e.g., end of study). Statistical null hypothesis H0: K = 0 i.e., allow nonzero  at other time points? (make sense?) James Hung, 2004 FDA/Industry

7. Unclear why testing only at the last time point is most relevant (for simplicity? avoid statistical adjustment for testing multiple times?) Drug effects over time are important information. e.g., inconceivable to market a drug that is effective only at Week 6, say. James Hung, 2004 FDA/Industry

8. For drug effect over time (or some period of time, e.g., at steady state), the relevant null hypothesis is H0: 1 = ∙∙∙ = K = 0 or H0: slope difference = 0 (if response follows straight-line model ) or others for relevant time period. James Hung, 2004 FDA/Industry

9. Informative Dropout In many disease areas, dropout rate is high and the results of any analyses for ITT population is not interpretable because of a large amount of missing data, particularly when dropouts are ‘informative’. James Hung, 2004 FDA/Industry

10. Dropout problems are multi-dimensional e.g., dropping out due to multiple reasons: side effects of the drug, health state is worsening, unperceived benefit Little knowledge of real causes of missing data, whether missing mechanism related to study outcome or treatment James Hung, 2004 FDA/Industry

11. Informative dropout has many different definitions, e.g., - dependent on observed data, dependent on missing data, treatment-related dropout, … - tied in with missing mechanism MCAR, MAR, MNAR, NIM, … O’Neill (2003, 2004) James Hung, 2004 FDA/Industry

12. For regulatory consideration, any treatment related dropout may be a suspect of informative dropout and missing mechanism probably needs to be considered informative (i.e., may severely bias estimates and tests) unless proven otherwise. James Hung, 2004 FDA/Industry

13. In a clinical trial, each cohort of dropout by reason or by dropout time can be very small. Difficult or impossible to assess whether missing values are informative. James Hung, 2004 FDA/Industry

14. James Hung, 2004 FDA/Industry

15. James Hung, 2004 FDA/Industry

16. Based on visual inspection, drug seems to perform better than Placebo. James Hung, 2004 FDA/Industry

17. James Hung, 2004 FDA/Industry

18. James Hung, 2004 FDA/Industry

19. James Hung, 2004 FDA/Industry

20. James Hung, 2004 FDA/Industry

21. Difficult to tell whether missing mechanism is ‘ignorable’ or not… e.g., in a linear response profile, MAR May be NIM. James Hung, 2004 FDA/Industry

22. James Hung, 2004 FDA/Industry

23. James Hung, 2004 FDA/Industry

24. James Hung, 2004 FDA/Industry

25. These plots show difficulty in classifying dropouts (informative or not) in individual trials where each cohort of dropout is small, (though total dropout rate could be high). These types of analysis should be done with external historical trials, at least for classification purpose. James Hung, 2004 FDA/Industry

26. Statistical Analysis Methods • Literature guidance • No satisfactory statistical analysis method for handling non-ignorable missing data • Likelihood-based methods require assumptions about missing data mechanism (unverifiable from current trial data) James Hung, 2004 FDA/Industry

27. Facts • Validity of any analysis method is • very much in question. • 2) Better alternative method is unclear. • Use of current trial data to seek • imputation method is futile. • 3) Dropouts and missing data are • unavoidable. James Hung, 2004 FDA/Industry

28. Glimpse of the analysis problem  = µ1 - µ2at last time point ni = # of completers in group i fi = ni/Ni If there is no missing value, we have D = Y1 – Y2 (unbiased for ) V(D) = estimated variance of D Z = D/[V(D)]1/2 James Hung, 2004 FDA/Industry

29. Missing values  { D , V(D), Z } not obtainable. Can try to get E( D | data) and V( D | data). and construct Z* = E( D | data ) / [V( D | data )]1/2 or Z+ = E( D | data ) / [V(D)]1/2 James Hung, 2004 FDA/Industry

30. Yoi = sample mean of completers Ri = vector of indicators for completion or dropout Ymi = unobservable sample mean of dropouts James Hung, 2004 FDA/Industry

31. Immediately, when f1≠ f2, this statistic has problem of interpretation, unless Ri and Ymi are independent (MI). Under MI, E(Ymi | Yoi, Ri ) = E(Ymi) . And if E(Ymi) = µi, then completer analysis might offer a reasonable estimate of . James Hung, 2004 FDA/Industry

32. When f1 = f2 = f, a linear combination of obs sample mean difference of completers and difference in conditional mean of dropouts (the latter requires models). James Hung, 2004 FDA/Industry

33. What about Var (D | data)? Another formidable task ! Nonlikelihood-based methods are difficult to provide useful solutions unless some kind of ad-hoc conservative imputation is feasible. James Hung, 2004 FDA/Industry

34. LOCF (last observation carried forward) LOCF tests H0: K = 0. LOCF can be biased either in favor of test drug (e.g., when its effect decays over time*) or against test drug, even in case of MCAR. *Siddique and Hung (2003) James Hung, 2004 FDA/Industry

35. For assessing drug effect over time, LOCF can seriously underestimate variability of measurement and is unrealistic (i.e., impute a constant value for every visit after the patient dropped out). James Hung, 2004 FDA/Industry

36. LAO (last available observation) Operationally identical to LOCF, this tests some global drug effect over time, H0:  w1hµ1h =  w2hµ2h Wih= E(dropout rate of drug group i at time h) μih = expected response of patients dropping out after time h in drug group i Is this null hypothesis relevant? Shao and Zhong (2003) James Hung, 2004 FDA/Industry

37. LOCF versus LAO (in red) 1 2 1 1 1 Y 2 2 3 3 v0 v1 v2 v3 James Hung, 2004 FDA/Industry

38. The global mean µi =  wihµih can be unbiasedly estimated by the sample mean. But the usual MSE from ANOVA may not estimate right target (Shao and Zhong). LAO results can be difficult to interpret if dropout reasons or dropout rates are different in treatment groups. James Hung, 2004 FDA/Industry

39. If drug effect over time is at issue, why not use all the pertinent data (longitudinal data analysis should be more efficient than LAO). - need medical colleagues’ buy in Ex. Analysis of cuff BP over time may be more powerful (value of test statistic is much larger) than LAO Hung, Lawrence, Stockbridge, Lipicky (2000) James Hung, 2004 FDA/Industry

40. MMRM* (mixed-effect model repeated measure with saturated model) Response = µ + treatment + time + treatment*time + baseline + subject (treatment) + error subject (treatment) and error are random effects treatment and time are class variables *Mallinckrodt et al (2001) James Hung, 2004 FDA/Industry

41. MMRM* analysis used to test H0: K = 0. - statistically valid under MAR - seem more stable in terms of type I error rate than LOCF under MCAR or MAR*# (LOCF can be very bad, depending on  at other visits) *Mallinckrodt et al (2001) #Siddique and Hung (2003) James Hung, 2004 FDA/Industry

42. LOCF, LAO, MMRM can be very problematic in case of informative missing. Don’t know how todo ‘conservative’ imputation with these methods. James Hung, 2004 FDA/Industry

43. Worst rank/score analysis Test drug effect at time K in the presence of events (e.g., death) that cause informatively missing values of the primary study outcome at time K. Example: In congestive heart failure trials, exercise time is missing after death from heart failure. Lachin (1999) James Hung, 2004 FDA/Industry

44. Assign a worst score to any informatively missing values (due to occurrence of an absorbing event related to progression of disease) and perform a nonparametric rank analysis. Valid and efficient for testing H0: no treatment difference in distributions of both event time and main study outcome Lachin (1999) James Hung, 2004 FDA/Industry

45. For a drug having little effect on non- mortal outcome (e.g., exercise time), this analysis when used to test non-mortal effect can be anti-conservative if the drug improves survival. Unclear how to perform a reasonable test for the non-mortal effect alone (e.g., labeling issue) James Hung, 2004 FDA/Industry

46. Time to treatment failure analysis In time to event analysis, if test drug has severe side effects that cause more dropouts, then time to treatment failure (event or dropping out due to side effects) analysis may provide a conservative analysis. James Hung, 2004 FDA/Industry

47. Like the worst score/rank analysis, it is unclear how to perform a reasonable test for time to the interested event alone - censoring on dropout due to failure ? James Hung, 2004 FDA/Industry

48. WLP opposite/pooled imputation For binary outcome, opposite imputation imputes sample event rate of completers in one arm for unobserved event rate of incompleters in the opposite arm. Wittes, Lakatos, Prostfield (1989) Proschan et al (2001) James Hung, 2004 FDA/Industry

49. Pooled imputation imputes sample event rate of completers from both arms for unobserved event rate of noncompleters in each arm. Treat imputed rate as ordinary rate. Compute Z statistic in the ordinary manner using a combination of the observed and the imputed rates. Wittes et al (1989), Proschan et al (2001) James Hung, 2004 FDA/Industry

50. WLP is less conservative than the worst case analysis (assign ‘event’ to dropouts in the test drug group and ‘nonevent’ to dropouts in the control group). Proschan et al (2001) James Hung, 2004 FDA/Industry