Challenges of Non-Inferiority Trial Designs

1. Challenges of Non-Inferiority Trial Designs R. Sridhara, Ph.D.

2. Terminology • Superiority = Drug T better than Placebo (P) or T better than Active Control (C) • C may or may not have known efficacy • Non-inferiority (NI) = T not much less effective than C  ‘Was Not Different’ or ‘Similar’ • C must have known efficacy Pediatric Advisory Committe

3. Clinical Trials to Demonstrate Efficacy • Superiority Claim • First choice • Direct evidence • Non-inferiority Claim • Last choice • Indirect evidence • Interpretation could be misleading Pediatric Advisory Committe

4. Non-inferiority Claim Implies: • The effect of T and C are close • T and C may be both equally beneficial or T and C may be both equally not beneficial Pediatric Advisory Committe

5. Assumptions for NI consideration • C has an effect (compared to placebo, P) • Therefore, C can not be another experimental therapy in NI trial • Can reliably estimate C effect size * C effect size compared to P • Control (C) effect in the future study population will be same as in the historical population. Pediatric Advisory Committe

6. Non-inferiority Trial Design Considerations • Endpoint - Overall Survival - Progression-free Survival - Response Rate • Control Effect •  % Retention Pediatric Advisory Committe

7. Non-inferiority or Superiority • Challenges with Non-inferiority Trial • How well we know the effect of the control • How much of the control effect can we afford to give up • At least ‘’% retention of the control effect necessary to make sure that the new treatment is significantly better than placebo Pediatric Advisory Committe

8. Superiority trial designs with OS as the primary endpoint • Null Hypothesis, H0: HR(T/C) = 1 vs. • Alternative, H1: HR (T/C) < 1 if T is better than C Pediatric Advisory Committe

9. Non-inferiority trial designs with OS as the primary endpoint • H0: HR(T/C) ≥ M vs. • H1: HR (T/C) = 1 if the two are expected to be similar, Or • H1: HR(T/C) = 0.95, for example, if the new treatment (T) is expected to be slightly better M is the margin determined based on the active control (C) effect size estimated from historical trials and percentage of the effect to be retained HR > 1 implies, T is worse than C Pediatric Advisory Committe

10. Percent Retention and Estimated Active Control Effect Size • Non-inferiority ≈ “not much less effective” • ‘X’ is the effect size of the active control; example: Point estimate of HR (C/P) = 0.5 (or HR(P/C) = 2.0) implies an estimate of active control effect size was a 50% reduction in the risk of event (ex: death) • Percent retention is a percentage of ‘X’ that is retained; example: a 50% retention of the 50% effect size is 25% effect size, i.e., the putative HR(T/P) = 0.75 (or HR(P/T) = 1.33) Pediatric Advisory Committe

11. Estimates of True Control Effect Point Estimate ( >> 0.025) Lower 95% C.L. ( << 0.025) Lower  % C.L. ( = 0.025) Pediatric Advisory Committe

12. Methods for estimating active control effect size: No method is ideal and no particular method is endorsed by the Agency • All methods assume that the control effect has not changed over time • Some methods do not consider the variation between and within studies • Other methods incorporate variation between and within studies Pediatric Advisory Committe

13. Hypothetical Example of a NI Trial with Overall Survival Endpoint • Suppose HR(P/C) = 2.0 (95% CI: 1.9, 2.1) • Then, Lower 70% Confidence limit = 1.97 Pediatric Advisory Committe

14. Hypothetical example: Estimates of True Control Effect Point Estimate HR(P/C) = 2.0 Lower 95% C.L. of HR = 1.9 Lower 70% C.L. of HR: 1.97 Pediatric Advisory Committe

15. Hypothetical Example of a NI Trial with Overall Survival Endpoint • Suppose we want to retain 50% of 1.97, i.e., HR(P/T) = 1.49, and we can accrue 100 patients per unit time • If H1: HR(T/C) = 1, then N = 407 events or 1045 patients will be required • If H1: HR(T/C) = 0.95, then N = 290 events or 836 patients Pediatric Advisory Committe

16. Sample Size Depends On: • How good the historical data is: • One randomized study – large confidence interval – more uncertainty about the effect size • Many randomized studies – smaller confidence interval – unless there were fundamental differences in the conduct of the studies: population, dose, etc. Pediatric Advisory Committe

17. Summary - 1 • Superiority trials provide direct evidence • New drug can be compared to placebo or control (established or not) • Non-inferiority trials provide indirect evidence • New drug must be compared to established control • Interpretation can be misleading Pediatric Advisory Committe

18. Summary - 2 • For NI trial consideration, active control effect must be well characterized • Able to estimate effect size • Control effect is same now as before • NI trials are generally large • Sample sizes for a NI trial is dependent on the estimate of control effect, population, % retention and the alternative hypothesis Pediatric Advisory Committe

19. Summary - 3 • In considering NI trial, any potential loss of efficacy must be weighed against risk-benefit ratio • Failed superiority does not imply non-inferiority Pediatric Advisory Committe