Statistics in Drug Regulation: The Next 10 Years

1. Statistics in Drug Regulation:The Next 10 Years Thomas Permutt Director, Division of Biometrics II Center for Drug Evaluation and Research The views expressed are those of the speaker and not necessarily of FDA.

2. Statutory Standards • Substantial evidence of efficacy • All tests reasonably applicable for safety • Balance not explicit, but history clear

3. Risk/Benefit • Formerly: • Very good evidence about direction of mean treatment effect • Too good? No. • Adverse events: • Common: statistical but unimportant • Rare: nonstatistical but important

4. What’s New? • Rofecoxib • Rosiglitazone • LABA

5. Rofecoxib • Heart attacks • Large outcome trial • which was trial in new indication • Now need outcome studies for COX-2 and maybe nonselective

6. Rosiglitazone • Nissin meta-analysis • We do meta-analysis • You do meta-analysis • You do outcome trial, maybe

7. Meta-analysis • Hard • Nonstatistical • Statistical • Both different in regulatory setting

8. Meta-analysis: Nonstatistical • Better information, but … • Doesn’t fit usual protocol-driven regulatory framework, either • Do it anyway, but … • Nobody will believe you (or us), so … ? • sensitivity analysis important

9. Meta-analysis: Statistical • Fixed vs. random effects • doesn’t matter much for global null, but • this doesn’t apply to noninferiority • Attributable vs. relative risk • relative risk “stable” across settings • different length of study, at least • but attributable risk is what matters • what about zeroes • Nissin to Congress: “no information”

10. What triggers this? • “Signal” • Class effects • Someone else’s meta-analysis • For diabetes, everything • For COX-2, probably everything • other COX?

11. LABA • Believed to cause death • not “side effect,” death from asthma • Effect mostly “seen” without steroid • So, with steroid?

12. With Steroid, Show What? • Noninferior to nothing? • i.e., combination therapy vs. steroid • Noninferior to realistic alternative? • e.g., increased dose of steroid • why not superior? • because of benefit • Interaction with steroid? • i.e., already “know” without steroid: Is with different? • maybe can’t do without steroid anyway

13. Noninferiority Margins • Not “1.3” • COX-2 • diabetes • asthma! • Risk-benefit • for direct measures • for surrogates

14. Surrogate • Everyone likes “hard” endpoints but … • They mostly don’t measure benefit • They are correlated with benefit

15. Correlation with Benefit • Does drug produce benefit or modify correlation? (anti-arrythmics, maybe glitazones) • Qualitative validation hard enough • Quantify benefit very hard • estimate strength of relationship • and hope it holds

16. Patient-Reported Outcomes • Hard endpoints are “nice” but they don’t measure utility • PRO are squishy but relevant • Psychometrics is not evil (now)

17. Linking Risk and Benefit • Expected utility • mean efficacy outcome • incidence of AE • (mean effect) X (goodness) – (AE rate) X (badness) • Other formulas are incorrect • provided utility is linear wrt effect

18. It Isn’t Linear • For surrogates • For PROs

19. Utility Calculations: Example • 50% symptom-free • 50% intolerable adverse events • Good or bad? • How bad were symptoms? • How bad were adverse events?

20. Women have efficacy Men have adverse events Women have efficacy Women have adverse events Men have nothing Two Drugs

21. Women have efficacy Men have adverse events Useful drug provided AEs are reversible Women have efficacy Women have adverse events Men have nothing Useless drug Two Drugs “Expected utility” does not distinguish!

22. Why Doesn’t Expectation Work? • Because you don’t really measure benefit • benefit at timepoint (or average over time) is surrogate for long-term benefit • don’t get long-term benefit if you drop out • LOCF makes it worse • “Mixing up” safety and efficacy is … • not illegal • not even stupid • “individualized medicine” • dropout is good biomarker!