Lessons Learned From Recent Safety Meta-Analyses

1. Lessons Learned From Recent Safety Meta-Analyses Mark Levenson, Ph.D. Quantitative Safety and Pharmacoepidemiology GroupOffice of BiostatisticsCenter for Drug Evaluation and Research, FDA v. 1 Oct. 2009

2. Disclaimer The views expressed in this presentation represent the opinions of the author, and do not necessarily represent the views of the United States Food and Drug Administration

3. Focus Today • Safety • Regulatory setting • Pre- and post-market • Clinical trials • Access to relevant data • Not: locating, accessing quality of, and extracting data from studies

4. Outline • Study inclusion criteria • Endpoints • Methodology • Examples • Suicidality meta-analyses • Aprotinin

5. Meta-Analysis Steps • Define research goals • Research/understand relevant trials • Define analysis plan (Prespecify!) • Research questions • Study inclusion criteria • Analysis set and subgroups • Endpoints • Primary methods • Sensitivity methods • Make request to sponsor(s) / obtain data • Implement analysis plan • Report and interpret findings

6. Study Inclusion Criteria

7. Study Inclusion Criteria • Valid comparison groups • Similarity in • Design • Interventions • Study population • Studied indication • Data availability

10. Survival 1 2 Years Duration: Example • Assume event of interest takes some time to develop (increasing hazard)

11. Duration: Example (Cont.) • Scenario 1: • 10 trials • 100 patients per trial • 3 month duration • 1,000 patients • 250 person-years • Scenario 2: • 1 trial • 100 patients per trial • 2 year duration • 100 patients • 200 person-years

12. Endpoints

13. Endpoints • First choice: prospectively collect and adjudicate endpoints • Second choice: use common post-hoc adjudication procedure across trials • Last choice: make do with existing information from trials

14. Endpoints: Follow-Up Time Time Randomization End of Treatment Event End of Follow-up

15. Endpoints: Follow-Up Time • Follow-up should be long enough to capture event of interest • Use common cut-off point across trials when possible • Need to balance lasting effect of drug versus confounding with post-trial therapy and dilution of drug effect (see: NEJM Vioxx APPROVE discussion, 2006)

16. Data Availability Patient-level data allows more thorough analysis • Time-to-event • Subgroup • Treatment duration effects • Internal validation

17. Methodology

18. Methodology • Need to use appropriate methods for problem • Need to justify method • Need to perform sensitivity analyses along several fronts

19. Methodology Considerations • Number of trials • Number of subjects per trial • Rates of events • Zero-event trials • Heterogeneity of effect

20. Methods • Inverse variance weighting • Mantel-Haenszel odds ratio or risk ratio • Exact method for odds ratio • Mantel-Haenszel risk difference • Bayesian methods • Encompass fixed- and random-effect models and hierarchical models

21. Sensitivity Analysis • Consequences of low event rate • Consequences of zero-event trials • Consequences of heterogeneity of trials • Random effects models • Trials with large influence

22. Sensitivity Strategy • Primary method: Exact method for OR • Sensitivity methods: • Mantel-Haenszel RD • GLMM, qualitatively compare results with primary method

23. Suicidality Meta-Analyses

24. Suicidality Meta-Analyses • Concern that drugs may be associated with suicidality • Requested all patient-level data from all placebo-controlled trials from sponsors • Patients retrospectively classified into suicidality outcomes by blinded experts

26. * * Reanalysisof FDA/Hammad 2004 data

27. Suicidal Behavior or Ideation Odds Ratio Estimates

28. Antiepileptic AC Paraphrase • Does committee agree with agency that findings should apply to all 11 drugs? Yes: 18, No: 3, Abstain: 0 • Does committee agree with agency that findings should apply to all approved antiepileptics? • Yes: 15, No: 5, Abstain: 1

29. Aprotinin

30. The Aprotinin Story • Aprotinin: used to reduce blood loss and transfusion in patients undergoing coronary artery bypass graft surgery (CABG) with cardiopulmonary bypass • 2006 NEJM Mangano paper raised safety concerns • FDA held 2 Advisory Committee meetings on the safety of aprotinin motivated by 3 observational studies

31. Disparate Findings • Mangano in-hospital death: no effect relative to no drug • Mangano 5-year death: 1.37 HR, p-value=0.008 relative to no drug • Sponsor Global CABG RCT Database: Death 2.9% aprotinin, 2.5% placebo (9/06 AC) • Meta-analyses Henry et al. Cochran Review 2007: RR=0.90 (0.67, 1.20) relative to control (no drug). No effects relative to other drugs.

32. BART • Blood Conservation Using Antifibrinolytics in a Randomized Trial (BART) • Compared aprotinin to two active drugs • 30-day death secondary endpoint

33. Jan. 2007 Second Interim Analysis • Aprotinin 5.0% vs. 3.9% and 4.3% for comparator drugs • DSMB: “did not consider the results of [Mangano Study] convincing.” • DSMB: [Systematic reviews] “less biased than the observational studies” • Four systematic reviews showed no death effect

34. October 2007 • Aprotinin 6.5% vs. 4.2% and 4.3% for comparator drugs (p-value near nominal significance) • DSBM recommends terminating trial

35. What Happened to the Meta-Analysis? • Ray Editorial (NEJM) • Trials not designed to collect follow-up mortality information • Limited data on head-to-head comparisons with other drugs • Trial and patient heterogeneity (surgical procedure, patient risk) may hide signal

36. Conclusions • Prospectively plan trials for pooled analyses, e.g. endpoint definition and ascertainment • Prespecify analysis plan • Select trials with similar and appropriate designs • Consider methodology issues of sparse events and perform sensitivity analyses • Understand the limitations of meta-analysis • Draft FDA Guidance for safety meta-analyses December 2009

37. References • Guidance for Industry Premarketing Risk Assessment (FDA, 2005) • Much ado about nothing: a comparison of the performance of meta-analytic methods with rare events (Statis. Med., Bradburn et al., 2007) • The Aprotinin Story – Is BART the final chapter? (NEJM, Ray, 2007) • Time-to-Event Analysis for Long-Term Treatments – The APPROVe trial (NEJM, Lagakos, 2006) • Understanding the New Drug Safety Standards: The Emerging Science of Meta-Analysis (The Pink Sheet, 2007)