Case Studies of the Use of Meta-Analyses of Drug Safety Questions in the Regulatory Setting

1. Case Studies of the Use of Meta-Analyses of Drug Safety Questions in the Regulatory Setting Jesse A. Berlin, ScD Johnson & Johnson Pharmaceutical Research and Development New Jersey Chapter of the ASA 10 October 2008

2. Outline • Case studies of the use of historical randomized trial data to address potential safety concerns • Emphasis on exploration of patient-level characteristics as potential effect modifiers • Some methodologic “heads up”

3. Is it sampling variability? • Problem: How do we distinguish sampling variability from “real” variability (possibly) associated with different effects of treatment in different subgroups of patients (or with different dosing algorithms or other specific aspects of treatment)?

4. Example 1: Galantamine • Acetylcholinesterase inhibitors (AchEIs) are used as a standard treatment for Alzheimer’s Disease (AD) • Galantamine, an AChEI, has been extensively studied in patients with mild to moderate AD • Galantamine has also been studied in patients with AD with concomitant cerebrovascular disease (CVD) and in patients with VaD (16). • The benefit is to slow the progress of cognitive decline (relative to placebo)

5. Safety “signal” for Galantamine in Mild Cognitive Impairment • Two 2-year randomized controlled trials • Individuals with mild cognitive impairment • Findings replicated in both studies • 13 deaths versus 1 death • Higher mortality observed in galantamine-treated patients, compared with placebo • Overall mortality rates were low in both groups • The findings prompted a reevaluation in patients with dementia

6. Galantamine Methods • All galantamine trials (J&J or Shire-sponsored) for which J&J could access data • Also searched MEDLINE and the Cochrane Controlled Trials Register (2005) Issue 4 • Trials included were independently reviewed, verified by two readers, and met the following criteria: • a) randomized • b) placebo-controlled • c) parallel group • d) blinded • e) at least one treatment arm with galantamine

7. Meta-analysis of survival in galantamine randomized trials (6 months duration)

8. Other Galantamine Analyses • Nested case-control study of deaths was used to investigate potential mechanism for the mortality increase • Baseline ECG findings • Comorbidities • Concomitant medications • Findings were inconclusive due to small sample size • Mortality analyses in press (Feldman et al.; Acta Neurologica Scandinavica) • We are doing a large, placebo-controlled study with mortality as the primary endpoint

9. Example: Erythropoietin-stimulating agents in oncology • Is meta-analysis the appropriate tool?

10. “Houston, we have a problem ...” • A randomized trial of the use of erythropoietin in breast cancer patients found an increased risk of mortality in the EPO arm, relative to the placebo arm. (More on this later.) • This led to extensive discussions with FDA and, ultimately, a presentation in May of 2004 to the Oncologic Drugs Advisory Committee (ODAC) • An obvious question is, what if we look at prior studies used in support of the current indication...

11. 25 Summary of Survival Over TimeEPO-INT-76 12-mo mortality N n (%)† Censored P value Placebo 470 115 (24) 92 .012 Epoetin alfa 469 148 (30) 81 12-mohazard ratio (95% CI) 1.37 (1.07, 1.74) Time, mo Vertical line represents end of double-blind phase at 12 mo. †Based on Kaplan-Meier estimate.

12. Definitions • Supportive anemia care: studies evaluating the established indication, treatment of anemia • Goal is to obtain sufficient rise in hemoglobin to reduce need for transfusion • Beyond correction of anemia: investigational studies treating to higher hemoglobin levels • Includes most of the recent studies evaluating effects of erythropoietic agents on cancer treatment outcomes (response, progression, survival)

13. Supportive Anemia Care StudiesAnalysis • Combined analysis from all completed, randomized, double-blind trials of anemic patients (n = 1,976; 10 trials) • Mortality hazard ratios • Thrombotic vascular event (TVE) frequency • Considerations for this analysis • Variations in design and patient heterogeneity • Studies primarily evaluated transfusion reduction over 12 to 24 weeks

14. Supportive Anemia Care StudiesNo Survival Effect Seen in Combined Analysis Favors Epoetin alfa Favorsplacebo (0.76, 1.28) 0.1 1 10 Test for heterogeneity, P = .66 HR (95% CI) log scale

15. J&J was not alone • Two other companies presented results at the ODAC meeting: Roche and Amgen

16. Pooled1 Analyses: Progression-free Survival Hazard Ratios Associated with Aranesp (A) vs. Placebo (P) by Tumor Type Ovarian (A = 11/49, P = 3/12) Breast (A = 16/94, P = 6/23) GI Other (A = 15/54, P = 4/13) SCLC (A = 40/60, P = 42/47) NSCLC (A = 99/146, P = 109/130) Lymphoma (A = 29/70, P = 32/60) Myeloma (A = 66/105, P = 57/86) CLL (A = 26/39, P = 22/28) Other (A = 23/91, P = 5/22) Overall(A = 325/708, P = 280/421) 0.1 1.0 10 Hazard Ratio (95% CI) 1Studies 980297, 20000161, 980291, 990114

17. Meta-analysis: Survival (Roche) Better withepoetin beta Better withplacebo N 1409 Category Total Subgroup 0.97 Study 1.01 MF4249 MF4250 MF4252 MF4253 MF4266 MF4313 MF4321 MF4421 MF4467 116 144 54 109 20 146 218 259 343 1.02 3.39 0.59 0.37 0.61 1.02 1.29 0.93 Tumor class Solid Hematological Other 613 791 5 1.04 0.2 0.4 0.6 1 2 3 4 5 6 10 20 30 Risk ratio

18. Policy implications (1) • Concerns about the meta-analyses • Studies are too heterogeneous • tumor types • cancer treatment regimens • other aspects of patient populations and treatment protocols (2004) • Meta-analysis may be “masking” signals in individual studies (2007) • Short-term follow-up, few events? • Bias toward the null due to crossover and other design flaws

19. Policy implications (2) • Next question to ODAC (2004, paraphrased): • We are asking for a large study in a single tumor type, with restricted treatment options to control extraneous sources of variability. Is it fair to generalize from a single tumor type to other sites? • (this is part of the dilemma)

20. Now here we are in 2008 • Further studies (in head and neck cancer and in anemia of cancer) have found a “signal” • Conducted outside the current label (high target Hb, or not chemotherapy-induced anemia) • So, we were back at ODAC in 2007 and again in March, 2008

22. Odds Lower Upper Study name ratio limit limit Dammacco (INT-2) 0.27 0.07 1.01 * Throuvalas 2000 0.31 0.01 7.95 Del Mastro 0.31 0.03 3.17 Dunphy 1999 0.31 0.01 8.28 * Vadhan-Raj 0.34 0.01 8.80 Oberhoff 0.35 0.12 1.04 Cazzola (Roche) 0.49 0.04 5.56 * EPO-GER-022 0.58 0.35 0.96 INT-3 0.61 0.26 1.41 Vansteenkiste (AMG 980297) 0.62 0.38 1.01 Henry_1995 0.69 0.34 1.40 * Blohmer (AGO-NOGGO) 0.73 0.37 1.44 Ten Bokkel (Roche) 0.75 0.13 4.28 Pirker( AMG 20010145) 0.79 0.52 1.21 Littlewood 0.81 0.52 1.28 Kotasek_2003 (AMG 980291) 0.83 0.33 2.05 Taylor 2005 (AMG 20030232) 0.84 0.45 1.59 Pangalis (P-174) 0.89 0.15 5.36 * EPO-CAN-15 0.93 0.43 2.00 Coiffer 0.97 0.35 2.66 Chang 2005 (EPO-CAN-17) 0.97 0.55 1.73 Witzig 2005 0.98 0.62 1.55 Razzouk 2006 updt 1.00 0.14 7.23 Aapro 2006 (BRAVE) 1.02 0.67 1.53 Mobus 1.05 0.70 1.57 Osterborg 2005 1.08 0.69 1.67 Osterborg 96 (Roche) 1.10 0.50 2.44 Milroy (INT-49) 1.16 0.79 1.72 Savonije 2005 1.18 0.73 1.90 * Thomas (GOG-0191) 1.19 0.55 2.56 Engert 2007 (HD 15 IA) 1.21 0.32 4.55 Thatcher combined 1.21 0.30 4.93 Case 1.22 0.65 2.30 Prozanto (INT-47) 1.23 0.63 2.39 Rose 1.39 0.64 2.98 PREPARE 1.40 0.90 2.15 Leyland-Jones (1 year ITT) 1.42 1.07 1.90 Hedenus 2003 (AMG 20000161) 1.48 0.97 2.27 Grote 2005 (N93-004) 1.54 0.64 3.72 Bamia 1.83 0.51 6.55 INT-I 1.86 0.50 6.85 O'Shaughnessy 2005 2.94 0.12 73.93 Wilkinson 2006 3.57 0.18 70.31 Random Effects Model 1.02 0.93 1.13 0.1 0.2 0.5 1 2 5 10 I2 = 0.3%; Fixed Effects Model = Random Effects Model If BEST LFU data are used, then I2 =0%; Random Effects Model = Fixed Effects Model = 0.97 (0.88, 1.07) a: 3 studies (Cascinu; Hedenus; Kurtz) reported no deaths in either arm *: Radiochemotherapy study Study-level Meta-analysis of Overall Death46 Controlled CIA Studies(n=12,034; 6505 ESA; 5529 Control)

23. Risks of ESAs (from 2008 FDA presentation) • 6 studies→ statistically significant evidence of ↑ tumor promotion and/or ↓ survival • BEST (Breast)* • ENHANCE (Head/Neck) • DAHANCA (Head/Neck) • 161 (Lymphoid Ca)* • CAN-20 (NSCLC) • 103 (Anemia of Cancer) (Many tumor types) • 2 studies→ trends of ↑ tumor promotion and/or ↓ survival • PREPARE (neoadjuvant breast)* • GOG 191 (cervical cancer)† †=pts receiving chemoRT * = pts receiving chemo

24. Study N1° EndpointESA Adverse (from label)Outcome Chemo BEST (breast) 939 12 mo OS ↓ 12 mo OS 161 (Lymphoid) 344 ∆ Hgb ↓ OS PREPARE (breast) 733 RFS, OS ↓ RFS*, ↓ OS* GOG 191 (cervical) 114 PFS ↓ OS* RT ENHANCE (H/N) 351 LR PFS ↓ LR PFS, ↓ OS DAHANCA (H/N) 522 LRC ↓ LRC, ↓ OS* No Chemo or RT CAN-20 (NSCLC) 70 QOL ↓ OS 103 (Heterogenous) 989 Transfusion ↓ OS *=trend

25. Endpoint ESA Control HR 95% CI PFS (3 yr) 59% 62% 1.06 0.58, 1.91 Survival (3 yr) 61% 71% 1.28 0.68, 2.42 Local+Distant Recurrence 27% 33% GOG-191 Cervical Results (N=114) HR = ESA : Control

26. Published meta-analysis (shortly before ODAC) • Bennett et al. found mortality relative risk of 1.10 (95% CI: 1.01, 1.20) 27 Feb 2008 JAMA • In an earlier presentation (Gleason et al., ASCO 2007, Dr. Bennett as senior author), analyses separated by “label status” (defined by baseline hemoglobin <12 versus >12: • Off-label: RR = 1.14 (1.02, 1.27), I2 = 12.7% • On-label: RR = 0.97 (0.86, 1.11), I2 = 0.0%

27. Mortality And ESA Responsea: (Landmark At 4 Weeks On Treatment) All Randomized Studies, ESA-treated Subjects

28. Mortality And ESA Responsea: (Landmark At 4 Weeks On Treatment) Beyond-anemia-correction studies, ESA-treated subjects

29. Mortality And ESA Responsea: (Landmark At 4 Weeks On Treatment) Anemia-correction studies, ESA-treated subjects

31. Methodologic issues (general) • Standardization of definitions across studies • Endpoints • Adverse events • Requires standardization of data collection across studies! (SPERT) • What about adjudication? • Does it depend on the endpoint? (Can we count deaths or not?) • Broad versus narrow definitions (numbers versus signal strength)

32. More methodologic issues • Long-term versus short-term follow-up • Non-constant hazards / hazard ratio? • When is it OK to discard data? • Studies with small numbers of events? • When is it OK to discard data? • Get down-weighted in analysis anyway • Beware of confounding of study-level characteristics!!!!!!! • E.g., low dose study of aspirin in women, higher dose study in men

33. A Few Words on Endpoint Definitions • Common view is that more sensitive definitions • Are more “conservative” by being inclusive • Increase power by generating more events • Overly broad inclusion of events can lead to an underestimation of the true relative risk • might include events less likely to be related to the true (but possibly unknown) mechanism of action or • by their nature, are simply more likely to be misclassified in clinical trials • Implications of “non-differential” misclassification in efficacy versus safety settings? • O’Neill RT. Assessment of safety. Biopharmaceutical Statistics for Drug Development. Peace KE, ed. New York: Marcel Dekker;1988:543–604. • Proschan MA, Lan KKG, Wittes JT. Statistical Methods for Monitoring Clinical Trials Springer, New York, 2006

34. Conclusions (1) • Meta-analysis has valuable applications in pharmacoepidemiology • Evaluation of safety using existing randomized trials • Evaluation of safety using non-experimental studies (need more time to show)

35. Conclusions (2) • There are challenging methodologic issues in the meta-analysis of safety data • Rare events, multiplicity, adjudication, … • Sensitivity analyses should always be performed • Then more sensitivity analyses should always be performed • Use patient-level data when possible