Hui Quan 1 , Tom Capizzi 1 , Ji Zhang 2 1 Merck Research Laboratories 2 Sanofi-Synthelabo Research

1. Multiplicity Adjustment For Clinical Trials with Two Doses of an Active Treatment and Multiple Primary and Secondary Endpoints Hui Quan1, Tom Capizzi1, Ji Zhang2 1Merck Research Laboratories 2Sanofi-Synthelabo Research FDA/Industry Statistics Workshop September 21-23, 2004

2. Outline • Background and clinical trial setting • Procedures for a two-dimensional problem • Procedures for a three-dimensional problem • Numerical examples for comparing powers • Real data example • Procedures without overall strong control • Discussion

3. Background • Multiple primary and secondary endpoints are often simultaneously considered in Phase III trials. • Positive findings on secondary endpoints could be for: 1: Supporting primary results or Hypotheses generating 2: Additional claims or description in drug label • For category 2  recent regulatory guidances and publications emphasized to have Type I error rate controlled for secondary endpoints.

4. Background(2) Different approaches for interpreting trial results: • Conventional standard for positive trial: positive effect on primary endpoints. • Others: a trial as positive even when effect is positive only on a secondary endpoint  secondary endpoints must be considered in multiplicity adjustment. For example: Moyé (2000) proposed a PAAS for controlling the FWE rate αF with αP for the primary endpoints and αF-αP=αS for the secondary endpoints.

5. Trial Setting Trial examples: • Parathyroid hormone fracture trial (Neer et al. 2000): 20 and 40 μg vs. placebo on vertebral fracture and other fractures • Rofecoxib RA trial (Geunese, et al. 2002): 25 and 50 mg vs. placebo on four primary and other secondary endpoints • Estrogen Alzheimer trial (Mulnard, et al. 2000): 0.625 and 1.25 mg vs. placebo on CGIC and other scores • Metformin and rosiglitazone combination trial (Fonseca, et al. 2000): Rosiglitazone 4 and 8 mg vs. placebo on glucose and lipid endpoints

6. Trial Setting(2) Three-dimensional multiplicity adjustment problem: • Two doses • Two priority categories of endpoints: primary & secondary • Multiple endpoints in each priority category • Higher dose-the primary dose, lower dose-the secondary dose (no monotonic order on parameter space for dose response is assumed for any endpoint) • Trial positive only if positive on one or more primary endpoints – except when PAAS is used for multiplicity adjustment • No further priority order for endpoints within each category

7. Trial Setting (3) • Traditional procedures may be lack of power. For example: for a trial with two primary and two secondary endpoints-a total of 8 active to control comparisons, if a Hochberg procedure is used and if the lower dose shows no effect on the two secondary endpoints, the higher dose on the primary endpoints will be assessed at most at level of /3. • Since no monotonicorder in dose response is assumed, stepdown trend tests are not applicable. • More specific gatekeeping procedures may have higher power.

8. Procedures for a Two-dimensional Problem: Two doses and multiple endpoints Hij is the null hypothesis for comparing the i th dose to the control on the j th endpoint, and pij is the corresponding p-value, i=1, 2 and j=1, 2, …, g. (Dose 1: the higher and primary dose) Modified Bonferroni-Closed Procedure (2.1) • Step1. Reject all Hij if all pij ≤, i=1,2, j=1,…, g. Otherwise, • Step 2. Reject H1j if p1j ≤/g Reject H2j if p1j ≤/g and p2j ≤/g

9. Modified Hochberg-Bonferroni Procedure (2.2) • Step 1. Reject all Hij if all pij ≤, i=1,2, j=1,…, g. Otherwise, • Step 2. Use Hochberg procedure for the higher dose at ’=2/3. • Step 3. Let l =# of endpoints of no effect for the higher dose. If l=0, use a Hochberg procedure for the lower dose at level . Otherwise, • Step 4: a) Use Hochberg procedure for the lower dose at  with all endpoints; b) Assume no effect on the l (>0) endpoints for the lower dose; c) Use Hochberg procedure for the lower dose on the other (g-l) endpoints at level /3; then d) Claim the effect for the lower dose on an endpoint if the effect is significant at both a) and c).

10. Weighted Modified Bonferroni-Closed Procedure (2.3) • Step 1. Reject all Hij if all pij≤ Otherwise, • Step 2. Reject H1j if p1j ≤wj; reject H2j if p1j ≤wj and p2j ≤wj. where wj≥0 and Σwj=1 Applicable to a three-dimensional problem by assigning more weights on the primary and less weights on the secondary endpoints. However, different from PAAS, no need to split the significance level if all pij ≤.

11. Properties of Procedures (2.n) • Reject all the hypotheses if all p-values are ≤. • Step down from the higher dose to the lower dose to form closed procedures. • Theoretically or based on simulation, provide strong control on the FWE rate under appropriate conditions. for (2.1) and (2.2): the condition for the Simes test for (2.3): positive regression dependency condition for the weighted Simes test.

12. Procedures for the Three-dimensional Problem Hij is the null hypothesis for comparing the i th dose to the control on the j th primary endpoint, i=1, 2 and j=1,…, gp Sikis the null hypothesis for comparing the i th dose to the control on the k th secondary endpoint, i=1, 2 and k=1,…, gs. Following Westfall and Krishen (2001), Gatekeeping Procedure (3.1) for gp=1 and gs=1 (a) {H11} → (b) {H21, S11} → (c) {S21} level α for each { }.

13. Generalized Gatekeeping Procedure (3.2) (a) {H1j} (b) {[H2j], [S1k]} One-dimensional (c) {Sik} Two-dimensional One-dimensional procedure at (a) and (b), and two-dimensional procedure at (c). • Reject H1j (H2j) if it should be rejected at (a) ((b)) • Reject S1k if it should be rejected at both (b) and (c). • Reject S2k if all H2j’s have been rejected, S1k has been rejected at (b) and (c), and S2k should be rejected at (c).

14. Other Procedures for the Three-dimensional Problem • PAAS: {Hij} two-dimensional at level αp and {Sik} two-dimensional at level αs (αp+αs= α) • Primary-Secondary Gatekeeping: (a) {Hij} two-dimensional ↓ (b) {Sik} two-dimensional each at level α • Procedure (2.2) or others for the two-dimensional Problems

15. Numerical Examples(R=Rejection, NR=No Rejection) 4 Procedures are compared to the regular Hochberg Procedure P1=PAAS with P(2.2) for primary and secondary endpoints P2=P(2.3) with weight wp for primary and ws for secondary P3=P(3.2) with P(2.1) for Step (c) P4=P(3.2) with P(2.2) for Step (c)

16. Numerical Example 1 gp=1, gs=3, αp=.0375, αs=.0125, wp=3/6and ws=1/6

17. Numerical Example 2 gp=2, gs=2, αp=.0375, αs=.0125, wp=3/8and ws=1/8

18. Real Data Example • A randomized trial to compare mitoxantrone 12 mg/m2 (n=64) or 5 mg/m2 (n=64) to placebo (n=60) on patients with worsening relapsing remitting or secondary progressive multiple sclerosis (Hartung et al. 2002). • Here, two primary and two secondary endpoints for which p-values for both doses are available: EDSS, ambulation index, with relapses and admitted to hospital.

19. Real Data Example (2) gp=2, gs=2, αp=.040, αs=.010, wp=4/10and ws=1/10

20. Procedures without Overall Strong Control • Strong control is theoretically appealing. However, the procedures are generally conservative. • Strong control in certain key sub-families of hypotheses may be sufficient. Procedure (*) (a) {H1j H2j} two-dimensional at level α ↓ ↓ (b) {S1k S2k } two-dimensional at level α Procedure (*) strongly controls the FWE rate in the sub-families of individual doses, individual endpoints, within the families of primary endpoints and secondary endpoints. Also, it weakly controls the overall FWE rate.

21. Procedures without Overall Strong Control (2) Procedure (*) does not strongly control the overall FWE rate. When gp=1 and gs=1, Procedure (*) becomes {H11} → {H21} ↓ ↓ each { } at level α {S11} → and → {S21} (pij is the p-value forHij and qik is the p-value forSik) The probability of rejecting H21S11 when H21S11 is true is Sup Pr [(p11 ≤α)(p21 ≤α) or (p11 ≤α)(q11 ≤α) | H21S11] =Sup Pr[(p21≤α) or (q11≤α) | H21S11]≥2α-α2

22. Other Procedures without Strong Control For example: Step-down Closed Procedure: • Step a. Apply a procedure for each dose vs. the control separately at level  for all primary and secondary endpoints. • Step b. Reject H1j (or S1k) if it should be rejected at Step a. Reject H2j (or S2k) if both H1j (or S1k) & H2j (or S2k) should be rejected at Step a. It controls the FWE rates in the strong sense within sub-families of individual doses and endpoints, but not the family of the primary endpoints nor the family of the secondary endpoints. It controls the overall FWE rate in the weak sense.

23. Discussion • After Phase IIb dose finding study, Phase III confirmatory trials may still have two doses of the active treatment. The setting considered here should have very broad application. • As shown by numerical examples (here and in our manuscript), no procedure is uniformly more powerful than the others. • All procedures need only individual p-values -- no need to calculate the adjusted p-values based on different configurations of intersection hypotheses. • One should think hard whether the overall strong control is necessary for a particular trial.

24. Major References • Benjamini Y and Hochberg T. Multiple hypotheses testing with weights. Scandinavian Journal of Statistics 1997; 24: 407-418. • CPMP Points to Consider on Multiplicity Issues in Clinical Trials. 2002. • D’Agostino RB. Controlling alpha in a clinical trial: the case for secondary endpoints. Statistics in Medicine 2000; 19, 763-766. • Dmitrienko A, Offen WW and Westfall PH. Getekeeping strategies for clinical trials that do not require all primary effects to be significant. Statistics in Medicine 2003; 22: 2387-2400. • Hartung HP, Gonsette R, Konig N, Kwiecinski H, Guseo A, Morrissey SP, Krapf H, Zwingers T and the Mitoxantrone in Multiple Sclerosis Study Group. Mitoxantrone in progressive multiple sclerosis: a placebo-controlled, double-blind, randomized, multicentre trial. Lancet 2002; 360, 2018-2025. • Hochberg Y. A sharper Bonferroni procedure for multiple tests of significance. Biometrika 1988; 75, 800-802. • Macus R, Peritz E and Gabriel KR. On closed testing procedures with special reference to ordered analysis of variance. Biometrika 1976; 64, 655-660. • Moye LA. Alpha calculus in clinical trials: considerations and commentary for the new millennium. Statistics in Medicine 2000; 19, 767-779. • Neer RM, Arnaud CD, Zanchetta JR, Prince R, Gaich GA, Reginster JY, Hodsman AB, Eriksen EF, Ish-Shalom S, Genant HK, Wang O and Mitlak BH. Effect of parathyroid hormone (1-34) on fractures and bone mineral density in postmenopausal women with osteoporosis. The New England Journal of Medicine 2001; 344, 19, 1434-1441. • Quan H, Luo E, Capizzi T. Xun Chen, Lynn Wei, and Bruce Binkowitz, Multiplicity adjustment for multiple endpoints in clinical trials with multiple doses of an active control. JSM Proceedings 2003. • Samuel-Cahn, E. Is the Simes improved Bonferroni procedure conservative? Biometrika 1996; 83, 928-933. • Simes RJ. An improved Bonferroni procedure for multiple tests of significance. Biometrika, 1986; 63: 655-660. • Treanor JJ, Hayden FG, Vrooman PS, Barbarash R, Bettis R, Riff D, Singh S, Kinnersley N, Ward P and Mills RG. Efficacy and safety of the oral neuraminidase inhibitor oseltamivir in treating acute influenza. JAMA 2000; 283, 8, 1016-1024. • Westfall PH and Krishen A. Optimally weighted, fixed sequence and gatekeeper multiple testing procedures. Journal of Statistical Planning and Inference 2001; 99, 25-40.