1 / 37

Issues of Simultaneous Tests for Non-Inferiority and Superiority

Issues of Simultaneous Tests for Non-Inferiority and Superiority. Tie-Hua Ng*, Ph. D. U.S. Food and Drug Admi n istration Ng@cber.fda.gov Presented at MCP 2002 August 5-7 , 200 2 Bethesda, Maryland _______

shelly
Download Presentation

Issues of Simultaneous Tests for Non-Inferiority and Superiority

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Issues of Simultaneous Tests for Non-Inferiority and Superiority Tie-Hua Ng*, Ph. D. U.S. Food and Drug Administration Ng@cber.fda.gov Presented at MCP 2002 August 5-7, 2002 Bethesda, Maryland _______ * The views expressed in this presentation are not necessarily of the U.S. Food and Drug Administration.

  2. Simultaneous Tests for Non-Inferiority and Superiority • Multiplicity adjustment is not necessary • Intersection-union principle (IU) • Dunnett and Gent (1996) • Closed testing procedure (CTP) • Morikawa and Yoshida (1995) • Indisputable

  3. A Big Question Is Multiplicity Adjustment Necessary?

  4. Is Multiplicity Adjustment Necessary?

  5. Outline • Assumptions and Notations • Switching between Superiority and Non-Inferiority • Is Simultaneous Testing Acceptable? • Use of Confidence Interval in Hypothesis Testing --- Pitfall • Problems of Simultaneous Testing • Conclusion

  6. Assumptions/Notations • Normality and larger is better • T: Test/Experimental treatment (t) • S: Standard therapy/Active control (s) • : Non-InferiorityMargin (> 0) • For a given d (real number), define • Null: H0(d): T S - d • Alternative: H1(d): T >S - d • Non-Inferiority: d =  • Superiority: d = 0

  7. ° • T Boundary Non-Inferiority (d = ) H0(): T S - against H1(): T >S - H1() H0()  S Worse Better Mean Response

  8. Superiority (d = 0) H0 (0): T S against H1 (0): T >S H1(0) H0(0) ° • S T Worse Better Boundary Mean Response

  9. Switching between Superiority and Non-Inferiority CPMP (Committee for Proprietary Medicinal Products), European Agency for the Evaluation of Medicinal Products Points to Consider on Switching Between Superiority and Non-Inferiority, 2000. http://www.emea.eu.int/htms/human/ewp/ewpptc.htm

  10. Switching between Superiority and Non-Inferiority (2) • Non-Inferiority Trial • If H0() is rejected, proceed to test H0(0) • No multiplicity issue, closed testing procedure • Superiority Trial • Fail to reject H0(0), proceed to test H0() • No multiplicity issue • Post hoc specification of 

  11. Switching between Superiority and Non-Inferiority (3) • Non-inferiority Trial • Intention-to-treat (ITT) • Per protocol (PP) • Superiority Trial • Primary: Intention-to-treat (ITT) • Supportive: Per protocol (PP) • Assume ITT = PP

  12. Simultaneous Testing One-sided 100(1 - )% lower Confidence Interval for T - S Superiority Non-inferiority Neither - 0 Test is worse Test is better Mean Difference (T – S)

  13. Simultaneous Testing (2) • Multiplicity adjustment is not necessary • Dunnett and Gent (1996) • Intersection-Union (IU): Superiority: Both H0() andH0(0) are rejected • Morikawa and Yoshida (1995) • Closed Testing Procedure (CTP): Test H0(0) whenH0()H0(0) is rejected

  14. Simultaneous Testing (3) • Discussion Forum (October 1998) • London • PSI (Statisticians in Pharmaceutical Industry) • Is Simultaneous Testing of Equivalence [Non-Inferiority] and Superiority Acceptable? • Superiority trial: • Fail to reject H0 (0) • No equivalence/non-inferiority claim • Ok: Morikawa and Yoshida (1995) • Ref: Phillips et al (2000), DIJ

  15. Is Simultaneous TestingAcceptable?

  16. Use of Confidence Interval inHypothesis Testing H0(d): T S - d (at significance level ) One-sided 100(1-)% lower CI for T-S Reject H0(d)if and only if the CI excludes -d Reject H0(d) Do not reject H0(d) -d Test is worse Test is better Mean Difference (T – S)

  17. Use of Confidence Interval inHypothesis Testing (2) • If CI = (L, ), then H0(d) will be rejected for all -d < L. • A Tricky Question • Suppose CI = (-1.999, ), L = -1.999 • H0(2): T S - 2 is rejected (d=2) since -d < L • Can we conclude that T > S - 2? • Yes, if H0(2) is prespecified. • No, otherwise.

  18. Use of Confidence Interval inHypothesis Testing (3) Post hoc specification of H0(d) is a No No

  19. Simultaneous Testing: Problems H0(d1) and H0(d2), for d1 > d2 One-sided (1 - )100% lower CI for T - S Reject H0(d2) Reject H0(d1) Neither -d1 -d2 Test is worse Test is better Mean Difference (T – S)

  20. Simultaneous Testing: Problems (2) H0(d1), H0(d2) and H0(d3), for d1 > d2 > d3 One-sided (1 - )100% lower CI for T - S Reject H0(d3) Reject H0(d2) Reject H0(d1) None -d1 -d2 -d3 Test is worse Test is better Mean Difference (T – S)

  21. Simultaneous Testing: Problems (3) H0(d1), H0(d2),…, H0(dk), for d1 > d2 > … > dk One-sided (1 - )100% lower CI for T - S Reject H0(dk) . . . Reject H0(d2) … Reject H0(d1) None -d1 -d2 -d3 … -dk Test is worse Test is better Mean Difference (T – S)

  22. Simultaneous Testing: Problems (4) • Choose k large enough • Pr[-d1 < Lower limit < -dk] close to 1 • Max |dk - dk-1| < a given small number • Simultaneous testing of H0(di), i = 1,…, k  Post hoc specification of H0(d)

  23. Confirmatory (oneH0(d)) Simultaneous H0() and H0(0) Exploratory (manyH0(d)) 1 2 3 4 …………. k ………… Number of Nested hypotheses Simultaneous Testing: Problems (5)

  24. Simultaneous Testing: Problems (6) • What is wrong with IU and CTP? • Nothing • Pr[Rejecting at least one true null]   • What kind of problems?

  25. Simultaneous Testing: Problems (7) • Post hoc specification of H0(d) • Let -d0 = 100(1 - )% lower limit -  • Reject H0(d0), since -d0 < lower limit • Repeat the same trial independently • Pr[Rejecting H0(d0)] = 0.5 +

  26. Simultaneous Testing: Problems (8) • Simultaneous testing of many H0(d) • Repeat the same trial independently • Low probability of confirming the finding • 1st trial: Reject H0(dj) but not H0(dj+1) • 2nd trial: Pr[Rejecting H0(dj)] is low (e.g., 0.5+)

  27. Simultaneous Testing: Problems (9) • Simultaneous testing of H0() and H0(0)? • Confirm the finding •  = 2 • Known variance • Let   T - S • Significance level  = 0.025 • 80% power for H0() (at  = 0)

  28. Simultaneous Testing: Problems (10) f() = Pr[Rejecting H0() | ] f0() = Pr[Rejecting H0(0) | ]

  29. Simultaneous Testing: Problems (11) • Test one null hypothesis H0() • Suppose that H0() is rejected • Repeat the same trial independently • Pr[Rejecting H0() again] = f()

  30. Simultaneous Testing: Problems (12) • Test H0() and H0(0) simultaneously • Suppose that H0() or H0(0) is rejected • Repeat the same trial independently • Pr[Rejecting the same null hypothesis again] = [1 - w()]· f() + w() · f0() = f() - f0() [1 – f0()/f()], where w() = f0()/f()

  31. Simultaneous Testing: Problems (13) Simultaneous tests in the 2nd trial [1 - w()]· f() + w() · f0() where w() = f0()/f()

  32. Simultaneous Testing: Problems (14) • Ratio: 1 – [f0()/f()] [1 – f0()/f()] • Ratio may be as low as 0.75

  33. Conclusion • Many H0(d): Problematic • Not type I error rate • H0() and H0(0): Acceptable? • If “zero tolerance policy”: No • If 25% reduction cannot be tolerated: No • If 25% reduction can be tolerated: Yes

  34. Is Simultaneous Testing of H0()andH0(0) Acceptable?

  35. You be the judge

  36. References • Dunnett and Gent (1976), Statistics in Medicine, 15, 1729-1738. • Committee for Proprietary Medicinal Products (CPMP; 2002). Points to Consider on Switching Between Superiority and Non-Inferiority. http://www.emea.eu.int/htms/human/ewp/ewpptc.htm • Morikawa T, Yoshida M. (1995), Journal of Biopharmaceutical Statistics, 5:297-306. • Phillips et al., (2000), Drug Information Journal, 34:337-348. 

  37. God Bless America

More Related