1 / 26

General Multiplicity Adjustment Approach and its Application

This talk discusses a general multiplicity adjustment approach and its application in evaluating several independently conducted studies. It explores the relationship between decision errors and rules, as well as the choice of decision error and power characteristics. An example is provided to illustrate the approach.

cecils
Download Presentation

General Multiplicity Adjustment Approach and its Application

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A General Multiplicity Adjustment Approach and Its Application to Evaluating Several Independently Conducted StudiesQian Li and Mohammad HuqueCDER/FDA MCP

  2. Disclaimer • The views expressed in this talk are those of the authors and do not necessarily represent those of the Food and Drug Administration MCP

  3. Outlines • Motivation • Extending the union-intersection method • Issues in application • relationship of the decision errors and decision rules • choosing a decision error • power characteristics • An example and closing remarks MCP

  4. Motivation • More than one independently conducted Phase III study in NDA submissions to support efficacy evaluation • Current practice • count the number of studies that are significant • combine studies • Interpretation of regulatory requirement • two successful studies • no consistent interpretation when more than two studies are conducted MCP

  5. Union-Intersection method • Roy (1953) first proposed a method of constructing a hypothesis test: H0: Ki=1H0i, for K hypothesis tests. PH0(Ti>, i=1,2,…,K)= where  is a critical cut point. MCP

  6. Extending the UI Approach • PH0(p(1)1 p(2)2...  p(K)K) ’ • where 1  2  ... K 1 are p-value cut points • p(1), p(2), …, p(K) are ordered p-values of p1, p2, …, pK • ’ is an overall type I error MCP

  7. Definitions of overall hypotheses • Possible choices of overall hypotheses: • at least one alternative is true H01/K: i=1KH0i vs. HA1/K: i=1KHAi • at least two alternatives are true H02/K: j=1K (i=1 to K,ijH0i ) vs. HA2/K :j=1K (i=1 to K,ijH0i ) … • all the alternatives are true H0K/K: j=1KH0i vs. HAK/K : j=1KHAi MCP

  8. Overall type I error and p-value cut points • For H01/k, the extended approach can be rewritten as follows when p-values are independent MCP

  9. Overall type I error and p-value cut points • For HAm+1/k (m>1) , • max overall type I error occur when m studies have power 1 to reject individual null • I’s satisfy the following and 1= 2= … = m m+1. MCP

  10. A special case of two hypothesesH01/2: H01H02 • H01: 10, H02: 2  0 • The null space is the third quadrant • max decision error occur at (1=0, 2=0) • 1 and 2 satisfy 21 2 -12’ • More than one set of 1 and 2 • For ’=0.05, • if 1 = 0.025, 2 =1 • if 1 = 0.05, 2 =0.525 2 (0,0) 1 MCP

  11. Rejection regions p1 p1 1 0.525 0.05 p2 p2 0.025 0.05 0.525 0.025 1 1=0.025 2=1.000 1=0.050 2=0.525 MCP

  12. Possible decision rules for two studieswhen ’= 0.0252 MCP

  13. Special case of two studiesH02/2: H01H02 • The null space is all the area except first quadrant • max decision error occurs at (1= , 2=0) & (1=0, 2=) • max overall type I error is controlled when 1 =2’ 2 (0,0) 1 MCP

  14. Issues in application • Choice of overall hypothesis • Choice of decision error • Power characteristics MCP

  15. Relationship of decision errors among different overall hypotheses • For a set of K independent p-values, there exists a common rejection regionp1 =p2 =…=pk . • The corresponding decision errors for the overall hypotheses are: H01/k: ’= k H02/k : ’= k-1 … H0k-1/k: ’= 2 H0k/k : ’=  MCP

  16. Relationship of decision rules • It can be shown that the decision rules derived from a stringent hypothesis can also be derived from a less stringent hypothesis, given the relationship of the decision errors among different overall hypotheses. MCP

  17. Relationship of decision errors • Exist a common rejection region in (p1, p2) That is to require two significant studies, p1 =p2  Decision errors : • H01/2: H01H02 ’= 2 • H02/2 : H01 H02 ’=  p1  p2  0 MCP

  18. Strategies for choosing decision errors • Considering 4 strategies • Use the same decision error for all HAk/k • Use the same decision error for all HA1/k • Find the decision errors that keep a constant power for HAk/k • Control the power increase for HAk/k MCP

  19. Same decision error for HAk/k • Require all the K studies significant at level ’ • Similar to UI decision rules • Power is low Each study has 90% power at level 0.025 K: 1 2 3 4 5 6 Error: 0.025 0.025 0.025 0.025 0.025 0.025 Power: 90.0 80.8 72.9 65.6 59.0 53.1 • Not fair to use the same decision error for HAk/k when K is large MCP

  20. Same decision error for HA1/K • Decision error and power for HAK/K Each study has 90% power at level 0.025K: 1 2 3 4 5 6 Error: 0.0006 0.025 0.085 0.158 0.229 0.292 Power: 50.0 81.0 96.9 98.7 99.4 99.6 • When K increases, there is a large increase in decision error, • therefore large increase in power MCP

  21. Keeping consistent power for HAK/K • This strategy is in between the first two strategies • Decision error and power for HAK/K Each study has 90% power at level 0.025K: 1 2 3 4 5 6 Error: 0.009 0.025 0.040 0.054 0.066 0.077 Power: 81.0 81.0 81.0 81.0 81.0 81.0 • In case not satisfied ... MCP

  22. Increasing power for HAK/K • Decide a reasonable power for HAK/K, then figure out the error rate • Decision error and power for HAK/K Each study has 90% power at level 0.025K: 1 2 3 4 5 6 Error: 0.007 0.025 0.046 0.068 0.093 0.121 Power: 78.6 81.0 83.0 85.0 87.0 89.0 • Less conservative than the previous strategy MCP

  23. Power characteristics • Power function yK=PHA(p(1)1 p(2)2...  p(K)K) • Tedious to write when K>3 • Can be evaluated numerically • Search the optimal power numerically MCP

  24. Two studies - H01/2: H01H02 Power curve for 1= 2=1.5, ’=0.05 MCP

  25. An example • Three studies are conducted • Use HA1/3 , ’=0.0463 • p-value cut points are • 0.025, 0.025, 0.067 • Observed p-values were • 0.0065, 0.0125, 0.06 MCP

  26. Remarks • having the flexibility to choose p-value cut point • allows us to control decision error for multiple studies • possible to balance variations among p-values MCP

More Related