Bayesian Methods for Benefit/Risk Assessment

1. Bayesian Methods for Benefit/Risk Assessment Ram C. Tiwari Associate Director Office of Biostatistics, CDER, FDA Ram.Tiwari@fda.hhs.gov

2. Disclaimer This presentation reflects the views of the author and should not be construed to represent FDA’s views or policies. Benefit-risk Assessment

3. Benefit-risk Assessment

4. Outline • Introduction • Commonly-used Benefit-risk (BR) measures • Methodology • BR measures based on Global benefit-risk (GBR) scores and a new measure • Bayesian approaches • Power prior • Illustration and simulation study • Future work Benefit-risk Assessment

5. Introduction • The benefit-risk assessment is the basis of regulatory decisions in the pre-market and post market review process. • The evaluation of benefit and risk faces several challenges. Benefit-risk Assessment

6. Commonly used B-R measures • Various measures have been proposed to assess benefit and risk simultaneously: • Q-TWiST by Gelbert et al. (1989) • Ratio of benefit and risk by Payne (1975) • The Number Needed to Treat and the Number Needed to Harm by Holden et al. (2003) • Global Benefit Risk (GBR) scores by Chuang-Stein et al. (1991) Benefit-risk Assessment

7. BR categories • A five-category multinomial random variable to capture the benefit and risk of a drug product on each individual simultaneously: Table 1: Possible outcomes of a clinical trial with binary response data Benefit-risk Assessment

8. Example: Hydromorphone Data was provided by Jonathan Norton. Benefit-risk Assessment

9. GBR scores Benefit-risk Assessment

10. Methodology: BR measures • BR measures based on the global scores proposed by Chuang-Stein et al. • BR measures based on the global scores are for each arm (treatment and comparator) separately. • BR_Linear can take a continuous value on a scale of -4 to 4 (inclusive). Benefit-risk Assessment

11. Methodology: New BR measure • A new indicator based measure is proposed: • BR_Indicator compares two arms simultaneously. • It takes a integer value between -6 to 6 (inclusive). Benefit-risk Assessment

12. Methodology: Dirichlet prior • Dirichlet distribution is used as the conjugate prior for multinomial distribution, and the posterior distribution of the five-category random variable is derived at each visit using sequentially updated posterior as a prior. Benefit-risk Assessment

13. Methodology: Sequential Updating • Sequential updating of the posteriors are given by: • The posterior mean (i.e., Bayes estimate) and 95% credible interval for each of the four measures are obtained using a Markov chain Monte Carlo (MCMC) technique. Benefit-risk Assessment

14. Methodology: Decision Rules • For a BR measure, • If the credible interval include the value zero, the benefit does not outweigh the risk; • If the lower bound of the credible interval is greater than zero, the benefit outweighs the risk; • If the upper bound of the credible interval is less than zero, the risk outweighs the benefit. Benefit-risk Assessment

15. Methodology: Power Prior • Power prior (Ibrahim and Chen, 2000) is used through the likelihood function to discount the information from previous visits, and the posterior distribution of the five-category random variable is obtained using the Dirichlet prior for p and a Beta (1, 1) as a power prior for . Benefit-risk Assessment

16. Methodology: Model Fit • The model fit of the two models (with and without power prior) is assessed through the conditional predictive ordinate (CPO) and the logarithm of the pseudo-marginal likelihood (LPML). The larger the value of LPML, the better fit the model is. Here, n(i) is the data with niremoved. Benefit-risk Assessment

17. Back to our example: Hydromorphone Benefit-risk Assessment

18. Illustration: Posterior Means and 95% Credible Intervals for BR_Linear Measure without power prior with power prior Benefit-risk Assessment

19. Illustration: Posterior Means and 95% Credible Intervals for BR_Indicator Measure without power prior with power prior Benefit-risk Assessment

20. Illustration: Results a. The model without power prior b. The model with power prior Benefit-risk Assessment

21. Illustration: Posterior Means and 95% Credible Intervals for Power Prior Parameter Benefit-risk Assessment

22. Illustration: Model Fit Benefit-risk Assessment

23. Simulation study • Correlated longitudinal multinomial data are simulated using the R package SimCorMultRes.R, which uses an underlying regression model to draw correlated ordinal response. • Two scenarios are simulated: • The treatment arm is similar to the control arm in terms of benefit-risk; • The treatment arm is better than control arm in the sense that the benefit outweighs risk. Benefit-risk Assessment

24. Simulation study: Scenarios Scenario 1: Treatment benefit does not outweigh risk compared to control Scenario 2: Treatment benefit outweighs risk compared to control Benefit-risk Assessment

25. Simulation study: Scenario 1 Treatment benefit does not outweigh risk compared to control a. The model without power prior b. The model with power prior Benefit-risk Assessment

26. Simulation study: Scenario 2 Treatment benefit outweighs risk compared to control a. The model without power prior b. The model with power prior Benefit-risk Assessment

27. Simulation study: Results Scenario 1: Treatment benefit does not outweigh risk compared to control Scenario 2: Treatment benefit outweighs risk compared to control Benefit-risk Assessment

28. Simulation study: Model Fit Benefit-risk Assessment

29. Future Work in BR Assessment Benefit-risk Assessment

30. Future work in BR assessment • Frequentist approaches: • Bootstrap approach • General linear mixed model (GLMM) approach • Other Bayesian approaches: • Normal priors • Dirichlet process Benefit-risk Assessment

31. Bootstrap Approach • Approximate underlying distribution using the empirical distribution of the observed data; • Resample from the original dataset; • Calculate the estimates and confidence intervals (CIs) of the BR measures based on the bootstrap samples; • Percentile bootstrap CIs; • Basic bootstrap CIs; • Studentized bootstrap CIs; • Bias-Corrected and Accelerated CIs. • Apply the decision rules. Benefit-risk Assessment

32. Bootstrap Approach-Results Benefit-risk Assessment

33. General linear mixed model (GLMM) approach • Within each arm (T or C), the ithsubject falls in the jth category (vs. the first category) at kth visit can be modeled as, • where, α0 is the baseline effect assumed common across all categories, βj is the category effect, and γkis the longitudinal effect at kth visit, with and, . Benefit-risk Assessment

34. GLMM approach • Note that different variance-covariance structures can be used for (γ1,γ2,…γ8), to model the longitudinal trend. • Compound-symmetry • Power covariance structure • Unstructured covariance structure • The estimates of the confidence intervals of the global measures can be derived from Monte Carlo samples, and the decision rules can be determined based on the confidence intervals. Benefit-risk Assessment

35. General linear mixed model approach-Results Benefit-risk Assessment

36. Bayesian approaches with GLMM • (α0, βj ; j=1,…,5)~ independent Normal with means 0 and large variances; • Variance parameters~ IG • Dirichlet Process Approach: Let α0 to depend on subjects, that is, assume thatα0i |G ~ iid G, with G~ DP(M, G0), M>0 concentration parameter and G0 a baseline distribution such as a normal with mean 0 and large variance. βj ; j=1,…,5 are independent normal with means 0, and large variances. • The posterior distributions of the probability and the global measures can be derived, and the decision rules can be determined based on the credible intervals. Benefit-risk Assessment

37. Discussion • Quantitative measure of benefit and risk is an important aspect in the drug evaluation process. • The Bayesian method is a natural method for longitudinal data by sequentially updating the prior; Power prior can be used to discount information from previous visits. • Frequentist approaches such as bootstrapping method and general linear mixed model can be applied for benefit risk assessment. • Continuous research in longitudinal assessment of drug benefit-risk is warranted. Benefit-risk Assessment

38. Benefit-risk Assessment

39. Selected References • Gelber RD, Gelman RS, Goldhirsch A. A quality-of-life oriented endpoint for comparing treatments. Biometrics. 1989;45:781-795 • Payne JT, Loken MK. A survey of the benefits and risks in the practices of radiology. CRC Crit Rev ClinRadiolNucl Med. 1975; 6:425-475 • Holden WL, Juhaeri J, Dai W. “Benefit-Risk Analysis: A Proposal Using Quantitative Methods,” Pharmacoepidemiology and Drug Safety. 2003; 12, 611–616. 154 • Chuang-Stein C, Mohberg NR, Sinkula MS. Three measures for simultaneously evaluating benefits and risks using categorical data from clinical trials. Statistics in Medicine. 1991; 10:1349-1359. • Norton, JD. A Longitudinal Model and Graphic for Benefit-risk Analysis, with Case Study. Drug Information Journal. 2011; 45: 741-747. • Ibrahim, JG, Chen, MH. Power Prior Distributions for Regression Models. Statistical Science. 2000; 15: 46-60. Benefit-risk Assessment

40. Q & A Benefit-risk Assessment