Enhanced Quantitative Decision Making - Reducing the likelihood of incorrect decisions

1. Enhanced Quantitative Decision Making - Reducing the likelihood of incorrect decisions Mike K. Smith, Jonathan French, (Pfizer) Ken Kowalski, (A2PG) Wayne Ewy (formerly Pfizer, retired).

2. Six Components ofModel-Based Drug Development* PK/PD & Disease Models Trial Performance Metrics Model-Based Drug Development Competitor Info. & Meta-Analysis Decision Criteria Design & Trial Execution Models Data Analysis Model * Lalonde et al, Clin Pharm & Ther, 2007; 82: pp21-32

3. Quantitative Decision Criteria • “I’ll know it when I see it…” • “Evidence of an effect…” • “Reasonable efficacy and safety tradeoffs” • WRONG!!!

4. Quantitative Decision Criteria • 2 points improvement over placebo. • Better. • At least it’s quantitative • How sure do you want to be? • Mean 2 points? • Lower CI 2 points? • Mean 2 points and lower CI > 0?

5. P(Criteria|Data) • Not just P(… | Data) • Data • Prior data, model assumptions, parameter uncertainties • Trial design • Dropouts, imputation methods etc. • Data analytic method

6. Truth vs Trial • For a given set of model parameters / assumptions there will be a “true” outcome against the decision criteria. • What is the chance of achieving 2 points improvement given current information? • For a given set of parameters we will know whether we achieve 2 points improvement or not. • Then for this same set of parameters, apply design, dropout / imputation models, analytic technique and assess decision criteria.

7. Truth vs. Trial - Formally •  is the true (unknown) treatment effect • =f(, , ) is specified for a given set of model assumptions •   vector of fixed effects parameters •   covariance matrix for between-unit (subject or study) random effects •   covariance matrix for within-unit (subject or study) random effects

8. Truth vs. Trial - Formally • Define quantitative decision rule under truth () and data-analytic results (T), e.g., • Truth: Go if TV, No Go if <TV • Data: Go if TTV, No Go if T<TV • Note TV denotes the Target Value • Note T could be a point estimate or confidence limit on estimate/prediction of 

9. Operating Characteristics Trial Go Trial No Go Total “True” No Go “True” Go Total P(correct) PTS P(Go)

10. Example • Comparing SC-75416 with ibuprofen in dental pain. • Published in Kowalski, K.G, et al. “Modeling and Simulation to Support Dose Selection and Clinical Development of SC-75416, a Selective COX-2 Inhibitor for the Treatment of Acute and Chronic Pain”. • Decision criteria based on 3 point difference from ibuprofen in TOTPAR6 endpoint.

11. PTS = P(  3) = 67.2 Mean = 3.27, SD=0.60 Obs Mean = 3.3 Example From Kowalski et al: A model-based framework for quantitative decision-making in drug development Presentation at ACOP, Tuscon, AZ 2008.

12. Example P(correct) = 70.72% P(Go) = 61.90% PTS = 67.20% 17 From Kowalski et al: A model-based framework for quantitative decision-making in drug development Presentation at ACOP, Tuscon, AZ 2008.

13. “Nominal” values for OCs • P(Correct) can be fixed at >=80% • PTS for initiating a new trial depends on quadrant, portfolio, stage of development. • Perhaps minimal “dignity level” for starting a trial. • Fixing these two implies P(False GO) and P(False NO GO) must float, depend on P(Correct) and PTS. • Driven by decision criteria. • E.g. For P(Correct) = 80%, P(Incorrect) = 20%, spent across P(False GO), P(False NO GO).

14. Iterate / Optimise If the operating characteristics “don’t look good”… Change the data analytic model Change the design constraints (↑ n /group) Change the data-analytic decision criteria for the trial. If we fix one or more of the above (e.g. n /group) then there is limited other things that can improve OCs. Change the data analytic model, change data-analytic decision criteria for the trial.

15. The components may change over time “Truth” model / prior will be refined over time. P(“True” Go given current knowledge / model) changes. Decision criteria may change. Commercial viability changes. [This may change both our compound target criteria – truth decision rule, as well as the data-analytic decision rule] Acceptable level of confidence for Trial Go decision changes. [This applies only to data-analytic decision rule]

16. Final Remarks (1) • Greater collaboration required among kineticists/modelers, statisticians and clinicians • Kineticists/modelers: • Explicit and transparent about the assumptions and limitations of their PK/PD and disease models • Think strategically about how model will be used to influence internal decision-making • Avoid excessive use of NONMEM-jargon and write reports to broader audience • Calibrate models against data-derived (non-model-based) statistics of interest

17. Final Remarks (2) • Statisticians: • Embrace assumption-rich nonlinear models for decision-making especially in early clinical development • Avoid “Phase 3” mentality when designing Phase 2 studies…relying on empirical (assumption-poor) models to make decisions in early clinical development can be costly • Clinicians: • Quantitatively define clinically relevant effects and commercial targets • Explicitly and quantitatively defined decision rules

18. Bibliography • Kowalski, K.G., Ewy, W., Hutmacher, M.M., Miller, R., and Krishnaswami, S. “Model-Based Drug Development – A New Paradigm for Efficient Drug Development”. Biopharmaceutical Report 2007;15:2-22. • Lalonde, R.L., et al. “Model-Based Drug Development”. Clin Pharm Ther 2007;82:21-32. • Kowalski, K.G., Olson, S., Remmers, A.E., and Hutmacher, M.M. “Modeling and Simulation to Support Dose Selection and Clinical Development of SC-75416, a Selective COX-2 Inhibitor for the Treatment of Acute and Chronic Pain”. Clin Pharm Ther, 2008; 83: 857-866. • Kowalski, K.G., French, J.L., Smith, M.K., Hutmacher, M.M. “A model-based framework for quantitative decision making in drug development”. Presentation at ACOP, Tuscon, AZ. 2008. http://tucson2008.go-acop.org/pdfs/8-Kowalski_FINAL.pdf