P romoting high-quality and informative exploratory development: obstacles and aids

1. Promoting high-quality and informative exploratory development: obstacles and aids Michael O’Kelly, Centre for Statistics in Drug Development, Innovation

4. Obstacles to informative exploratory development • precedent • when clinical - quantitative questions overlap • money

5. Obstacles to informative exploratory development • precedent • when clinical - quantitative questions overlap • money “We’ve developed a similar treatment with these study sizes before – your calculations must be wrong” Take account of the variability in your assumptions – but isn’t that Bayesian? Phase III, same calculation as phase II? sample size

6. Obstacles to informative exploratory development • precedent • when clinical - quantitative questions overlap • money Senn and Julious, 2009, Measurement in clinical trials, Statistics in Medicine Responder, required in obesity, diabetes, PD, depression.... Percentage increase from baseline endpoints Fixed “adjustment” of e.g. QTc, CCI

7. Obstacles to informative exploratory development • precedent • when clinical - quantitative questions overlap • money I heard adaptive designs are great. Let’s try one right now. “We can’t afford the original design – use an adaptive design instead” “Use a dose-response curve? But we like having proof of a working dose.” study design But people don’t usually use log response here... Two doses worked for the KOL...

8. Obstacles to informative exploratory development • precedent • when clinical - quantitative questions overlap • money “...he must either destroy himself or the whole theory of probable errors” 1922: Fisher identifies the correct degrees of freedom for Pearson’s chi-squared Pearson: “erroneous...the writer has done no service to statistics by giving it broad-cast circulation in the pages of the JRSS...” Fisher could not get his rebuttal published in the JRSS

9. Obstacles to informative exploratory development • precedent • when clinical - quantitative questions overlap • money Mixed models repeated measures LOCF This statistical approach got approval last time Bonferroni and robust models?

11. Aids to informative exploratory development

12. Aids to informative exploratory development • High level executive support for quantitative approach • Continuous education • Better communication • A requirement for one company’s effort to reform trial design: “exhibit desirable behavioural characteristics and communication skills” • Modelling and simulation – an aid to communication • case study

13. Case study: how modelling and simulation helps • partnership arrangement, CRO with sponsor • multifunctional team responsible for development of treatment, MOA not well known • phase IIa small dose escalation study • phase III large two-arm study • phase IIb design that fits with these to get the best dose approved? • lean, two-arm phase IIb study submitted to regulator • alternatives?

14. Case study: how modelling and simulation helps • partnership arrangement, CRO with sponsor • multifunctional team responsible for development of treatment, MOA not well known • phase IIa small dose escalation study • phase III large two-arm study • phase IIb design that fits with these to get the best dose approved? • lean, two-arm phase IIb study submitted to regulator • difficult for team to get handle on risks and benefits of other designs • “get handle on”: • explore, think about, become familiar with, feel some confidence in, talk freely about, lose fear of... • many possible scenarios • scenarios: plausible safety and efficacy profiles across doses

15. Scenarios to cover best cases… Scenario analyses … through to worst cases

16. Case study : how modelling and simulation helps • Modelling and simulation helped the multifunctional team to think and talk about alternative designs in a quantitative way • team discussed simulation of 1st strategy (1 dose in phase IIb), • ->led to 2nd strategy: 4 doses in phase IIb – team discussion of new simulation • ->led to 3rd strategy: 2 doses in phase IIb – team discussion of new simulation • ->led to 4th strategy: adaptive design 4 doses, select 2 • Modelling an entire program allows complex implications to be thought of in graspable terms • implications of the study design • of the statistical model • of the statistical analysis • of plausible potential quirks or irregularities in the efficacy or safety of the doses of the treatment

17. Problems with simulations... • often the results depend heavily on the assumptions • resource-intensive • quality control of simulations rarely described • difficult to prove wrong! • E.g. current “contest of the methods” in dose-finding, PhRMA group and others • Kirby, Colman, Morris, 2009, “Dose response...using smoothing splines”, Pharmaceutical Statistics • “Although not explicitly stated the results for the Dopt method in the PhRMA paper are for a logistic model with a polynomial trend (Dmitrienko, personal communication).”

18. Simulations: what can we do? • collaborative approach to simulation • shared “machinery” • like-for-like comparisons => more rapid discovery of real improvements in study design • e.g. Pfizer MStoolkit software available in R • PSI new special interest group (SIG) for Modelling and Simulation • meeting tomorrow at Amgen in Cambridge ! • (email Vincent Haddad at VHaddad@amgen.com) • all welcome to attend (numbers limited)

19. Questions?

20. Backup slides: details of the case study results

21. Strategy #1: original plan Phase IIa Phase IIb Phase III Active Active Active dose 1 2 3 4 500 subjects 500 subjects 12 12 12 12 6 6 6 6 Placebo 500 subjects 500 subjects Placebo Placebo

22. Strategy #2: statisticians’ idea: use many doses in IIb Phase IIa Phase IIb Phase III Active Active Active dose 1 2 3 4 200 200 200 200 500 12 12 12 12 6 6 6 6 Placebo 200 20 500 Placebo Placebo

23. Strategy #3: team came back with a “less radical” design Phase IIa Phase IIb Phase III Active Active Active dose 1 2 3 4 333 333 500 12 12 12 12 6 6 6 6 Placebo 333 500 Placebo Placebo

24. Strategy #4: the virtues of #2 and #3? Adaptive design Phase IIa Phase IIb part 1 Phase III Active Active Active dose 1 2 3 4 100 100 100 100 500 12 12 12 12 6 6 6 6 100 Placebo 500 Placebo Placebo

25. Strategy #4: the virtues of #2 and #3? Adaptive design Phase IIa Phase IIb part 2 Phase III Active Active Active dose 1 2 3 4 260 260 500 12 12 12 12 6 6 6 6 Placebo 260 500 Placebo Placebo

26. Scenarios to cover best cases… Scenario analyses

27. Scenarios to cover best cases… Scenario analyses … through to worst cases

28. Scenarios with viable doses Scenario analysis results – Phase III success

29. Scenarios with viable doses Scenario analysis results – Phase III success

30. Scenarios with no viable doses Scenario analyses results – stopping in IIb