Modelling and simulation in the pharmaceutical industry

1. Modelling and simulation in the pharmaceutical industry Reflections from a statistician’s and engineer’s perspective Carl-Fredrik Burman, PhD, Assoc Prof Senior Principal Scientist AstraZeneca R&D Mölndal

2. Agenda • Seven theses about good modelling • It is about making better decisions • It is driven by the underlying question • It is based on applied sciences • It uses a diversity of information sources • It is not made unnecessarily complicated • It is a continuous process • It facilitates communication • Some thoughts on simulation • Concluding remarks Global Medicines Development | Biometrics & Information Science

3. Seven theses about good modelling • Based on Burman & Wiklund (Pharm Stat 2011) • The seven theses are partly overlapping • The intention is rather prescriptive than descriptive Global Medicines Development | Biometrics & Information Science

4. Are the theses just saying the obvious? Global Medicines Development | Biometrics & Information Science

5. 1. Good modellingis about making better decisions

6. Don’t model unless you see which decision could be improved by your model Global Medicines Development | Biometrics & Information Science

7. Value of Modelling (Cf. Value of Information) • Data, information, models may have a larger or smaller value depending on the situation • X available data • D decision • M “modelling” • V value (in e.g. Swiss franc, or total patient benefit) • Value of Modelling = VoM = E[ V( D(X,M) ) ]  E[ V( D(X) ) ] • Value only(?) through changing the decision Global Medicines Development | Biometrics & Information Science

8. Value of Modelling A simple example • Value of Modelling = VoM = E[ V( D(X,M) ) ]  E[ V( D(X) ) ] • Say that we have a pure go / no go (dichotomous!) phase III investment decision • If it’s pretty obvious that we should “go”, modelling doesn’t help • However, if this is a tough decision, modelling (e.g. predicting through biomarker-clinical endpoint relation, extrapolating over time and population) may be very valuable. • Frontload: Start modelling ph III before ph II investment Global Medicines Development | Biometrics & Information Science

9. Value of Modelling Example (cont’d) • Value of Modelling = VoM = E[ V( D(X,M) ) ]  E[ V( D(X) ) ] • Say that we face a phase III investment decision that is not purely dichotomous, but concerns • Go / No Go • Dose selection • Population • Sample size • Then, all these may add more or less to the overall value • Highly dependent on specific situation Global Medicines Development | Biometrics & Information Science

10. Decision focus Don’t necessarily have to be one single decision • Useful to update model continuously (Thesis 6) • Series of decisions • Adaptive Programmes • Cross-over to new projects • Obvious for same indication • But trial data for one drug can sometimes help to give useful background information for completely different indications (variability, disease progression) Global Medicines Development | Biometrics & Information Science

11. “Soft” but important? • (Too) many modellers have been vague about the aim of their work • Describe the system • Improve understanding • Write scientific papers • ... • Modelling can be used for summarising information, predicting, gaining insights, etc. • However, the benefits of e.g. a ‘gained insight’ will be realised when the insight is reflected in an actual decision. Global Medicines Development | Biometrics & Information Science

12. Industry vs. Academia • I think the luxury of “non-purpose” modelling should sometimes be allowed • Cf. pure mathematics • but more often in academia Global Medicines Development | Biometrics & Information Science

14. 2. Good modellingis driven by the underlying question

15. “All models are wrong, some are useful” George Box Global Medicines Development | Biometrics & Information Science

16. Don’t search for the ultimate model • “All models are wrong ...” A model cannot incorporate all available information and be capable of answering all relevant project questions. • “... some are useful” The usefulness depends on the purpose of modelling, on which decision we set out to support. • Fit for purpose The model should thus be tailored to the concrete project question (Thesis 5). Global Medicines Development | Biometrics & Information Science

17. Different objectives  different models needed • Example 1: Testing overall placebo-controlled efficacy • Example 2: Is the drug less efficacious than a competitor drug in elderly? If so, where’s the cut-off? • Example 3: Dose adjustment possible based on age or pharmacokinetic exposure (that correlates with age) Global Medicines Development | Biometrics & Information Science

18. Need to understand the real question • Common consultancy experience: • Clients may ask the wrong question • The modeller should try to understand the overall context Global Medicines Development | Biometrics & Information Science

19. 3. Good modellingis based on applied sciences

20. Be a scientist, not a narrow-minded statistician! Global Medicines Development | Biometrics & Information Science

21. Modelling “Reality” Mathematics To me, modelling is about translating from the real problem to mathematics, and going back from a mathematical solution to a practical solution. Global Medicines Development | Biometrics & Information Science

22. Note! Modelling in my narrow sense is not about solving the mathematical problem! • Modelling process: • Understand the project problem • Formulate objective • Map reality onto mathematic • Solve mathematical problem • Translate back to give decision support • Check robustness • Communicate Global Medicines Development | Biometrics & Information Science

23. Good modelling requires the utilisation of several scientific areas, not only statistics • Statisticians often have excellent skill-sets for modelling work. • However, we may need to transcend traditional roles and eagerly seek to understand the essence of the project’s problem. • Think and act as scientists in a wider sense, focussed on providing useful decision support, irrespectively of what kind of methods that are needed. • Avoid “He that is skilled with a hammer tends to think everything’s a nail” • Collaborate! Global Medicines Development | Biometrics & Information Science

24. 4. Good modellinguses a diversity of information sources

25. Combining different information sources Several pieces to the puzzle • Often, we need several components, e.g. • Dose-response • Time dependence • Biomarker – clinical endpoint • Variability (between patients, over time, day-to-day, measure-to-measure) • to build a useful model • Information needed will often come from different sources Global Medicines Development | Biometrics & Information Science

26. Different information sources • In-house randomised clinical trials • Pre-clinical data • Competitor data • Observational studies • Literature information • Expert knowledge • ... Global Medicines Development | Biometrics & Information Science

27. Combining different information sources … to estimate a common parameter • Scientific insights needed to see the relevance of information • In principle, the Bayesian framework is readily applicable as it treats all types of uncertainties in the same way. • In-house design decisions can be guided by Bayesian decision theory, even if regulators and other costumers require frequentist analyses Global Medicines Development | Biometrics & Information Science

28. 5. Good modellingis not made overly complicated

29. Not overly complicated Ockham’s razor: “Pluralitas non est ponenda sine neccesitate” (“Entities should not be multiplied unnecessarily'‘) William of Ockham, 1285–1347/49 “A model should be as simple as possible and yet no simpler” Albert Einstein, 1879-1955 Global Medicines Development | Biometrics & Information Science

30. Efficiency of modelling work • Don’t spend time on modelling that likely have ignorable impact on the decision problem. • The first question to ask is whether modelling is worthwhile, i.e. to assess whether the net ‘Value of Modelling’ is positive. Global Medicines Development | Biometrics & Information Science

31. Marginal Value of Modelling • How much effort to put into modelling? • Engineering: Quick and dirty! • Value of Modelling VoM = E[ V( D(X,M) ) ]  E[ V( D(X) ) ] Global Medicines Development | Biometrics & Information Science

32. Example • Pharmacokinetics / pharmacodynamics (PK/PD) modelling can give great benefits. • However, if it’s clear that one single dose is sought, the PK component will typically not be important. • Why estimate f(PK(d)), when you only need f(d)? Global Medicines Development | Biometrics & Information Science

33. 6. Good modellingis a continuous process

35. Continuous learning • A model made for one decision point can often be re-used, updated and applied to a later decision. • The ‘Learning Loop’ provides a description of the continuous modelling process and the interaction with (design) optimisations and information retrieval. • The greatest benefits will likely be achieved with ’model-based drug development’, where modelling is fully integrated in the process. Global Medicines Development | Biometrics & Information Science

36. 7. Good modellingfacilitates communication

37. Transparency • Modelling is not a concern only for the quantitative scientists. • Modelling is not replacing the decision-makers, but supporting them. Global Medicines Development | Biometrics & Information Science

38. Decision support • Present not only the optimal solution, but also • Assumptions • Robustness checks. • The ideal is that the decision-makers can challenge assumptions and interactively study the consequences of altering them. • Successful modelling processes provide major benefits in transparency within the teams regarding underlying assumptions, and facilitate communication with governance bodies and decision-makers. Global Medicines Development | Biometrics & Information Science

39. Some thoughts about simulation

40. Simulation A piece of cake? • We are considering “simple” clinical trial simulation, not e.g. MCMC • For most statisticians, it is trivial to simulate a simple clinical trials, with sufficient precision • However, many modelling problems are more complicated, including • a range of scenarios • multidimensional optimisation. • In our experience, simulation studies are often ineffective, leading to inadequate precision in results. Global Medicines Development | Biometrics & Information Science

41. Give confidence intervals for simulation results! Global Medicines Development | Biometrics & Information Science

42. Simulation Alternatives • Consider alternatives to simulation • analytic solutions • Numerical analysis • Approximations • In many cases, parts of the problem can be solved by such means, leaving a much simpler problem to stochastic simulations. Global Medicines Development | Biometrics & Information Science

43. Simulation Simplify • Don’t make your simulation model overly complicated! • Simulate sufficient statistics, not individual data • Approximate. E.g. central limit theorem. Global Medicines Development | Biometrics & Information Science

44. Simulation Reduce simulation variability • Simulations are often applied to compare the efficiency of competing designs, e.g. of a dose-finding trial. • Two designs with different doses can effectively be compared by using the same simulated residuals for both designs. • This can greatly reduce the variance of a difference of estimates. • A similar trick can be used when the doses are the same but different allocations (e.g. larger placebo group) are considered. Global Medicines Development | Biometrics & Information Science

45. Remember: Confidence intervals! Global Medicines Development | Biometrics & Information Science

46. Concluding remarks

47. The role of the pharma statistician • An arguably conservative attitude has served us well • Appropriate in the traditional core area: the analysis of (confirmatory) clinical trial data. • However, time they are a-changing. • An increasing importance on complex issues in programme design and decision support • To more effectively add value, statisticians need to adopt a more flexible mindset and be willing to embrace new, useful methodology. Global Medicines Development | Biometrics & Information Science

48. Which designs are possible? Alternative designs What do we know already? Where do we want to go? Optimise design, based on model & objectives Modelling Objectives Simulations / Computations

49. Confidentiality Notice This file is private and may contain confidential and proprietary information. If you have received this file in error, please notify us and remove it from your system and note that you must not copy, distribute or take any action in reliance on it. Any unauthorized use or disclosure of the contents of this file is not permitted and may be unlawful. AstraZeneca PLC, 2 Kingdom Street, London, W2 6BD, UK, T: +44(0)20 7604 8000, F: +44 (0)20 7604 8151, www.astrazeneca.com