An Update on FDA’s Critical Path Initiative Statistical Contributions

1. An Update on FDA’s Critical Path InitiativeStatistical Contributions Robert T. O’Neill Ph.D. Director , Office of Biostatistics Center for Drug Evaluation and Research Presented at the 2005 FDA/Industry Statistics Workshop: September 14-16, 2005 Marriott Wardman Park Hotel, Washington, DC

2. The Critical Path Initiative • Refers to the product development path from candidate selection to product launch • Covers drugs, biologics, and medical devices – but today’s talk is mostly about drugs / biologics • Initiative was announced publicly by Dr. McClellan Tuesday, March 16, 2004

3. What the “Critical Path” Is • A serious attempt to bring attention & focus to the need for more scientific effort and publicly-available information on evaluative tools • Evaluative tools: The techniques & methodologies needed to evaluate the safety, efficacy & quality of pharmaceuticals as they move down the path

5. Despite Advances in Science, Success Rate of Product Development has NOT Improved • New compounds entering Phase I development today have 8% chance of reaching market, vs. 14% chance 15 years ago. • Phase III failure rate now reported to be 50%, vs. 20% in Phase III, 10 years ago.

6. Perceived Problem: The development process itself is becoming a serious bottleneck • Current applied science and infrastructure date from last century • Funding and progress in Development science has not kept pace with basic biomedical science. • Science to evaluate safety and efficacy of potential new medical products, and enable manufacture, is different from basic discovery science. • Need to fill gap in applied science-- to increase productivity and efficiency --to reduce cost of development process.

7. Stakeholder Input: Overwhelming Support • Overwhelming concurrence with: • recognition of science infrastructure problem • CP Initiative focus on research and collaboration, • We heard this from: drug industry, patient groups, device companies and groups, biotech companies, others

8. This is what we heard !Demand Exceeds Supply • Docket Demand for FDA Action Exceeds FDA Capacity: Far more proposed than FDA can undertake. • Principles for setting priorities for FDA actions are on Science Board agenda.

9. Overriding Concerns • Clinical Trials • Biomarkers and Endpoints

10. What is the problem • Phase III trials are failing at a rate that is higher than expected - root causes ? • What is the typical planning process for drug development / phase 3 trials • What can we change; what new tools can we use, and what can we do better in the future to improve Phase III success and efficiency of drug development

11. Possible solutions / strategies Can statisticians help ? • Are new study designs needed • Impetus for Adaptive designs, two stage designs, enriched target population designs • Are we planning correctly - Rethink how the study planning process occurs • It’s the dose • It’s the scenario needing better planning - or analysis methods • Bring consensus / closure to most pressing statistical issues at the core of decision making • Get involved in new emerging subject matter areas and impact them -genomics, proteonomics, nanotechnology • Broaden the multi-disciplinary roles, in industry, academia and regulatory bodies - internationally

12. Our Proposal for the Critical Path • Conduct Research , Gain Consensus, and Develop Guidance to Remove Obstacles to Efficient Drug Development and Enhance Success Rates of Clinical Trials • Improve the Processes and Approaches to Quantitative Analysis of Clinical Safety Data from Clinical Trials to Enhance Risk Assessment and Management Initiatives • Improve the Statistical Understanding and Application of Modern Statistical Approaches to Product Testing and Process Control

13. Clinical Trial Proposals for the Critical Path • Missing data due to patient withdrawals and dropouts in clinical trials • Flexible / adaptive clinical trial designs to improve the information and success rate of trials • Non-inferiority active control studies when placebos can't be used - getting to consensus on appropriate methods for margin setting, data analysis and interpretation for various data rich and data poor scenarios • Development of consensus on the statistical handling of multiple endpoints in clinical trials. • Clinical trial modeling and simulation as a tool for better design and interpretation of clinical trials • Application of Bayesian Methods to Enhance the Success Rate of Clinical Trials

14. Prioritize Efforts - Three separate yet related approaches • Guidance Development • Multiple endpoints • Non-inferiority • Topics of high interest • Adaptive / Flexible designs • Modeling / simulation / planning/Phase 2a • Other Critical Path needs: safety , product quality

15. Safety and Quantitative Risk AssessmentClinical Trials - Pre-Marketing • Methods of application • Planning, data collection, statistical analysis plan • Process • Newly formed statistical safety team for more concentrated and focused advice • Earlier planning, modeling and simulation

16. FDA Risk Management GuidancesLife cycle of a drug • Premarketing Risk Assessment (Premarketing Guidance) • Development and Use of Risk Minimization Action Plans (RiskMAP Guidance) • Good Pharmacovigilance Practices and Pharmacoepidemiology Assessment (Pharmacovigilance Guidance)

17. Enhancing Product Quality • Modern in process testing raises the possibility that alternatives to product quality should be considered • There have also been advances in Process Analytical Technology (PAT) which depends on in process assessment of product quality all along the drug manufacturing process

18. The Non -Inferiority ProblemCurrent guidance is inadequate and the issues are poorly understood - must be fixed • Term introduced in ICH E9 ‘Statistical Principles for Clinical Trials’ • Some issues described in ICH E10 ‘Choice of Control Groups’ • A study design that provides an indirect measure of evidence of efficacy / safety

19. What are the various objectives of the non-inferiority design • To prove efficacy of test treatment by indirect inference from the active control treatment • To establish a similarity of effect to a known very effective therapy - e.g. anti-infectives • To infer that the test treatment would have been superior to an ‘imputed placebo’ ; ie. had a placebo group been included for comparison in the current trial. - a new and controversial area - choice of margin is the key • To preserve a specified % effect of the AC

20. How is the margin “  “ chosen based upon prior study data • For a large treatment effect, it is easier - a clinical decision of how similar a response rate is needed to justify efficacy of a test treatment - e.g. anti-infectives is an example. • For modest and variable effects, it is more difficult ; and some approaches suggest margin selection based upon several objectives.

21. Complexities in choosing the margin (how much of the control treatment effect to give up) • Margins can be chosen depending upon which of these questions is addressed: • how much of the treatment effect of the comparator can be preserved in order to indirectly conclude the test treatment is effective - a clinical decision for very large effects; a statistical problem for small and modest effects • how much of a treatment effect would one require for the test treatment to be superior to placebo, had a placebo been used in the current active control study - a lesser standard than the above

22. How convincing is the prior evidence of a treatment effect ? • Do clinical trials of the comparator treatment consistently and reliably demonstrate a treatment effect - when they do not, what is the reason ? • Study is too small to detect the effect - under powered for a modest effect size • The treatment effect is variable, and the estimate of the magnitude will vary from study to study, sometimes with NO effect in a given study - a BIG problem for active controlled studies (Sensitivity to drug effect)

23. Importance of the assumption of constancy of the active control treatment effect derived from historical studies • It is relevant to the design and sample size of the current study, to the choice of the margin, to the amount of bias built into the comparisons, to the amount of effect size one can preserve (both of these are likely confounded), and to the statistical uncertainty of the conclusion. • Before one can decide on how much of the effect to preserve, one should estimate an effect size for which there is evidence of a consistent demonstration that effect size exists.

24. Four approaches to the problem • The simple case: specify a delta - not estimated • Indirect confidence interval comparisons (ICIC)(CBER/FDA type method, etc.) - thrombolytic agents in the treatment of acute MI • Virtual method (Hasselblad & Kong, Fisher, etc.) - Clopidogrel, aspirin, placebo • Bayesian approach (Gould, Simon, etc.) - treatment of unstable angina and non-Q wave MI

25. Current Guidance on Multiple Endpoints is inadequateMultiple primary endpointsMultiple secondary endpointsComposite endpointsMultiple compositesHierarchiesPatient reported outcomesDecision Criteria for success A collaborative effort: PhRMA 2004 meeting on co-primary endpoints, manuscript

26. Emerging Interest in Adaptive / Flexible Trial Designs • Adaptive designs • Enrichment / pharmacogenomics • Sample size re-estimation • Design modification

27. New study designsWhy a need for adaptive / flexible designs ? • Enriching trials with patients having genomic profiles likely to respond or less likely to experience toxicity • Goal of an adaptive / flexible design • Mid study changes that prospectively plan for modifications that preserve Type 1 errors and maximize chances for success

28. Information adaptive designs / flexible designsControversialStatististical Methodology is AvailableWhy and where to use them?

29. Why the need for adaptation? Design specifications often entail at least partial knowledge of the values of many planning (primary or nuisance) parameters that are unknown or at best might be guessed crudely Sample size planning entails “educated” guess of effect size. Selection of a composite endpoint requires “educated” guess of where the potential effects lie and what noises may be. Others….. Hung

30. Addressing a process issue:Scenario Planning:A Tool to Increase the Success Rate of Phase III trials and to Enhance Drug Development PlanningIncorporates: Several linked linked study phases - continuumMultiple endpointsMissing dataUse of all information in the process Safety PlanningModeling and simulationFlexible designs / development sequence / international

31. What is Scenario Planning • Modern approach to protocol planning and choice of clinical study designs • Utilizing models for disease progression and endpoint selection • Utilizing simulation strategies for what if scenarios • Assumes input from other studies and planning efforts - planned sequences of studies may matter • An aid for prospectively planning integrated analyses

32. Disease Progression Modeling • Endpoint selection and evaluation • Trial Duration determination • Frequency and number of subject measurements • Tradeoffs between clinical endpoints and patient reported outcomes • Evaluate impact of missing data, informative treatment related censoring • Evaluate multiplicity implications

33. What would be observed if subjects had stayed in trial ? Impute values from subects staying in longer Test Control Which path do you choose ? Baseline Higher is bad 1 2 3 4 5 Visit

34. Disease Progression Models and Clinical Outcomes • What model captures the functional relationship of the disease progression and the clinical outcome(s) to be used to measure treatment effect • Can one function capture each of the clinical outcomes adequately • If not are several disease progression models used to express ‘response’

35. Modern Protocol /Development PlanningSensitivity / Scenario planning • Different statistical tools and strategies • Challenge and explore assumptions • More multidisciplinary involvement • It is more than sample size planning • Structured planning meetings that are different that current – formal Q & A’s not broad enough • Links between phase planning and modeling efforts – currently too limited and stove piped

36. Concluding remarksMeeting the Challenges of the Critical Path will require collaboration and resource allocation • Multidisciplinary / collaborative planning and evaluation is needed now more than ever because issues becoming more complex - guidances can’t solve this - resources, exposure, experience, training will • Efforts to move available appropriate statistical methods and concepts , possibly more complex, into the main stream by emphasis on understanding by the audience appropriate to the application • Guidances don’t help here - need resources that can understand and communicate • Efforts to maximize contributions of industry, academic and regulatory statisticians

37. Concluding remark -Priority setting - • Choosing the most pressing needs and the chances for success - currently being updated • This is a national effort - not just FDA’s initiative - it will take a major coordinated effort to make progress