Critical Path: An Overview

1. Critical Path: An Overview Challenges and Opportunities for Statisticians in The Drug Development Process Charles Anello, Sc.D. Deputy Director OB, OPaSS, CDER, FDA C. Anello BASS XI 2004

2. Disclaimer The views expressed in this talk are those of the author and do not necessarily represent those of the Food and Drug Administration C. Anello BASS XI 2004

3. Talk Outline • Drug Development Process • Definition of Critical Path • Three Dimensions of Critical Path • Statistical aspects of Critical Path • Strategic Plan or Way forward • Summary C. Anello BASS XI 2004

4. Ten Year Investment Trends Doubling over 5 years of NIH funding Pharmaceutical R&D investment increasing at the same rate Major investments in biotechnology C. Anello BASS XI 2004

5. C. Anello BASS XI 2004

6. Ten Year Trend in Product Submissions Decline in original BLAs Submissions Decline in NMEs

7. C. Anello BASS XI 2004

8. Development Costs are Escalating • Costs of bringing a successful drug to market estimated between $0.8 – 1.7B • Higher failure rate of candidates in clinical development C. Anello BASS XI 2004

9. Critical Path Initiative • Federal Register: April 22,2004 • Innovation Challenge and Opportunity on the Critical Path to New Medicinal Products • Request for comments ( hurdles , priorities and possible solutions, FDA role etc.) • Interested parties were given unit a chance to comment C. Anello BASS XI 2004

10. The Critical Path for Medical Product Development

11. Critical Path • Defined as those aspects of the drug developmental process that can impact the safety , efficacy and quality of medial products C. Anello BASS XI 2004

12. Three Dimensions of the Critical Path Assessment of Safety – how to predict if a potential product will be harmful? Proof of Efficacy -- how to determine if a potential product will have medical benefit? Industrialization – how to manufacture a product at commercial scale with consistently high quality?

13. Working in Three Dimensions on the Critical Path

14. Office of Biostatistics: Critical Path Initiatives • Conduct Research, Gain Consensus and Develop Guidance to Remove Obstacles to efficient Drug Development • Improve the processes and Approaches to Quantitative Analysis of Safety Data • Apply Modern Statistical approaches to Product Testing and Process Control C. Anello BASS XI 2004

15. Critical Path Statistical Issues • Missing Data • Flexible and Adaptive designs • Non-Inferiority • Multiple endpoints • Modeling and simulation • Bayesian Methods • Drug Quality Control Methodology C. Anello BASS XI 2004

16. Clinical Trial Methodology • Better use and analysis of safety data collected in clinical trials to facilitate risk estimation, risk management and risk communication • Enhance product testing and characterization during the drug manufacturing process C. Anello BASS XI 2004

17. Missing Data due to patient withdrawals and drop out • Patients leave a study for many reasons (lack of effect, side effects, removed by PI because patient stopped study medication, etc) • Clinical Trials may have few (1%) to many (say 50%) of drop outs. • Since the goal is to analyze all the patients randomized missing data presents a real problem. • Of special concern is informative censoring, how to identify it and how to adjust for it. C. Anello BASS XI 2004

18. Missing Data due to patient withdrawals and drop out • For decades the Agency has relied on Last Observation Carried Forward (LOCF) and has been criticized for this. • Numerous alternative imputation methods have been developed and proposed • But consensus on the best approach has been hard to achieve C. Anello BASS XI 2004

19. Missing Data due to patient withdrawals and drop out • This problem impacts both safety and efficacy and the ability to make sound benefit risk decisions • FDA is in a unique situation because it sees many different types of missing data problems and many different types of approaches to the analysis of Clinical Trials with missing data • Also, how the study is designed could impact the nature and extent of the missing data problem. C. Anello BASS XI 2004

20. Adaptive/Flexible Clinical Trial Design • Most Phase III clinical trials are based on a fixed sample size design • The success of the trial will depend on the validity of the protocol planning assumptions • Our experience shows that trials may fail because of inadequately chosen doses, patient populations, primary endpoints, or anticipated effect size C. Anello BASS XI 2004

21. Adaptive/Flexible Clinical Trial Design • Flexible / Adaptive designs may provide more structure to the learn and confirm paradigms • Flexible designs require interim looks at the data ( maybe unblinded) • The applicability to the regulatory setting needs to be further explored C. Anello BASS XI 2004

22. Active Control Studies • The ethics of clinical trial has led to the more extensive use of active control clinical trails • ICH E10 has dealt extensively with ACTs • The importance of assay sensitivity and defining the margin have emerged as critical design features. C. Anello BASS XI 2004

23. Active Control Studies • There is a need for clarity for a regulatory perspective. (How to define the margin, what is the goal of the non-inferiority trial) • Several methods have emerged as approaches to NI trial ( the syntheses method and the confidence method) • But there is a lack of agreement on the best approach. C. Anello BASS XI 2004

24. Active Control Studies • The use of ACTs depends on the availability and the quality of the historical evidence • If there is little confidence in the historical data should ACTs be conducted? • Is the concept of % retention a viable approach? • The CP goal is to try to reach a consensus C. Anello BASS XI 2004

25. Multiple Endpoints • All drug-disease areas try to define clinically meaningful treatment outcomes • Often more than one endpoint is studied in clinical trials • Currently there is little consensus on how to treat these multiple endpoints and what kinds of adjustments are needed to maintain a pre-specified alpha error. C. Anello BASS XI 2004

26. Multiple Endpoints • Multiple endpoints in a regulatory setting have approval implications • Multiple endpoints in a regulatory setting have label implications • The recent( Oct. 20-21) PhRMA workshop in Washington DC focused completely on this topic. C. Anello BASS XI 2004

27. Multiple Endpoints • The main problem is how to get a consensus on multiple primary and secondary endpoints. • This is not because of a lack of statistical models or proposals, the literature has many suggestions. • FDA is trying to take advantage of it’s regulatory experience and has engaged the clinical and statistical staff to try to specify how to define this problem and to recommend best practices C. Anello BASS XI 2004

28. Multiple Endpoints (Huque PhRMA 2004) Why in some trials more than 1 endpoints needs to show effects? • Multiple Endpoints involve both clinical and statistical concepts • The choice of endpoints may depend on where the endpoints lie on the causal pathways of the disease process and the mechanisms of actions of the study intervention • Clinical expectation of the desired clinical benefit C. Anello BASS XI 2004

29. Multiple Endpoints (Huque PhRMA 2004) Acne trial example: “Clinical Win” criterion • Three primary endpoints: X = non-inflammatory lesion counts, Y = inflammatory lesion counts, Z = physician global • Clinical Decision rule for effectiveness: (1) Z must show statistical significance. (2) In addition, X or Y must show statistical significance (without showing worsening in any) • Possible Rationale: X and Y lie on different causal pathways, and Z intersect with both. C. Anello BASS XI 2004

30. Multiple Endpoints (Huque PhRMA 2004) Rheumatology Example (ACR20): • Required: • at least 20% improvement in tender joint count • at least 20% improvement in swollen joint count • Plus at least 20% improvement in 3 out of the 5 • patient pain assessment • patient global assessment • physician global assessment • patient self-assessed disability • acute phase reactant (ESR or CRP) C. Anello BASS XI 2004

31. Multiple Endpoints (Huque PhRMA 2004) each endpoint at level 0.025 (1-sided) Y-axis: Type I error probability X-axis: Correlation C. Anello BASS XI 2004

32. Multiple Endpoints (Huque PhRMA 2004)Adjustments in the Type I error rateCase of 2 Endpoints Adjstment by Sidak’s method on accounting for correlation C. Anello BASS XI 2004

33. Multiple Endpoints (Huque PhRMA 2004)Power Comparison Case of K=2 endpoints: C. Anello BASS XI 2004

34. Multiple Endpoints (Huque PhRMA 2004)Inference about individual endpoints in clinical trials: • Uses a procedure that controls the family-wise Type I error rate appropriate fore endpoint specific claims • Examples: • “closed testing” procedure • gate-keeper/fixed-sequence methods • alpha-calculus approach • other approaches Reference (SAS publication 99’):.Westfall PH, Tobias RD, Rom D. Wolfinger, R.D.; Hochberg,Y. Multiple Comparisons and Multiple Tests C. Anello BASS XI 2004

35. Multiple Endpoints (Huque PhRMA 2004)Extent of multiplicity adjustments between endpoints correlation high Practically no adjustments Small adjustments Good case for combining endpoints Large adjustments low high low Causal dependence (Homogeneity of treatment effects across endpoints) C. Anello BASS XI 2004

36. Multiple Endpoints (Huque PhRMA 2004)Triaging of multiple endpoints into meaningful families by trial objectives • Hierarchical ordered families 1) Prospectively defined 2) FWE controlled Primary endpoints Secondary endpoints Exploratory endpoints (usually not prospectively defined) • Primary endpoints are primary focus of the trial. Their results determine • main benefits of he clinical trial’s intervention. • Secondary endpoints by themselves generally not sufficient for characterizing • treatment benefit. Generally, tested for statistical significance for extended • indication and labeling after the primary objectives of the trial are met. C. Anello BASS XI 2004

37. Modeling and Simulation • Has had a minor role to date in the planning and analyses of clinical trials in a regulatory setting • But with the advent of high speed computers, a better understanding of the mechanism of drug action and a desire to minimize the number of clinical trials that are failing FDA is taking a another look at this approach. C. Anello BASS XI 2004

38. Modeling and Simulation • Failures of clinical trials can be traced to ill chosen dose, inadequate numbers of patients, studies that where to short in length, not anticipating the number and nature of dropouts and the impact of combining different subgroups into a clinical trial • M/S has a potential role in testing the implications of the underlying assumptions before the first patient is entered into a clinical trial C. Anello BASS XI 2004

39. Modeling and Simulation • The first problem will be to understand the scope and utility of M/S in the drug development process • The second level will be to find or train the necessary expertise in the use of M/S • The third issue is to find or develop the necessary computational tools that are designed for the clinical trial planners and analyzers and are user friendly C. Anello BASS XI 2004

40. Defining the role of Bayesian Methods in a regulatory Setting • Traditionally the design and analyses of clinical trials has been centered on the frequentist approach of hypothesis testing, estimation, p-values and confidence intervals • In recent years, with the advances in computational methods we are beginning to see Bayesian designed clinical trials and analyses in the regulatory setting. C. Anello BASS XI 2004

41. Defining the role of Bayesian Methods in a regulatory Setting • Bayesian methods focus on how belief can be modified by the data • Even though this is an active area of statistical research and methodological development it’s utility in a regulatory setting has yet to be established. C. Anello BASS XI 2004

42. Defining the role of Bayesian Methods in a regulatory Setting • FDA has been proactive in recent years by providing training to the statistical staff on Bayesian methods and acquiring computational tools which will support the evaluation of submission which include Bayesian analyses • In May of 2004 FDA and the Johns Hopkins University held a workshop on Bayesian Approach. (the proceeding of this workshop are being prepared for publication) C. Anello BASS XI 2004

43. Defining the role of Bayesian Methods in a regulatory Setting • To date there is no guidance on what kinds of trials or clinical settings are candidates for the application of Bayesian methods • There is concern about can these methods give the right answer to the right question • Can the type I error rate be properly controlled C. Anello BASS XI 2004

44. Defining the role of Bayesian Methods in a regulatory Setting • Will the Bayesian approach be able to deal with the other issues discussed, multiple endpoints, missing data, and non-inferiority • Will priors be based on data or subjective opinions • While CDRH has been adopting Bayesian methods for years CDER is just beginning to explore this approach. • We need to develop an understanding of what are the regulatory situations where BM can be applied with confidence and how to document the results of these types of designs and analyses C. Anello BASS XI 2004

45. Risk assessment, management and Communication • In June 2002, Congress passed the Prescription Drug User Fee Act (PDUFA III) • FDA had a performance goal in the area of drug safety • In May of 2004 FDA issued three draft guidance to comply with this Act. FDA is currently reviewing to comments to these drafts C. Anello BASS XI 2004

46. Risk assessment, management and Communication • It is clear that over the past few decades less attention has been given to the design and analyses of safety compared to efficacy • Safety data is not consistently collected, analyzed and reported • While there is a requirement for a statistical analysis plan for efficacy none has been required for safety. C. Anello BASS XI 2004

47. Risk assessment, management and Communication • The assessment of early phase I and II safety data needs to happen before one proceeds to Phase III trial • Certain safety areas require special attention ( hepatotoxicity, QT prolongation, nephrotoxicity and the rigorous assessment of informative dropouts from clinical studies) C. Anello BASS XI 2004

48. Risk assessment, management and Communication • FDA draft “ Guidance for Industry Premarketing Risk Assessment”( Docket Number 2002D-0187) defines the problem and has suggestions for improvement • There has been significant FDA/PhRMA interaction on this topic (including workshops and working groups) • Also, FDA has several CRADAs to develop efficient tools for reporting, looking at and analyzing safety data C. Anello BASS XI 2004

49. Enhancing Product Quality • Traditionally pre-marketing product quality has been tested by an end product multiple batch approach • Often post-marketing product quality requires a modification of this approach • The current designs for testing product quality my not be efficient C. Anello BASS XI 2004

50. Enhancing Product Quality • Modern in process testing raises the possibility that alternatives to product quality should be considered • There have also been advances in Process Analytical Technology (PAT) which depends on in process assess of product quality all along the drug manufacturing process C. Anello BASS XI 2004