Challenges and Opportunities for 21 st Century Pharmaceutical Statisticians

1. Challenges and Opportunitiesfor 21st Century Pharmaceutical Statisticians Christy Chuang-Stein, PhD Statistical Research and Consulting Center Pfizer Inc

2. Outline • Evolving role of statisticians in pharmaceutical industry • The external environment • Responses to the changes in the environment • Develop partnerships and collaborations • Examples of collaborations • A new metric for determining sample size for a confirmatory trial • Summary

3. Evolving Role of Stat in Pharma Industry Little use of Statistics ==> “Required” use of Clin Statistics ==> Tactical use of Statistics ==> Strategic use of Statistics & “Statistical Thinking” 1955 2009 and beyond

4. Industry Perspective: “Then” • Hired some statisticians to get things through the regulatory agency (mostly in the US) • “Number crunchers” to get analyses done • “Blessed” clinical trial designs with minimal intellectual participation except sample size • Clinical and manufacturing focus • Very little input outside of “necessary”, low involvement in non-clinical areas • Statisticians played a secondary role

5. Statistician’s Role: Now and the Future • Full and equal partner with basic, clinical & regulatory scientists. • Focus on experimental design anddevelopment strategy. • Application of statistical thinking throughout the life cycle of a pharmaceutical product. • Parallel development in other disciplines such as epidemiology, genomics, biomarker development, and risk management expands statistician’s contributions.

6. Statisticians in Pharma Industry • Being technically smart is not enough: • Understand the broad clinical, regulatory and public-health context • Communicate statistical strengths and weaknesses • Proactive, not passive: • Design: Options available, decision analysis • Execution: Quality control and risk mitigation • Analysis: Planned and unplanned, strengths/weaknesses • Interpretation: Pre-planned or data-driven • Presentations/Publications: Keeping audience in mind. Statistics is a Collaborative Science!!

7. The External Environment • Desire for more transparency in clinical research and clinical reporting • A perceived need for more public “control” over the search for valuable medicines • Loss of public confidence in the clinical research process and the pharmaceutical industry • Loss of public confidence in the regulators • An increasing demand for product safety

8. Our External Stakeholders • Health care providers • Patient care givers • Regulators • Journal editors • Shareholders • Health care payer • And most importantly -THE PATIENTS

9. Data Disclosure • Desire for more “transparency” in the clinical research process • Protocols • Results • Logistics • Registries of trials (www.clinicaltrials.gov) • Results from trials involving marketed products are posted (www.clinicalstudyresults.org). Some companies have extended this to all trials and maintained their own trial results websites.

10. How Can Statisticians Help? • Leverage the drive for transparency. • Develop partnerships with academia and government. • Within the limits allowed by internal rules and external regulations, share our processes and approaches to the outside world.

11. Leverage the Drive for Transparency • Improve study designs • All studies will be public and designs will be subject to public scrutiny. • Improve communication • Provide informative displays and presentations of results that are customized for the audience. • Endorse the use of common standards • Maximize the efficiency of collecting and sharing data via the use of common standards.

12. Develop Partnerships • Foster partnerships among academia, industry and government. • Reasons: • Solve complex problems by sharing resources and knowledge • Integrate ideas and increase credibility through peer review • Create solutions that take into consideration differing perspectives • Belief: • Working together is more effective than working in isolation

13. Examples of Collaborations • Adaptive Designs • Multiple Co-primary Endpoints • Active Control Study • QT Effect of a Pharmaceutical Product • Methods to Handle Missing Data • Dichotomizing Continuous Endpoint • Biomarker Quantification • Predictive Model (Pre-clinical to Clinical) • Safety Data Evaluation • Multi-regional Clinical Trials

14. PhRMA Definition uses accumulating data to decide on how to modify aspects of the study without undermining the validity and integrity of the trial ADWG: Adaptive Design Working Group Validitymeans providing correct statistical inference (such as adjusted p-values, estimates and confidence intervals) minimizing operational bias Adaptive Designs Integrity means • preplanning, as much as possible, based on intended adaptations • maintaining confidentiality of data

15. Population End points Dose range Dosing frequency Exposure Drug formulation Sample size Add treatment Drop treatment Randomization fraction Population Sample size Modify delta Randomization fraction Drop treatment Population Level of uncertainty Development process Drug Development Process Exploratory phase Confirmatory phase

16. Adaptive Design Working Group • The Group was formed in 2005 of industry and academic members to investigate and facilitate opportunities for wider acceptance and usage of adaptive designs and related methodologies. • The Group met face-to-face about twice a year, has published numerous papers & given multiple presentations. • It has met with regulators in US, Canada, Europe and Japan to share experience and discuss issues of concern. • It continues to hold monthly telecons and sponsor a monthly Key Opinion Leader lecture series. • Is planning a workshop with the US Food and Drug Administration (FDA) to discuss a forthcoming FDA guidance on adaptive designs.

17. 10 Workstreams of ADWG • Education Effort • ICH harmonization • Data monitoring issues and processes • Good adaptive practices • Case studies to share experience • Software user requirements • Material manufacturing and supply • Communications • Key Opinion Leader lecture series • Documentation

18. Multiple Co-Primary Endpoints • There are many disorders where regulators require a new treatment to demonstrate statistically significant benefit on multiple endpoints before accepting the efficacy of the new treatment.

19. Definition of “Co-Primary” Endpoints • These endpoints describe the disease in the sense that an experimental drug that does not show superiority over placebo on all these endpoints is not a viable treatment for the disease. • This could arise because clinicians cannot agree on which endpoint (among several endpoints) is the most relevant one. So, they want evidence of effect on all of them. • In other words, they want statisticians solve the problem for them!

20. How Prevalent Is the Problem? Offen et al. Drug Information Journal (2007) 41(1):31-46.

21. Two Examples • Alzheimer’s Disease • Alzheimer’s Disease Assessment Scale - Cognitive (ADAS-Cog) • Clinician Interview-Based Impression of Change (CIBIC) • Vaginal Atrophy • Self-reported frequency of the moderate or severe symptoms that were most bothersome • Vaginal pH • % of vaginal parabasal cells from the maturation index • % of vaginal superficial cells from the maturation index

22. The Hypotheses • Assume (D1, …, DK) represents the mean effect of a new treatment over a placebo on K co-primary endpoints, positive Di implying positive treatment effect. • The null (H0) and alternative (H1) hypotheses are H0: D1£ 0 or D2£ 0 or … or DK£ 0 H1: D1 > 0 and D2 > 0 … and DK > 0 • Usual requirement: false positive rate £ 2.5% (one-sided).

23. Current Regulatory Practice • Test H0i vs H1i (the effect on the ith endpoint), each at the one-sided 2.5% level, i = 1, …, K. H0i: Di£ 0 H1i: Di > 0 • Declare efficacy only if all H0i (i = 1, …, K) are rejected at the one-sided 2.5% level. This is the intersection-union test (IUT). • IUT is necessary to control the false positive rate at the one-sided 2.5% level over the entire null space.

24. Overall Statistical Power *Power for each endpoint is 80%, one-sided a=0.025

25. Correlation Coefficient (AD) • Alzheimer’s Disease • Alzheimer’s Disease Assessment Scale - Cognitive (ADAS-Cog) • Clinician Interview-Based Impression of Change (CIBIC) • Correlation coefficient between ADAS-Cog and CIBIC is 0.22.

26. Correlation Coefficient (VA)

27. % Increase on Sample Size • Assume same effect size on each endpoint, tested at one-sided 2.5% level. The objective is to have an 80% overall power.

28. Possible Solutions • Optimal solution is a medical one • Reduce to a single dimension • Choose a single primary endpoint • Create a composite endpoint such as ACR20 for rheumatoid arthritis • Prioritize endpoints, requiring only a positive “trend” in some endpoints. A positive trend could be, e.g. a two-sided P-value < 0.15. • Propose statistical approaches that could provide rationales for adopting a higher significance level for testing each endpoint.

29. ACR20 (-50, -70) Definition • At least 20% (50%, 70%) improvement in • # of swollen joint count, AND • # of tender joint count, AND • At least three of the following • Patient’s global assessment of disease activity • Physician’s global assessment of disease activity • Patient’s assessment of pain (VAS, Likert, NRS) • Acute-phase reactants (ESR, CRP) • Patient’s assessment of disability (HAQ) • Often the primary endpoint to evaluate RA treatments.

30. Adopting a Higher Sig Level (1-sided) Chuang-Stein et al (2007), Stat in Med, 26:1181-1192.

31. Impact on Sample Size (K = 2) • The entry in black (red) corresponds to % increase in sample size under the new (current) approach.

32. Impact on Sample Size (K = 3) • The entry in black (red) corresponds to % increase in sample size under the new (current) approach.

33. A More Serious Issue • A common question – Why can’t a sponsor just increase the sample size to get an adequate overall power? • Statistical power gives the probability of concluding a treatment effect when the true treatment effect is the amount used to size the study. • For a trial to be successful, the new treatment needs to deliver the above assumed effect on all endpoints. • A greater demand of a new treatment!

34. A New Sample Size Paradigm • Relevant to trials expected to provide confirmatory evidence for product registration. • The primary objective of a confirmatory trial is to confirm that the effect of the new treatment is statistically better than a placebo. Recall that statistical power gives the probability of concluding a treatment effect when the true treatment effect is equal to the effect used to size the study. • But, we don’t know for sure if the treatment could deliver on the assumed effect.

35. An Example • A small PoC study, n = 25 per group, estimated treatment effect over a placebo = 2.5 units, estimated standard deviation = 7.14 units. So, estimated effect size = 0.35 (2.5/7.14). • The temptation is to design the next study to detect an effect size of 0.35 based on a two-sided test at the 5% significance level. • Need 128 subjects per group for 80% power, or 172 per group for 90% power. Source: Chuang-Stein (2006), Pharmaceutical Statistics, 5(4):305-309.

36. Question • What is the probability that we will have a successful trial? Define a successful trial as obtaining a P-value less than 5%. Other criteria can be used. • Is it 80% if we enroll 128 subjects per group, or 90% if we enroll 172 subjects per group?

37. A Frequentist-Bayesian Approach • We know there is variability in our estimate for the treatment effect. Assume we can describe this variability by D | PoC data ~ N(2.5 ; (2/25) (7.14)2) • We could obtain “average success prob” by

38. A Frequentist-Bayesian Approach • The success prob is 63.3% for 128 subjects per group (80% power); and 67.7% for 172 subjects per group (90% power). These are much lower than the corresponding statistical power. • If the PoC study had 70 subjects per group (instead of 25) with the same estimated effect and standard deviation, then the prob of success is 69.2% for 128 subjects per group and 75.6% for 172 subjects per group.

39. Observations • If the required number is much higher than the number obtained under the traditional approach, this could be a signal that our prior information on treatment effect is not sufficiently strong. • The above suggests that we need to select appropriate metrics to make decisions. • For proof-of-concept study, the metric might be the precision of the treatment effect estimate. • For a dose-response study, the metric might be the probability for selecting the right dose. • For a confirmatory trial, a better metric might be the probability of a successful trial.

40. Summary • Pharmaceutical industry, as a whole, is facing unprecedented challenges. These challenges also create a lot of opportunities. • A successful statistician needs to be responsive to these challenges. Statisticians need to be able to anticipate these challenges and offer solutions! • It is not the strongest of the species that survive, nor the most intelligent, but the one most responsive to change. - Charles Darwin