1 / 52

Curtis Meinert Keynote Lecture Society of Clinical Trials, 2019

This keynote lecture discusses the need for randomization, addressing pervasive interest in positive results, maintaining confidentiality of emerging evidence, and evidence-based controls in clinical trials. It also examines the ethical considerations in evaluating medical interventions and the use of replacement endpoints in disease processes.

amim
Download Presentation

Curtis Meinert Keynote Lecture Society of Clinical Trials, 2019

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Curtis Meinert Keynote Lecture Society of Clinical Trials, 2019 Striving to Achieve & Protect the Integrity of Clinical Trials May 20, 2019 Thomas R. Fleming, Ph.D. Professor, Dept. of Biostatistics University of Washington

  2. Design, Conduct & Analysis of Clinical Trials: Some Substantive Issues • The Need for: • Randomization • ‘Feels, Functions, Survives’ Endpoints Robert Temple, FDA • Addressing Pervasive Interest in Positive Results • Maintaining Confidentiality of Emerging Evidence • The Need for Evidence-based Controls • Conclusions

  3. Therapeutics for Ebola Virus Disease (EVD) • The 2014-’15 major outbreak of EVD in West Africa  ‘urgent’ need for more effective approaches, both for prevention of spread and for treatment • In treatment settings, 28-day mortality appeared to be substantially impacted by ‘optimized Standard of Care’ • (e.g., aggressive IV fluid resuscitation, hemodynamic monitoring & support, • point-of-care diagnostic modalities, other aspects of critical care medicine) • Considerable interest in experimental interventions • (e.g., monoclonal antibodies & antiviral drugs) • Debate: • Is randomization ethical in a public health emergency?

  4. Therapeutics for Ebola Virus Disease (EVD) “Randomized trials are the preferable approach, and unless there are compelling reasons not to do so, every effort should be made to implement randomized trial designs.” NAM (2017); WHO R&D Working Group (2018) “PREVAIL II” ZMapp (Monoclonal Antibody) Optimized Standard of Care R Primary Endpoint: 28-day Mortality Sample Size: Beginning in March 2015, 200 participants to be enrolled from the outbreak in Liberia, Guinea, & Sierra Leonne

  5. Guiding Considerations Medical interventions must be evaluated in a manner that is ethically acceptable, efficient, and reliable. Ethical considerations relate to safeguarding the interests of study participants, and to achieving timely and reliable insights about interventions to enhance the health of the public. While we strive to achieve efficiency by reducing financial costs, number of participants, burdens on medical personnel and study duration, these efforts should not lead to diminished reliability; The goal of clinical research is not simply to provide those pursuing safe and effective interventions a “choice,” but rather an “informed choice.” *Fleming TR, Ellenberg SS. Evaluating Interventions for Ebola: The Need for Randomized Trials. Clinical Trials 2016; 13: 16-19

  6. Design, Conduct & Analysis of Clinical Trials: Some Substantive Issues • The Need for: • Randomization • ‘Feels, Functions, Survives’ Endpoints Robert Temple, FDA • Addressing Pervasive Interest in Positive Results • Maintaining Confidentiality of Emerging Evidence • The Need for Evidence-based Controls • Conclusions

  7. Using Replacement Endpoints: Risks due to Multiple Causal Pathways of Disease Process Off-target effects Immuno-oncology agent ORR, PFS (vs. Chemotherapy) Overall Survival Cancer Longer-term Anti-tumor effects

  8. Using Replacement Endpoints: Risks due to Multiple Mechanisms of Action SBP / DBP Torcetrapib HDLCV Morbidity Cholesterol & Mortality CHD LDL Cholesterol Atorvastatin + Torcetapib Atorvastatin alone R

  9. “A Correlate does not A Surrogate Make” *Fleming TR, DeMets DL: Surrogate endpoints in clinical trials: Are we being misled? Annals of Internal Med 1996; 125:605-613. *IOM, 2010. “Evaluation of Biomarkers & Surrogate Endpoints in Chronic Disease:. Washington DC. National Academies Press *Fleming TR, Powers JH: Biomarkers and Surrogate Endpoints in Clinical Trials Statistics in Medicine 2012; 31: 2973-2984

  10. Design, Conduct & Analysis of Clinical Trials: Some Substantive Issues • The Need for: • Randomization • ‘Feels, Functions, Survives’ Endpoints • Addressing Pervasive Interest in Positive Results • Maintaining Confidentiality of Emerging Evidence • The Need for Evidence-based Controls • Conclusions

  11. Interest in “Positive” Results in Clinical Trials • Industry Sponsors ~Company profits, ↑ value of stock options, promotion • Government Sponsors ~Claims of success in advancing health care  Leverage for ↑ in federal funding • Journal Editors (Publication bias) • Academic Investigators / Caregivers ~Increased ability to publish results ↑ professional stature, earlier promotion, ↑ salary ~Desire to offer more therapeutic options to patients ….Result: Wide Spread & Significant Conflicts of Interest

  12. Survival following Surgery for Rectal Carcinoma Princess Margaret Hospital, Toronto (1977) 100 90 80 70 60 50 40 30 20 10 60 65 Pre-operative Irradiation Survival % Control # = no. at risk 1 2 3 4 5 6 7 Years

  13. Survival of Patients with Rectal Carcinoma Exploratory Subgroup: Dukes’ Stage C Disease 100 90 80 70 60 50 40 30 20 10 22 John Oliver Commentary: “P-hacking” (2016) 2-sided p = 0.01 Pre-Operative Irradiation 16 Survival % Control # = no. at risk 1 2 3 4 5 6 7 Years

  14. Medical Research Council (MRC) Confirmatory Trial 100 80 60 40 20 0 No XRT (275) Single fraction (277) Multiple fractions (272) Survival rate, % 0 6 12 18 24 30 36 42 48 54 60 66 Time, mo

  15. MRC Subgroup Analysis: Dukes’ C Cases 100 80 60 40 20 0 No XRT (111) Single fraction (110) Multiple fractions (79) Survival rate, % 0 6 12 18 24 30 36 42 48 54 60 66 Time, mo

  16. Some Insights regarding Exploratory Analyses • P-values are only interpretable when you understand the sampling context from which they were derived • “Random High” bias is real • Exploratory Analyses usually should be viewed to be “Hypothesis Generating” • Confirmatory Trials greatly enhance the reliability of conclusions

  17. Data Driven Hypothesis for the Cancer Risk with Ezetimibe + Simvastatin in Aortic-Valve Stenosis • SEAS TrialNCA. IncidenceCA. Deaths Ezet + Sim 944 101 37 Placebo 929 65 20 Relative Risk: 1.551.78 95% C.I.: (1.13, 2.12) (1.03. 3.11) • IMPROVE-IT & SHARP Trials NCA. IncidenceCA. Deaths Ezet + Sim 10,391 313 97 Control 10,298 326 72 Relative Risk: 0.96 1.34 95% C.I.: (0.82, 1.12)(0.98, 1.84)

  18. Interest in “Positive” Results in Clinical Trials • Protocol Specified ‘Primary Objective’ of the Clinical trial: • Very frequent wording: ~“ To establish that the experimental regimen is safe and effective” • Scientifically unbiased wording:

  19. Interest in “Positive” Results in Clinical Trials • Protocol Specified ‘Primary Objective’ of the Clinical trial: • Very frequent wording: ~“ To establish that the experimental regimen is safe and effective” • Scientifically unbiased wording: ~“ To determine whetherthe experimental regimen is safe and effective” …building a story with supportive analyses…

  20. Bias for “Positive” Results in Clinical Trials ~What is the definition of a successful clinical trial? • A very common response: “A clinical trial that achieves a positive result” • The proper scientific response:

  21. Bias for “Positive” Results in Clinical Trials ~What is the definition of a successful clinical trial? • A very common response: “A clinical trial that achieves a positive result” • The proper scientific response: “A clinical trial that addresses a clinically important issue, & reliably answers the questions it was designed to address”

  22. Some Conclusions • Recognize bias resulting from strong interest to achieve “positive” results • When refereeing journal publications, request: • the clinical trial protocol • the statistical analysis plan (SAP) • the clinical study report (CSR) • P-values presented in CSRs & publications only for α-spending analyses pre-specified in the SAP • Recognize unreliability of Exploratory Analyses… …generating hypotheses, but with “random high” bias

  23. “If you Torture Data, They will Confess” * Fleming TR “Clinical Trials: Discerning Hype from Substance” Annals of Internal Medicine 2010; 153:400-406

  24. Design, Conduct & Analysis of Clinical Trials: Some Substantive Issues • The Need for: • Randomization • ‘Feels, Functions, Survives’ Endpoints • Addressing Pervasive Interest in Positive Results • Maintaining Confidentiality of Emerging Evidence • The Need for Evidence-based Controls • Conclusions

  25. “CPCRA #002” Trial ddC/ddI: Rate of Progression to AIDS/Death 8/29/91 (39/19) 11/7/91 (66/50) 2/13/92 (91/77) 8/21/92 (130/130) ] ) * 2.08 1.25 0.88 ( [ ] ) * 2.44 2.04 1.41 1.00 0.82 ( [ ] ) * 1.75 1.64 1.20 0.89 0.82 ( ) * 1.25 1.00 0.80 2.5 1.7 1.25 1.0 0.8 * Had 39 vs 19 data been released Pre-judgment

  26. “VALUE Trial” Hypertensive Patients at High Cardiovascular Risk Events on Valsartan / Amlodipine ; Relative Risk Outcome May ’98 to May ’98 to Measure August ‘00 December ’03 (n = 15,290) (n = 15,245) Death 178/141; 1.253 841/818; 1.021 M.I. 102/76; 1.332 369/313; 1.171 Stroke 124/92; 1.338 322/281; 1.138 H.F. Hosp 104/112; 0.922 354/400; 0.879 Diabetes No data 690/845; 0.811 * Had May ‘98 data been released Misleading Insights

  27. Maintaining Confidentiality of Emerging Data DAMOCLES*: “The current prevailing view is that the trial investigators should not see the unblinded interim results, and the argument that releasing interim results would aid enthusiasm and accrual is false.” * The United Kingdom NHS Health Technology Assessment Program commissioned the ‘Data Monitoring Committees: Lessons, Ethics, Statistics Study Group’ (DAMOCLES): ─ to investigate existing processes of monitoring accumulating data ─ to identify ways of improving the DMC process. Grant, Altman, Babiker, et al. Health Technology Assessment 2005

  28. Evidence from NIH Cancer Cooperative Group Studies Maintaining Confidentiality  ↓ Pre-judgment ↑ Trial Integrity NIH Cancer Cooperative Group NCCTG SWOG Interim Data shown only to DMCs: YES NO Declining accrual rate 0/10 5/10 Number closed 9/10 9/10 Full accrual 8 6 Term early appropriately 1 1 Term early inappropriately 0 2 Completed studies with current results inconsistent with early 0/9 2/9 published results

  29. Enhancing Trial Integrity by Maintaining Confidentiality Maintaining Confidentiality of Emerging Data from Ongoing Clinical Trials: • Reduces the Risk of Pre-judgment, correspondingly increasing the ability to achieve: • Timely Enrollment • Targeted levels of Adherence & Retention • Timely Trial Completion with Reliable Results • Reduces the Risk for Early Release of Misleading Results • Protects the Flexibility to Modify Trial Design Based on Insights from Emerging External Data

  30. Access to Interim Safety & Efficacy Data Should be only on a “Need to Know Basis” • Access to interim data to seek regulatory approval, in trials continued post-approval to provide definitive evidence about principal efficacy or safety outcomes (eg., ‘accelerated approval’ or CV safety trials) • Access to pooled data on efficacy or safety (eg., by designated statisticians to assess event rate for sample size adjustments to achieve the targeted # of events in a time-to-event analysis) • Access to safety data by medical monitors for timely reporting of SUSARs and SAEs

  31. Access to Interim Data to Seek “Accelerated Approval” Illustration: • Potential Registration Endpoint: e.g: ‘Reasonably likely to predict’ Biomarker • Clinical Endpoint of Principal Interest: e.g: Overall Survival (OS) …for full regulatory approval… Approach to maintain integrity of Overall Survival data: When data on the ‘Registration Endpoint’ are complete, and if the monitoring boundary for OS is not crossed: ─ Release data on the Registration Endpoint ─ Maintain confidentiality of OS data until the boundary is crossed or target # of events is achieved

  32. Access to Interim Safety & Efficacy Data Should be only on a “Need to Know Basis” • Access to interim data to seek regulatory approval, in trials continued post-approval to provide definitive evidence about principal efficacy or safety outcomes (eg., ‘accelerated approval’ or CV safety trials) • Access to pooled data on efficacy or safety (eg., by designated statisticians to assess event rate for sample size adjustments to achieve the targeted # of events in a time-to-event analysis) • Access to safety data by medical monitors for timely reporting of SUSARs and SAEs

  33. Maintaining Confidentiality of Emerging Data “LIGHT Trial” Naltrexone SR/Bupropion SR: “Contrave” CV risks in Overweight/Obese Subjects With CV Risk Factors Key Design Objectives: At 90 events: 2.0 Margin for CVDeath / Str / MI At 378 events: 1.4 Margin for CVDeath / Str / MI …FDA’s Part 15 Open Public Hearing, 8/11/2014…  “Confidentiality of Interim Results in Cardiovascular Outcome Safety Trials”

  34. CVD Overall DeathsD Stroke CV Total Stroke MI Stroke MI Non-CV MI “1st Quadrant”: Up to 11/23/2013 Contrave35 5 5 10 7 24 40 Placebo 59 19 3 22 11 34 62 HR 0.590.64 • DMC rec: ‘Release data to FDA per Data Access Plan’

  35. CVD Overall DeathsD Stroke CV Total Stroke MI Stroke MI Non-CV MI “1st Quadrant”: Up to 11/23/2013 Contrave35 5 5 10 7 24 40 Placebo 59 19 3 22 11 34 62 HR 0.590.64 • DMC rec: ‘Release data to FDA per Data Access Plan’ “2nd Quadrant”: Between 11/23/2013 and 3/3/2015 Contrave55 12 21 33 15 31 74 Placebo 43 15 14 29 10 23 57 HR ≈1.29≈1.30 • On 3/3/2015, DMC recommended trial continuation…

  36. CVD Overall DeathsD Stroke CV Total Stroke MI Stroke MI Non-CV MI “1st Quadrant”: Up to 11/23/2013 Contrave35 5 5 10 7 24 40 Placebo 59 19 3 22 11 34 62 HR 0.590.64 • DMC rec: ‘Release data to FDA per Data Access Plan’ “2nd Quadrant”: Between 11/23/2013 and 3/3/2015 Contrave55 12 21 33 15 31 74 Placebo 43 15 14 29 10 23 57 HR ≈1.29≈1.30 • On 3/3/2015, DMC recommended trial continuation… That day, sponsor released “1st Quadrant” in Patent Filing  Steering Committee recommends trial termination

  37. CVD Overall DeathsD Stroke CV Total Stroke MI Stroke MI Non-CV MI “1st Quadrant”: Up to 11/23/2013 Contrave35 5 5 10 7 24 40 Placebo 59 19 3 22 11 34 62 HR 0.590.64 JAMA 3/8/2016 Final 64%: ‘End of Study’ Results Contrave119 26 39 65 31 69 156 Placebo 124 42 29 71 23 71 151 HR 0.951.02 Key insights: • Potential unreliability of interim data • Breaches in confidentiality provide potential for:  Dissemination of misleading results  Risks to irreversibly bias subsequent trial conduct

  38. Confidentiality of Interim Data ─ DAMOCLES: * Grant, Altman, Babiker, et al. Health Technology Assessment 2005 “There is near unanimity that the interim data and the deliberations of the DMC should be absolutely confidential… …Breaches of confidentiality are to be treated extremely seriously” ─ Formal statements of concordance have been issued by NIH, WHO, EMA and FDA* * Fleming et al. Maintaining confidentiality of interim data to enhance trial integrity and credibility. Clinical Trials 2008; 5: 157–167

  39. Some Scientific & Ethical Considerations ꟷ2nd principle of clinical equipoise: “Anything that jeopardizes the trial’s ability to disturb the initial state of equipoise is to be avoided” ꟷMaintaining Confidentiality: ↓ Risk of Pre-judgment  ↑ Trial Integrity  Q: If an (Ebola) outbreak wanes, before trial results are complete or conclusive, should that evolving evidence remain confidential to allow trial continuation at next outbreak?

  40. “PREVAIL II” Therapeutics for Ebola Virus Disease ZMapp (Monoclonal Antibody) Optimized Standard of Care R N Deaths ZMapp 36 8 Control 35 13 (Intended 100/arm not reached, when outbreak waned in late 2015 in Liberia, Guinea, and Sierra Leonne) NIH October 13, 2016 Press Release “Study finds Ebola treatment ZMapp holds promise, although results not definitive” 2p = 0.18

  41. Therapeutics for Ebola Virus Disease (EVD) Trial Design in ‘18-’19 EVD Outbreak in DRC o SOC o SOC + ZMapp o SOC + Exp’l Treatment R o SOC + ZMapp o SOC + Exp’l Treatment R ‘o SOC’ denotes ‘Optimized Standard-of-Care’ (Aggressive IV fluid resuscitation, hemodynamic monitoring and support, point-of-care diagnostic modalities, and other aspects of critical care medicine)

  42. Design, Conduct & Analysis of Clinical Trials: Some Substantive Issues • The Need for: • Randomization • ‘Feels, Functions, Survives’ Endpoints • Addressing Pervasive Interest in Positive Results • Maintaining Confidentiality of Emerging Evidence • The Need for Evidence-based Controls • Conclusions

  43. Some Scientific & Ethical Considerations Q: If an (Ebola) outbreak wanes, before trial results are complete or conclusive, should that evolving evidence remain confidential to allow trial continuation at next outbreak? A: A new clinical trial paradigm is under discussion by the WHO R&D Working Group to address this specific issue in trials conducted in the setting of public health emergencies.

  44. Choice of a Proper Control Arm: Ethical Considerations Q:If there are local treatment guidances, yet these are not based on definitive evidence, should these be judged to establish limits on the choice of a proper control regimen. E.g: HPTN 052 Trial, in 1780 HIV-discordant partners : Initiating ART at CD4 < 350 cells vs. at CD4 < 200 cells

  45. Choice of a Proper Control Arm: Ethical Considerations Q: If a regimen has wide-spread use in clinical practice, is a placebo-controlled trial no longer ethical? Illustration: Ventricular Arrhythmia after M.I. Arrhythmia ─ Risk factor for Sudden Death • Class IC antiarrhythmic drugs suppress arrhythmias  yearly use in hundreds of thousands of patients

  46. Choice of a Proper Control Arm: Ethical Considerations Q: If a regimen has wide-spread use in clinical practice, is a placebo-controlled trial no longer ethical? Illustration: Ventricular Arrhythmia after M.I. Arrhythmia ─ Risk factor for Sudden Death • Class IC antiarrhythmic drugs suppress arrhythmias  yearly use in hundreds of thousands of patients Cardiac Arrhythmia Suppression Trial: Class IC antiarrhythmic drugs R Placebo

  47. Choice of a Proper Control Arm: Ethical Considerations Q: If a regimen has wide-spread use in clinical practice, is a placebo-controlled trial no longer ethical? Illustration: Ventricular Arrhythmia after M.I. Arrhythmia ─ Risk factor for Sudden Death • Class IC antiarrhythmic drugs suppress arrhythmias  yearly use in hundreds of thousands of patients Cardiac Arrhythmia Suppression Trial: The Class IC antiarrhythmic drugs, relative to Placebo, TRIPLE the death rate

  48. Design, Conduct & Analysis of Clinical Trials: Some Substantive Issues • The Need for: • Randomization • ‘Feels, Functions, Survives’ Endpoints • Addressing Pervasive Interest in Positive Results • Maintaining Confidentiality of Emerging Evidence • The Need for Evidence-based Controls • Conclusions …Revatio…

  49. “FDA Drug Safety Communication: FDA recommends against use of Revatio in children with pulmonary hypertension” “Plot of mortality in the pediatric clinical trial as a function of Revatio dose.” The “The hazard ratio for high dose compared to low dose was 3.5 (p=0.015)” 

  50. Design, Conduct & Analysis of Clinical Trials: Some Substantive Issues • The Need for: • Randomization • ‘Feels, Functions, Survives’ Endpoints …Goal…not a ‘Choice’, but an ‘Informed Choice’… • Addressing Pervasive Interest in Positive Results • Maintaining Confidentiality of Emerging Evidence • The Need for Evidence-based Controls …Distributive Justice… • Conclusions

More Related