1 / 66

Upcoming Events

Upcoming Events. Thursday, April 25: Group 8 presentation on interim analysis Tuesday, April 30: Groups 1 and 2 HIV early treatment presentations; assignment 3 due Thursday, May 2: Groups 3 and 4 HF presentations Tuesday, May 7: Groups 5 and 6 polypill x aspirin presentations

ifeoma-burton
Download Presentation

Upcoming Events

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Upcoming Events • Thursday, April 25: Group 8 presentation on interim analysis • Tuesday, April 30: Groups 1 and 2 HIV early treatment presentations; assignment 3 due • Thursday, May 2: Groups 3 and 4 HF presentations • Tuesday, May 7: Groups 5 and 6 polypillx aspirin presentations • Thursday, May 9: Groups 7 and 8, influenza vaccine presentations • Wednesday, May 15, 10:30-12:30: Final exam

  2. What Should the DSMB Do? • 19/161 (A) vs 39/165 (B) – p=0.009 (HR=2.1; 95% CI:1.2-3.8) • Setting • A and B are active treatments; no alternatives; findings are for primary endpoint (AIDS or death); 2nd interim analysis ; 6 (A) vs 12 (B) deaths • Phase 2 data (CD4+ count) indicates A is likely superior. • 24% of targeted number of primary events (243) • Safety outcome (CVD morbidity and mortality) for a trial of a chemoprophylaxis for cancer • Safety outcome (CVD morbidity and mortality) for a trial for a novel cancer treatment

  3. ddI/ddC HIV Trial, Good Thing We Did Not Stop ddC Better ddI Better FINAL 9/20/92 157:152 (ddI:ddC) Confidence Intervals: Repeated 95% DSMB #5 8/21/92 130:130 (ddI:ddC) DSMB #4 2/13/92 77:91 (ddI:ddC) DSMB #3 11/08/91 50:66 (ddI:ddC) DSMB #2 8/29/91 19:39 (ddI:ddC) 5.7 3.6 0.80 1.00 1.25 Relative Risk (on a log scale) Fleming TR, Neaton JD, et al, JAIDS, 1995.

  4. What Should the DSMB Do? • 99/206 (Exp) vs 81/190 (Placebo) – p=0.28 (OR=1.12; 95% CI: 0.91-1.38) • Setting • Experimental treatment for shock following MI; 30 day mortality is outcome; 60% of patients enrolled. • Same results but only 20% of patients enrolled. • HIV early treatment trial with 20% “information” available?

  5. Interim Analyses ofClinical Trials A Requirement

  6. Outline • Background and how DSMBs arose and function • Group sequential methods • Examples

  7. Suggested Reading Ellenberg S, Fleming TR, DeMets DL Data Monitoring Committees in Clinical Trials. A Practical Perspective John Wiley & Sons, LTD, 2002

  8. Structure for Cooperative Studies (Greenberg Report) Policy BoardorAdvisory Committee National Advisory Heart Council Initialreviewgroup Institutestaff Executive CommitteeorSteering Committee CoordinatingCenter Participating Units Cont Clinical Trials 9:137-48, 1988.

  9. Monitoring Committee Acronyms • PAB = Policy advisory board • DSMB = Data and Safety Monitoring Board • DMC = Data Monitoring Committee • ESMB = Efficacy and safety monitoring board • OSMB = Observational study monitoring board

  10. Steering Committee Study design Patient recruitment and follow-up Data collection Quality assurance Study reports DMC or DSMB Safety of patients Protection of integrity of study Review of blinded data on safety and efficacy of treatments Review of trial conduct, amendments and external data Responsibilities

  11. Interim AnalysesRationale • Safety is best assured by comparing the rate of adverse events with a control group • Studies should not stop before there is a definitive answer and should not continue longer than necessary to obtain one • Regular assessment of the relevance of the question • Regular assessment of whether the trial will address the question posed There are ethical,scientific and economic reasons for interim monitoring of data for clinical trials.

  12. Reasons for Early Terminationof Clinical Trials • Based on accumulated data from the trial: • Unequivocal evidence of treatment benefit or harm • Unexpected, unacceptable side effects • No emerging trends and no reasonable chance of demonstrating benefit • Based on overall progress of the trial: • Failure to include enough patients at a sufficient rate • Lack of compliance in a large number of patients • Poor follow-up • Poor data quality

  13. Today • All NIH sponsored clinical trials are required to have a data monitoring plan • NIH-sponsored trials with clinical endpoints have a DSMB • Many industry sponsored studies have a DSMB • The FDA has prepared a guidance document (Establishment and Operation of Clinical Trial Data Monitoring Committees) http://www.fda.gov/RegulatoryInformation/Guidances/ucm127069.htm • There is variation in operating procedures for DSMBs

  14. When is an Independent DSMB Needed • Early phase studies • Monitoring usually at local level; independent DMC not usually needed. • Phase III & IV studies with morbidity/mortality outcomes; pivotal phase III trials • Frail populations, e.g., children, elderly • Trial with substantial uncertainty about safety, e.g., gene therapy See FDA Guidance and ICH/E9, section 4.5.

  15. DSMBComposition: Multidisciplinary • Clinical experts in the subject matter area • Biostatisticians with expertise in clinical trials and preferably in the subject matter area • Others depending on the nature of the study, e.g., ethicist, pharmacologist, patient advocate Senior investigators without significant conflicts of interest

  16. Independence of DSMB: • Voting members should not be part of the investigative team or work for the sponsor • There should be a clear “need to know” policy for non-DSMB members, e.g., the statistician preparing interim summaries needs to know and may be an employee of the sponsor or member of the investigative team • Members should state potential conflicts This view is not shared by all. See Meinert CL and discussion, Cont Clin Trials, 1998

  17. Typical DSMB Meeting Format • Open Session • Progress report using open data (no outcome data by treatment group) • Sponsor, e.g., NIH, Executive Committee, Protocol Chairs, DSMB and unblinded statisticians • Closed Session • Outcome data by treatment group (usually coded) • DSMB and unblinded statisticians only • Executive Session (DSMB only) • Debriefing Session • DSMB, Sponsor, Executive Committee, Protocol Chairs, and unblinded statisticians

  18. DSMB Confidentiality • Interim data reviewed by the DSMB must remain confidential • Members must not share interim data with anyone outside DSMB • Leaks can affect • Patient recruitment • Protocol compliance • Outcome assessment • Trial integrity and support

  19. Outline • Background and how DSMBs function • Group sequential methods • Examples

  20. References • Ellenberg SS, Fleming TR, DeMets DL, Data Monitoring Committees in Clinical Trials, Wiley, 2002. • DeMets DL, Furberg CD, Friedman LM. Data Monitoring in Clinical Trials. A Case Studies Approach, Springer, 2006. • Jennison C and Turnbull BW, Group Sequential Methods with Applications to Clinical Trials, Chapman and Hall, 2000. • Reboussin DM et al, Cont Clin Trials, 2000. • http://www.biostat.wisc.edu/landemets

  21. DSMB Decision Making Can Be Complex • Internal consistency • Benefit/Risk • External consistency • Current versus future patients • Clinical and public health impact • Statistical issues – monitoring guidelines

  22. Overall Probability of Achieving a Result with Given Nominal Significance of 0.05 After N Repeated Tests Under Ho 1 .05 2 .083 3 .107 4 .126 5 .142 10 .193 25 .266 No. of Tests (N) Probability Ref: McPherson, NEJM, 1974.

  23. Value of Nominal Significance Level Necessary to Achieve a True Level of 0.05 After N Repeated Tests 1 .05 2 .0296 3 .0221 4 .0183 5 .0159 10 .0107 Significance Level Which Should be Used No. of Tests (N) Ref: McPherson, NEJM, 1974.

  24. Simulated Trial(T. Fleming Example) • Patients enter trial over a 3-year period • 1 year minimum follow-up • 60 on A; 60 on B • Survival distributions are equal • Log-rank test used for analysis • 5 situations • 1 Log-rank test - at 4 years • 2 Log-rank tests - every 2 years • 4 Log-rank tests - every year • 8 Log-rank tests - every 6 months • 16 Log-rank tests - every 3 months • 100 simulations of the study

  25. Results The log-rank p-value was less than 0.05 at • the final test (i.e., at 4 years) in 5 of 100 studies • either the 2- or 4-year test in 10 of 100 studies • at least 1 of 4 yearly tests in 17 of 100 studies • at least 1 of 8 semi-annual tests in 21 of 100 studies • at least 1 of 16 3-month tests in 25 of 100 studies

  26. Early Work • Acceptance sampling • Wald (1947) sequential probability ratio test Manufacturing problems, continuous monitoring of the data, no upper bound on sample size

  27. Critical Values (z) for 2-sided Group Sequential Design with .05 Overall Significance and 7 Looks InterimO-Brien/ Haybittle/ AnalysisPocockFleming Peto 1 2.49 5.46 3.0 2 2.49 3.85 3.0 3 2.49 3.15 3.0 4 2.49 2.73 3.0 5 2.49 2.44 3.0 6 2.49 2.23 3.0 7 2.49 2.06 1.96 (2.00)

  28. Critical Values

  29. Choosing Critical Values

  30. General Approach • Compute sample size as if a single look (fixed sample approach) • Specify number of interim analyses and stopping boundary (usually OBF). • Inflate sample size to preserve assumed power using constants in table (not always done as adjustment is minor). • Compute the standardized statistic Zk at each analysis and compare with critical values corresponding to monitoring boundary chosen. • At the end or upon early termination determine P-values and confidence intervals in the usual manner.

  31. Problems with Initial Approach • Difficult to specify number of analyses in advance • Logistically difficult to organize reviews after equal increments of information. Solutions: Slud and Wei and Lan-DeMets

  32. Flexible Approaches • Slud and Wei (JASA, 1982) – specify exit probabilities for each look (stage) such that they sum to , e.g., the prob of exiting the kth stage is the joint prob of not exiting the 1st k-1 stages and exiting the kth one. • Lan-DeMets (Biometrika, 1983) – specify a use function or type I error spending function, e.g., at time zero,  used = 0 and with full information  used = 0.05 (or nominal level)

  33. (t2 ) (t1 ) Spending Function  (t) Alpha .05 (number of events observed at monitoring) (total number of anticipated events)  (t ) }  .0 t1 t2 1 Information Fraction  spending function plotted over fraction of total information to be obtained in the study, evaluated at two arbitrary points, t1 and t2 in the study t = Cont Clin Trial 2000;21:190-207

  34. Plots of Pocock-type and O’Brien Fleming-type spending functions for a one-sided 0.025 significance level, for four analyses at 25%, 50%, 75% and 100% of the expected information. Spending Functions Pocock OBF

  35. Approximate O’Brien Fleming Boundaries Using Lan-DeMets Spending Function Approach: Overall Significance =0.05 and 4 Looks Interim O-Brien OBF AnalysisFlemingLan-DeMets 1 4.05 4.33 2 2.86 2.96 3 2.34 2.36 4 2.02 2.01

  36. Usual Choices for Information • Planned number of events in event-driven trial with common closing date chosen to achieve event target. • Follow-up time, e.g., percent of participants attending final follow-up visit in trial with fixed follow-up for each participant. • Calendar time, e.g., trial with common calendar closing date (e.g., to ensure some minimum follow-up for each participant) but not event-driven.

  37. Beta-Blocker Heart Attack Trial (BHAT) • Placebo-controlled trial of propranolol in patients with a recent MI • Recruitment began in June 1978; planned termination June 1982; average of 3 years of follow-up and maximum of 4 • Primary endpoint – all-cause mortality • Event target - 629 deaths • Stopped early in October 1981 JAMA 1982; 247:1707-1714.

  38. Example 1: Beta-Blocker Heart Attack Trial Early Termination for Efficacy PropranololPlacebo Deaths 138/1916 188/1921 7.2% 9.8% Planned termination: June 1982 Early termination: October 1981 JAMA 1982

  39. Interim Monitoring of BHAT Study 1 May 1979 11 (.23) 56 (.09) 1.68 2 Oct 1979 16 (.33) 77 (.12) 2.24 3 Mar 1980 21 (.44) 126 (.20) 2.37 4 Oct 1980 28 (.58) 177 (.28) 2.30 5 Apr 1981 34 (.71) 247 (.39) 2.34 6 Oct 1981 40 (.83) 318 (.51) 2.82 LookNumber MonitoringDate Months Since Start CumulativeDeaths Logrank Statistic

  40. Critical Values (z) for 2-sided Group Sequential Design with .05 Overall Significance and 7 Looks(BHAT) Interim Lan-DeMets (OBF) AnalysisOBFEventsCalendar 1 5.46 8.00 4.53 2 3.85 8.00 3.73 3 3.15 4.86 3.20 4 2.73 4.08 2.75 5 2.44 3.41 2.47 6 2.23 2.95 2.28 7 2.06 1.97 2.05 Observed LogrankZ=2.82

  41. 1.96

  42. Flexible Number of Looks • Another advantage of the Lan-DeMets spending function approach is the flexibility with the number of looks. • Suppose BHAT was not stopped and there were 3 more looks before the end (10 total). • Looks 7-10 correspond to information fractions considering the number of events of 0.65, 0.75, 0.85 and 1.0. • Stopping boundaries can be calculated conditioned upon the previous tests

  43. Critical Values (z) for 2-sided Group Sequential Design with .05 Overall Significance and 7 Looks(BHAT) Interim Lan-DeMets (OBF) Analysis7 Looks10 Looks 18.00 8.00 28.00 8.00 34.86 4.86 44.08 4.08 53.41 3.41 62.95 2.95 71.97 2.58 8 2.41 9 2.26 10 2.06

  44. Suppose We Get To the 6th Analysis by A Different Route • Information fractions are .05, .20, .30, .40, .45 • Instead of .09, .12, .20, .28, and .39

  45. Critical Values (z) for 2-sided Group Sequential Design with .05 Overall Significance and 7 Looks(BHAT) Interim Lan-DeMets (OBF) Analysis7 Looks7 Looks 18.00 8.00 28.00 4.89 34.86 3.93 44.08 3.33 53.41 3.19 6 2.95 2.98

More Related