University of Pennsylvania Annual Conference on Statistical Issues in Clinical Trials

1. University of Pennsylvania Annual Conference on Statistical Issues in Clinical Trials Panel Discussion II Jim Neaton University of Minnesota April 13, 2011

2. Buyse and Shepherd • On-site monitoring is expensive and evidence to guide when and how to do it is lacking • In some NIH-sponsored trials, approximate costs per visit by an independent contractor were $10,000 in U.S. and $18,000 internationally. • A sampling approach to audits powered to detect a specific error rate is logical but rarely done. • The truth may lie somewhere in between the CRF and auditor’s report • Cost-benefit of on-site monitoring can be experimentally studied within randomized trials efficiently • Prevention of data alteration and fraud should be focus as it may not be detected even with central monitoring • There are many other examples of regulatory requirements which are increasing the cost of trials • Local versus central data entry is not a big ticket cost item.

3. Clinical Trial Transformation Initiative:Survey of Current Monitoring Practices • A wide variety of practices are employed. • There are variations by type of organization in terms of frequency of monitoring and what source documents are examined. • The rationale for using a specific monitoring approach does not appear to be based on empirical evidence. • “More research is needed to assess the potential impact of the variation of monitoring practices we observed.” 31st Meeting of the Society for Clinical Trials. Abstract P31.

4. Monitoring Study Embedded in Large International Trial Cluster Randomized Trial n = 75 sites; 1,500 patients n = 75 sites; 1,500 patients Central monitoring, QA protocol at site, and on-site monitoring at least annually Central monitoring, QA protocol at site, and on-site monitoring for cause

5. Major Outcomes • Primary composite: • Major eligibility violation • Use of antiretroviral drug not permitted by protocol • Unreported primary endpoint of trial • Unreported serious adverse event • Data alteration/fraud • Secondary outcomes: • Cost • Treatment adherence • Delayed reporting of outcomes • Missed visits and losses to follow-up • Number of data queries • Actual versus projected enrollment

6. MRFIT Data Alteration (Cont Clin Trials 1991)Distribution of Difference (Original-Altered Value) in DBP (mmHg) for Randomized Participants Screened in 2 Month Period with Eligibility Change Due to Mark-outs -5 to -9 6 27.2 -4 4 18.2 -3 3 13.6 -2 5 22.7 -1 1 4.5 0 1 4.5 1-3 0 0.0 4-5 2 9.1 Total 22 100.0 Difference (Original-Altered) No. Percent Average: -3.0 mmHg No. <95 to ≥95 mmHg: 14 No. ≥115 to <115 mmHg: 2

7. Other Areas To Work On • Harmonization of regulations. • EU Clinical Trials Directive • Sponsorship problems with NIH-funded trials • Trial insurance • With EU, declining number of clinical trial applications, phase III and IV trials, and number of trial participants. • Study drug labeling and importation. Clinical Trials 2010;7:705-718 and Concept Paper on Revision of Directive, http://ec.europa.eu/health/files/clinicaltrials/concept_paper_02-2011.pdf

8. EU Guidance on Sponsorship • Sponsor must provide the insurance or indemnity to cover the liability of the investigator and sponsor. • Variable implementation of this requirement by EU Member States. • In most cases, the sponsor is the funder. • However, NIH General Counsel has determined that NIH cannot sponsor trials in EU because of indemnity and insurance requirements.

9. Labeling and Distribution of Study Drug Study drug must be relabeled in appropriate language as clinical trial material. Good Manufacturing Practice (GMP) compliance not consistently assessed. Content of Qualified Person (QP) declarations and batch release certificates are not clearly defined. Preparation of multi-country/language label booklets compliant with each country’s regulations. Geographic variation in pharma use of drug trade names and $ values. Complexities and inconsistency of drug release in different countries. Delays = $$$

10. Wittes • The percentage of trials stopped early for efficacy is low (< 1% between 1990 and 2004 in top-impact journals – 97/10,559) – JAMA 2005; 294:2203-2209. • Magnitude of regression to the “truth” depends on information fraction (Freidlin and Korn, Clinical Trials, 2009). • In some situations, the regression effect may be reduced by the time of publication with a later cutoff date and more events. • Yogi was right -- “Prediction is hard, particularly about the future”.

11. Pooled Analysis of Immediate vs. Deferred AZT 0- 209 0.52 (0.39 - 0.68) 1- 357 0.94 (0.76 - 1.16) 2- 440 1.05 (0.87 - 1.27) 3- 369 1.12 (0.91 - 1.38) 4- 307 0.98 (0.78 - 1.23) 5+ 226 1.10 (0.84 - 1.43) Year ofFollow-up No. AIDS/Death Events Hazard Ratio HIV Trialists’ Collaborative Group Lancet, 1999.

12. Ritonavir and Indinavir TrialsLancet 1998 and N Engl J Med 1997 ≤ 50 274 0.47 67 0.49 51 - 100 30 0.39 101 - 200 – – Ritonavir Indinavir BaselineCD4+ No. AIDS/Death Events Hazard Ratio* No. AIDS/Death Events Hazard Ratio* 0.51 29 Median follow-up 5.9 8.7(months) * PI plus combination nucleoside versus combination nucleoside alone.

13. Historical Comparison of Protease Inhibitorversus Nucleoside Therapy Interval ofFollow-up(months) 0 - 6 167 0.49 6 - 12 141 0.41 12 - 18 134 0.29 18 - 24 93 0.26 24 - 30 80 0.25 30 - 36 47 0.25 Hazard Ratio* No. AIDS/Death Events Cumulative *Adjusted for baseline CD4+

14. Snapinn • Multiple endpoints are a reality and a “Consumer Reports analysis” is desirable – show all major outcomes in a manner to assess benefit/risk. • In a regulatory setting, type 1 error control is also a reality – more consideration should be given to a priori defining a set of key secondary outcomes for which a global test is 1st performed. • Is the solution NI for components of a composite or more clinically relevant definition of composites?