1 / 33

Key Issues in Analysis

Key Issues in Analysis. Who gets analyzed? How are they grouped for analysis?. Approaches. “Treatment received” (also referred to as “per protocol” or “as treated”) Analyze only fully eligible and compliant subjects with no missing data, e.g., “valid” and “evaluable” subjects

kenneth-smith
Download Presentation

Key Issues in Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Key Issues in Analysis • Who gets analyzed? • How are they grouped for analysis?

  2. Approaches “Treatment received” (also referred to as “per protocol” or “as treated”) Analyze only fully eligible and compliant subjects with no missing data, e.g., “valid” and “evaluable” subjects “Intention to treat” (also referred to as “full analysis” set and “as randomized”) Analyze all randomized subjects utilizing as much information from each as possible See ICH Guidelines E9 for description of analysis sets

  3. There is a Gray Area • Modified Intention to Treat (MITT) population • Participants included in assigned treatment group regardless of treatment actually received. • Ineligible participants based on measures made before randomization, but delayed, are excluded (e.g., patients with HIV who are in enrolled in an HIV prevention trial or patients without TB who are enrolled in a TB treatment trial). • Safety population • Participants who take the experimental treatment even if assigned control treatment.

  4. Example: Randomized Trial of Prevention of HIV with Acyclovir in Couples Where One Partner is Co-infected with HIV and HSV-2 “The primary analysis was a modified intention-to-treat analysis of linked transmissions of HIV-1; unlinked transmissions, seroconversions that occurred among men when their female partners who were infected with HIV-1 were pregnant and not taking the study drug, and seroconversions that occurred after the death of the HIV-1 infected partner were excluded. The secondary analysis was intention-to-treat” N Engl J Med 2010; 362:427-439.

  5. Key Points • Deviation from intent-to-treat can result in bias from not comparing like with like. • This method of analysis needs to be firmly in mind when designing the study, e.g., realistic estimates of treatment effect that account for non-adherence. • There are practical difficulties in carrying out a strict intention to treat analysis and a strict per protocol analysis.

  6. Arguments for Intention to Treat • Consistent with randomization – get the right significance probability for hypothesis testing. • Addresses the question of practical interest – a comparison of treatment policies. • If the objective is to understand the implication of using a specific intervention in practice, this is the right analysis (e.g., non-adherence is a consequence of using a strategy in practice).

  7. Arguments for Per Protocol (As Treated) Analysis • Better estimate of pure pharmaceutical effect of treatment (i.e., including non-compliers dilutes the treatment difference). • The relevant question is whether the treatment can work when used as intended, e.g., is it effective among patients who can tolerate it? • In a non-inferiority study, this may be a more conservative analysis (less dilution toward no difference) Key Point: Make sure you discuss the question before you start the study.

  8. Arguments for Both • Trials can ask two questions: • “Can Drug A reduce tumor size”? (explanatory) • “Does prescribing Drug A to patients with tumors do more harm than good”? (management) • Can it work versus does it work study designs. Also referred to pragmatic and explanatory approaches by Shwartz and Lellouch (J Chronic Dis, 1967) Sackett and Gent, N Engl J Med 1979. As previously discussed, in non-inferiority studies you should consider both ITT and per protocol analyses.

  9. Pre-Exposure Prophylaxis (PrEP) Trials to Prevent HIV Acquisition • Several trials, results not all consistent, does adherence explain all/part of the difference? • IprEx trial in men who have sex with men (36 vs 64 infections -- 44% reduction in incidence with FTC-TDF; 95% self-reported adherence but about 50% based on drug levels.) • FEM-PrEP trial in heterosexual women (33 vs 35 infections with FTC-TDF; 95% self-reported adherence but about 38% based on drug levels) • Partners trial in discordant couples (13 vs 52 infections – 75% reduction in incidence with FTC-TDF; 97% self-reported adherence and 82% adherence based on blood levels)

  10. Obstacles to Intention to Treat (ITT) • Losses to follow-up • Missing data Important to note that ITT not only requires all randomized participants be included in the analysis, but also requires that all randomized participants be followed and have the outcomes of interest measured no matter adherence to the protocol.

  11. Obstacles to Per Protocol Analysis • Defining adherence to treatment • What is an acceptable level of adherence and how do you measure it? • Do you count events that occurred within 2 days, 7 days, 30 days of treatment discontinuation? • Depends on the study and it may not always be clear where to draw the line.

  12. Treatment Received Advantage: Undiluted treatment effect Disadvantage: Comparison of groups may be biased and it is not predictable in which direction.

  13. Intention-to-Treat Advantage: Comparability of treatment groups; no bias resulting from exclusions. Disadvantage: Possible dilution of treatment effect; loss of power unless sample size was increased to account for it.

  14. Intent-to-Treat May be More Powerful • Not only larger sample size, but… • If the treatment under study has an effect even after discontinuation (e.g., disease progression slowed, lingering pharmacologic effect)

  15. ICH Guidelines – Full Analysis Set • Exclusions may occur for failure to meet major entry criteria, failure to take at least one dose of medication, and for lack of any data after randomization • Exclusion of ineligibles may only occur if: • Criterion measured prior to randomization • Eligibility can be objectively assessed • There equal scrutiny for all patients • All violations of specific type are excluded

  16. ICH Guidelines – Per Protocol SetTypical Criteria • Completion of pre-specified minimum exposure to treatment • Availability of measurements or primary outcomes • Absence of major protocol violations

  17. Examples of “Protocol Deviations” or Realities of Field Trials • Enrollment of ineligible participants • Incorrect diagnosis, e.g., TB • Errors in applying eligibility criteria • Knowing violation of eligibility criteria • Incorrect treatment assigned • Incorrect treatment given • Less than 100% adherence to study treatment • Refusal to take study treatment after randomization • Withdrawal from study treatment during the study • Failure to adhere to instructions for taking study medication • Use of prohibited concomitant treatments • Losses to follow-up

  18. Examples of Eligibility Errorsin AIDS Trials • Liver enzyme tests are mixed up for Patient X and Patient Y; 2 weeks after randomization it is determined study drugs are contraindicated for Patient X • Patients have CD4+ cell counts mixed up and the wrong patient is randomized. • Patient X is randomized and is discovered 4 weeks later to be HIV negative • Qualifying lab measurements made 45 days before randomization instead of within 30 days

  19. Policies for Handling Eligibility Errors • 1st Priority is Prevention • Simple inclusion/exclusion criteria • Eligibility checks before randomization • Regular summary reports to monitor performance • Possible Policies • Don’t enroll until eligibility is verified • Enroll those possibly eligible and withdraw later if ineligible; decision to withdraw is blinded to treatment group and based on pre-randomization measurements • Enroll those possibly eligible and keep them Peto R et al., Br J. Cancer 1976; 34:585-612.

  20. What do you do about eligibility errors? • Document them. • Determine whether it is safe for patients to continue treatment. • If safe, assess whether patient should be allowed to continue treatment. • In most cases, follow the patients like other randomized patients so that an intent to treat analysis can be carried out. • Pre-specify a plan for handling them in the protocol.

  21. Examples of “Adherence” Problemsin AIDS Trials • Patient X reports taking a study medication which is not allowed by the protocol • Patient X dies after randomization but before study drug is picked up from pharmacy • Patient X quits taking study treatment 2 weeks after randomization because she decides he does not want to participate in a placebo controlled study • Patient X quits taking study drug 8 weeks after randomization due to side effects • Patient X stops taking study drug before outcome assessment because their condition is worsening. • Patient X is randomized twice because he did not like the first assignment

  22. Anturane Trial Total 813 74 816 89 0.28 Ineligible patients 38 10 33 4 Eligible patients 775 64 783 85 0.10 Nonanalyzable deaths 20 23 Analyzable deaths 44 62 0.08 Analyzable cardiac deaths 43 62 0.06 Analyzable sudden cardiac 22 37 0.04 Analyzable sudden cardiac deaths in 1st 6 months 6 24 0.003 Anturane Placebo No. of Patients No. of Events No. of Patients No. of Events P-value N Engl J Med 1980; 302:250-256.

  23. Coronary Drug ProjectMortality Results No. patients 1103 2789 No. deaths in 5 years 221 583 Percent dead 20.0 20.9 Clofibrate Placebo p-value = 0.55 JAMA 231:360-81, 1975.

  24. Coronary Drug Project –Adherence to Clofibrate(3 capsules, 3 times per day) Percent Adherence* * (No. of capsules taken/No. that should have been taken) x 100 Averaged over all 4 month visits for 5 years for those alive after 5 years. JAMA 231:360-81, 1975.

  25. Coronary Drug ProjectMortality According to Adherence to Clofibrate <80% 24.6 80%+ 15.0 Overall 20.0 Adherence Percent Dead p=0.0001 NEJM 303:1038-41, 1980.

  26. The Obvious, But Naïve, Solution Percent dead 15.0 20.9 ClofibrateAdherers Placebo p = 0.04

  27. Coronary Drug ProjectAdherence to Clofibrate and Placebo(3 capsules, 3 times per day) Clofibrate Percent Adherence Placebo Percent Adherence JAMA 231:360-81, 1975.

  28. Coronary Drug Project Mortality According to Adherenceto Clofibrate and Placebo <80% 24.6 28.2 80%+ 15.0 15.1 Overall 20.0 20.9 Adherence Clofibrate Placebo p=0.0001 p=0.0000001 NEJM 303:1038-41, 1980.

  29. Deviations from Intent-to-Treat May Not be Obvious • Patients who permanently stop study treatment (data collection should continue for ITT but often does not) • Intent-to-treat for primary but not secondary outcomes • Losses to follow-up (including competing events) • Withdrawal of consent • Missing data - minimize this with design, e.g., event-driven versus visit driven data collection, choice of investigators, and patient consent process

  30. Some Protocols Define Situations When Patients Should No Longer Be Followed: Off Study • Did not start treatment • Ineligible • Unacceptable toxicity • Disease progression • Incarceration • Lost to follow-up • Withdrawal of consent Bad idea if ITT is goal – follow everyone until the end of the study or until some defined follow-up period has been achieved. “Off study” is a confusing and bad term.

  31. Recommendation 3, National Research Council Report on Missing Data “Trial sponsors should continue to collect information on key outcomes on participants who discontinue their protocol specified intervention in the course of the study, except in those cases for which a compelling cost-benefit analysis argues otherwise, and this information should be recorded and used in the analysis.”

  32. Recommendation 1, National Research Council Report on Missing Data • Define what you want to estimate in advance of the study and put it in the protocol. Possible causal estimands: • Difference in outcomes for all randomized participants. • Difference in outcomes for those who can tolerate the treatment. • Difference in outcomes if all participants tolerated and adhered to the treatment. • Difference in the area under the outcome curve during adherence. • Difference in outcomes during adherence to treatment.

  33. Summary / Recommendations • Primary analysis should usually be ITT (need to continue collecting data to do this right) – it addresses a pragmatic policy/management question which is always relevant. • ITT analysis requires excellent trial conduct. • It is appropriate to carry out secondary “per protocol” or “as treated” analyses but these have to be interpreted with caution. • Think about what you want to estimate in advance. • For analyses which are not intent-to-treat it is often difficult/impossible to quantify bias resulting from not comparing like with like • If exclusions after randomization are to be made as part of secondary “per protocol” analyses, they should be specified in the protocol.

More Related