Follow-up and compliance

1. Follow-up and compliance • Compliance/adherence • Importance • How to measure and maximize • Follow-up • Importance of complete follow-up • ITT and other issues in analysis

2. Follow-up in RCT’s • What happens after randomization • Carefully lay out procedures to be followed • Describe on forms and in Operations Manual • First reaction: do everything on everyone at every visit • e.g. labs at all visits • But great opportunities for efficiencies • Ask the following: • Do only at some visits? • Do only on a subset? • Don’t do at all

3. Large and Simple Trials • Get a whole lot of people • Randomize, do as few follow-up measurements as possible • Difficult to carry out in practice • Examples • Physicians’ Health study: Randomize to aspirin or placebo, mail out drugs, follow-up by mail • Use data collected for other purposes for follow-up/endpoints • Population mortality • Medical systems (Medicare, Kaiser in U.S.; Gov’t. health in Europe)

4. Large and Simple Trials: Vitamin D • 2700 men and women from general medical practice • Oral vitamin D (100,000 units) given 4 x per year (oral) for prevention of fractures in UK • Sent oral D (and placebo) via mail • Ascertained fractures via “post” (and National Health Service data base) • Mortality via National statistics • Found decrease in fracture risk (possibly mortality) Effect of four monthly oral vitamin D3 (cholecalciferol) supplementation on fractures and mortality in men and women living in the community: randomised double blind controlled trial.BMJ. 2003 Mar 1; 326(7387): 469.

5. Large and Simple Trials: Vitamin D • OBJECTIVE: To determine the effect of four monthly vitamin D supplementation on the rate of fractures in men and women aged 65 years and over living in the community. • DESIGN: Randomised double blind controlled trial of 100 000 IU oral vitamin D3 (cholecalciferol) supplementation or matching placebo every four months over five years. • SETTING AND PARTICIPANTS: 2686 people (2037 men and 649 women) aged 65-85 years living in the general community, recruited from the British doctors register and a general practice register in Suffolk. • MAIN OUTCOME MEASURES: Fracture incidence and total mortality by cause. • RESULTS: After five years 268 men and women had incident fractures, of whom 147 had fractures in common osteoporotic sites (hip, wrist or forearm, or vertebrae). Relative risks in the vitamin D group compared with the placebo group were 0.78 (95% confidence interval 0.61 to 0.99, P=0.04) for any first fracture and 0.67 (0.48 to 0.93, P=0.02) for first hip, wrist or forearm, or vertebral fracture. 471 participants died. The relative risk for total mortality in the vitamin D group compared with the placebo group was 0.88 (0.74 to 1.06, P=0.18). Findings were consistent in men and women and in doctors and the general practice population. • CONCLUSION: Four monthly supplementation with 100 000 IU oral vitamin D may prevent fractures without adverse effects in men and women living in the general community.

6. Vitamin D: Summary of 5 Yr. Results VIT D PBO RR (p) (n=1345) (n=1341) Any fracture 8.8% 11.1% 0.8 (.04) Hip, wrist, vert. 4.5% 6.5% 0.67 (.02) Mortality 16.7% 18.4% 0.88 (.18)

7. Large and Simple Trials: Vitamin D • Efficiencies: • No selection criteria (other than age) • No follow-up clinic visits or measurements • No follow-up labs or measurements • Outcomes from self-assessed mail-in • Study very efficient (VERY) and very generalizable • Cost of Vitamin D trial: about $500,000. • Cost of WHI: about $1,000,000,000. Effect of four monthly oral vitamin D3 (cholecalciferol) supplementation on fractures and mortality in men and women living in the community: randomised double blind controlled trial.BMJ. 2003 Mar 1; 326(7387): 469.

8. Compliance or (mpc) adherence • Trial is meaningless unless participants adhere to interventions • Term “drop out” often used but is ambiguous • Two aspects • 1. Adherence to medications/interventions • 2. Adherence to visit schedules/reporting • Lack of adherence leads to: • Bias • Decreased power • Uninterpretable results

9. Potential effect of incomplete visit follow-up on results in clinical trials - Fracture Intervention Trial (alendronate vs. placebo) - X-rays obtained at baseline, 2 years, 3 years - Vertebral fractures defined from changes in radiographs - FU radiographs on 97% of participants @ year 3 Time (yrs)Relative risk (CI) BL to 2 0.34 (66% reduction) BL to 3 0.49 ( 51% reduction)

10. Effect of Incomplete Follow-up: Virtual Experiment • FIT I: Follow-up x-rays on 97% of surviving participants at year 3 • What if follow-up less complete? • Randomly “lose” 50% between year 2 and 3

11. Use of Survival Analysis for X-Rays in FIT I: Virtual Experiment Time (yrs)Relative risk 2 0.34 3 0.49 3 (50% LTFU) 0.37 LTFU = Lost to follow-up

12. Effect of High Rate of Loss to Follow-up on Results • If early results differ from later results, could create bias when comparing one study to another • Even a “random” (therefore unbiased) loss to follow-up can affect results

13. Effect of High Rate of Loss to Follow-up on Results • More generally, people lost to follow-up may be different than those who remain • Could be differential in two treatment groups • Due to treatment (e.g. estrogen) • Advantages of randomization may be lost • Groups may no longer be comparable • Could bias results

14. Measuring adherence • Medication-taking • Just ask! (self report) • Pill counts • Biochemical assays for some drugs • High tech pill bottles • Visit schedule • N missed visits • Visits within schedule • etc.

15. Adherence goals • Ideal: all participants continue to take medication (perfectly) throughout the trial and attend all follow-up visits until the very end • Why might participants stop medication? • Side effects (real or perceived) • Complex regimens • Want to take true active medication • New info on old medication • New competing medication • Want to stop active medication • New info on old medication (e.g, ERT increases BC risk)

16. Some Examples of “Bad Adherence Days” • Women’s Health Initiative • After first year, letter sent to all participants “observed a small increase in cardiovascular disease among ppts on HRT”… • Many stopped medications • PROOF trial (effect of Calcitonin on osteoporosis) • 1994 to 1999 • 1997: Alendronate approved with significant marketing and excellent results

17. Effect of Stopping Medication: Classical interpretation • Placebo’s start active medication==>become more like actives • Actives stop active medication and start “inactive”==>become more like placebo • Two groups become more similar • Treatment effect is underestimated/conservative • Comforting • “Classical interpretation” may not hold: • Example: patients stop study meds to take a medication that is better than active study medication

18. Strategies to enhance compliance • Warm and fuzzy stuff • Participants to feel appreciated • Staff in clinic spend enough time • Sensitive to ppts. scheduling needs • Parties/events with all participants • Ease of logistics/transportation to clinics • Birthday cards • Gifts • Information, Newsletters, other

19. Strategies to enhance compliance II • Most drop outs occur in early study period • FIT (4 years total); 2/3 of drop outs occurred in first year, most of those in first 6 months • Make certain that ppt’s understand study requirements • Run-in period • Trial run of drug/treatment • Typically 2-4 weeks, usually of placebo (not always) • Value controversial

20. Study adherence: follow-up visits • Goal: visits all on time (within window) • Set appointments flexibly • Reminders prior to appt. • Give study calendar • Listen to concerns/problems

21. Example of Compliance Monitoring in a Randomized Trial • Horizon study • IV dosing of zoledronic acid • 1 dose per year • Endpoint: hip and vertebral fractures • 7400 women in 23 countries, 269 sites • Adherence to visit schedule monitored via IVRS • (telephone “punch 1, punch 2”) system • Reports on web

22. Can analysis be restricted to those who adhere? Coronary Drug Project (CDP, NEJM 1980) 5 year mortality Overall Adherence > 80% (2/3)< 80% (1/3) Clofibrate (n=1065) 18% 15% 25%

23. Can analysis be restricted to those who adhere? Coronary Drug Project (CDP, NEJM 1980) 5 year mortality Overall Adherence > 80% (2/3)< 80% (1/3) Clofibrate (n=1065) 18% 15% 25% Placebo (n=2695) 19% 15% 28% Lessons • Unknown/unmeasured confounders associated with compliance • Differ in placebo and active groups • Be wary of post-randomization groupings

24. Adherence to medication is not the same as adherence to visit schedule • “Drop out” is very vague term • Can have perfect visit adherence (come to all visits on time) but-- • Not take a single study med pill • Take only 60% of pills • If miss visits or stop coming to visits, then generally don’t take study medication • Exceptions do occur: Trial of once-yearly infusion treatment. May have perfect medication compliance but poor visit compliance

25. Follow-up visits for those who have stopped study medications? • Practice varies dramatically across studies • Option 1: Stop follow-up as soon as drug stops • Option 2: Continue to collect follow-up info • Advantages of each • ??

26. Follow-up visits for those who have stopped study medications? • Practice varies dramatically across studies • Option 1: Stop follow-up as soon as drug stops • Option 2: Continue to collect follow-up info • Advantages of each • O-2: Biased per previous slides (generally conservative) • O-1: Biased, but cannot predict direction • Choice related to analysis (ITT)

27. Intention to Treat Analysis (ITT) • ITT coined by AB Hill in textbook on Stat (1961) • One of the main Commandments of RCT bible • Original definition “All subjects will be analyzed according to the treatment group they were originally intended by the randomization process” • “generally interpreted as including all patients, regardless of whether they satisfied entry criteria, treatment actually received or withdrawal or deviation from protocol”-- Hollis BMJ article

28. Beware of “we did an ITT analysis” • Generally considered sacred, almost god-like virtue • The term “ITT” used differently in different studies • ITT does NOT always mean that people were followed beyond stopping study medications

29. Two Purposes of ITT • 1. Maintain validity of original randomization • Groups can only differ by chance • Exclusion of some subjects post-randomization (e.g. didn’t take any pills) could create bias • Medical/surgery example in Hollis (table 1) • 2. Makes clinical trial more like real-world situation • Clinicians in real world have been known to deviate from protocols

30. True Meaning of ITT is Ambiguous • Most rigorous interpretation includes the following principles (Hollis, discussion): • All patients included in analysis even if they didn’t start intervention • Patients who were randomized but did not meet inclusion criterion but were randomized should be included in ITT • All patients included regardless of compliance • Ascertainment for primary outcome on all randomized

31. How are ITT Principles Applied in Real Trials? • Hollis (1999) evaluated application of ITT in 249 trials in BMJ, JAMA, Lancet, NEJM • About 50% (119 trials) reported using ITT • Found that most trials with (ITT claimed) violated one or more of the 4 ITT principles

32. Alternatives in Analysis • per protocol or as treated analysis • If all ppts. are followed regardless of adherence to medications, several types of options • Include only those patients who took all study medications and completed all protocol visits • Include all patients but only for the time that they remained on study medications • If obtain complete follow-up on all ppts., can run several different types of analyses and any discrepancies could be informative.

33. Analysis based on post-randomization variables • Per-protocol limits analysis to adherers • Per-protocol is one example of analysis which stratifies based on post-randomization experience • Other examples? • More generally, subgroup analyses by post-rand. factors are biased, sometimes extremely biased--BEWARE

34. Completion of follow-up for all participants regardless of adherence? • ITT is ambiguous: at best-- • All included, according to original randomization • Follow-up completed on all ppts. • Events are included even when they occurred after medications stopped • Most rigorous approach • Generally conservative estimate of treatment effect • Per protocol or as-treated analyses are possible as secondary analyses

35. Problems with ITT/full follow-up approach • ITT/full follow-up not holy grail • Does not estimate full biologic efficacy of drug/intervention • Advising individual patients may depend on efficacy • Utility underestimated • May be anti-conservative for adverse effects • per-protocol may be preferred

36. Alternatives in Analysis • per protocol or as treated analysis • If all ppts. are followed regardless of adherence to medications, several types of options • Include only those patients who took all study medications and completed all protocol visits • Include all patients but only for the time that they remained on study medications • If obtain complete follow-up on all ppts., can run several different types of analyses and any discrepancies could be informative.

37. Summary of ITT:Those Brits • They don’t know how to cook nor run a monarchy but….

38. Summary of ITT:Those Brits • They don’t know how to cook nor run a monarchy but…. • They sure to talk good

39. ITT: Key Messages (Hollis) • ITT gives a pragmatic estimate of benefit of treatment policy rather benefit in patients who receive treatment exactly as planned • Full application of ITT possible only when complete outcome available on all randomized • Many trials that claim ITT varied in handling of missing data, deviations from protocol, etc. • ITT often inadequately described and applied

40. Recommendations for ITT (Hollis) • ITT best regarded as complete trial strategy (design, conduct, analysis) and not simply analysis • Design • Justify in advance any inclusion which if violated merit exclusion from ITT • Conduct • Minimise (sic) missing response on primary outcome • Follow up subjects who withdraw from treatment • Analysis • Investigate potential effects of missing response

41. Recommendations for ITT • Reporting • Specify how ITT used (explicitly describe handling of deviations from randomization and missing responses) • Report deviations and missingness • Discuss potential effects of missing reponse • Base conclusions on ITT analyses

42. Follow-up and Analysis: summary • Best trial: • All participants remain on medication • All participants are followed until end of study • Pre-planned analysis and handling of deviations from protocol • Where possible, minimize subjectivity and adhoc-ness