Follow-up and Compliance: Outline

1. Follow-up and Compliance: Outline • Follow-up • Contents, frequency • Importance of complete follow-up • Compliance/adherence • Importance • How to measure and maximize • Reported follow-up and compliance • CONSORT guidelines • ITT and other issues in analysis • Definitions • Different flavors

2. Follow-up in RCT’s • What happens after randomization • Carefully lay out procedures to be followed • Describe on forms and in Operations Manual • First reaction: do everything on everyone at every visit • e.g. labs at all visits • But great opportunities for efficiencies and cost savings • Ask the following: • Do only at some visits? • Do only on a subset? • Don’t do at all

3. Follow up “Styles” • Intensity of follow up depends on study needs (and budget) • Maximal • In patient GCRC-type study • Phase I and (sometimes phase II) studies • Lots of measurements done frequently • Minimalist • Few follow up visits and measurements • Advantages and disadvantages of each

4. Large and Simple Trials • Get a whole lot of people • Randomize, do as few follow-up measurements as possible • Can be difficult to carry out in practice • Examples • Physicians’ Health study: Randomize to aspirin or placebo, mail out drugs, follow-up by mail • Use data collected for other purposes for follow-up/endpoints • Population mortality • Medical systems (Medicare, Kaiser in U.S.; Gov’t. health in Europe) • Internet follow-up direct from participants

5. Large and Simple Trials: Vitamin D • 2700 men and women from general medical practice • Oral vitamin D (100,000 units) given 4 x per year (oral) for prevention of fractures in UK • Sent oral D (and placebo) via mail • Ascertained fractures via “post” (and National Health Service data base) • Mortality via National statistics • Found decrease in fracture risk (possibly mortality) Effect of four monthly oral vitamin D3 (cholecalciferol) supplementation on fractures and mortality in men and women living in the community: randomised double blind controlled trial.BMJ. 2003 Mar 1; 326(7387): 469.

6. Vitamin D: Summary of 5 Yr. Results VIT D PBO RR (p) (n=1345) (n=1341) Any fracture 8.8% 11.1% 0.8 (.04) Hip, wrist, vert. 4.5% 6.5% 0.67 (.02) Mortality 16.7% 18.4% 0.88 (.18) CONCLUSION: Four monthly supplementation with 100 000 IU oral vitamin D may prevent fractures without adverse effects in men and women living in the general community BMJ. 2003 Mar 1; 326(7387): 469.

7. Large and Simple Trials • Efficiencies: • No selection criteria (other than age) • No follow-up clinic visits or measurements • No follow-up labs or measurements • Outcomes from self-assessed mail-in • Study very efficient (VERY) and very generalizable • Cost of Vitamin D trial: about $500,000. • Cost of WHI: about $1,000,000,000.

8. Compliance or (mpc) adherence • Trial is meaningless unless participants adhere to interventions • Think in very precise terms about meaning • Term “drop out” often used but is ambiguous • Several aspects • 1. Adherence to medications/interventions • 2. Adherence to visit schedules/reporting • 3. Adherence to protocol (eg. Admitting only eligibles) • Lack of adherence leads to: • Bias in either efficacy or safety • Decreased power • Uninterpretable results

9. Impact of Loss to Follow up • Can lead to bias especially when LTFU varies by treatment • Eg. Treated drop out more than placebo • Other forms of bias may exist in those lost • Older, younger, sicker • Even random loss to follow up could have impact

10. Potential effect of incomplete visit follow-up on results in clinical trials - Fracture Intervention Trial (alendronate vs. placebo) - X-rays obtained at baseline, 2 years, 3 years - Vertebral fractures defined from changes in radiographs - FU radiographs on 97% of participants @ year 3 Time (yrs)Relative risk (CI) BL to 2 0.34 (66% reduction) BL to 3 0.49 ( 51% reduction)

11. Effect of Incomplete Follow-up: Virtual Experiment • FIT I: Follow-up x-rays on 97% of surviving participants at year 3 • What if follow-up less complete? • Randomly “lose” 50% between year 2 and 3

12. Use of Survival Analysis for X-Rays in FIT I: Virtual Experiment Time (yrs)Relative risk 2 0.34 3 0.49 3 (50% LTFU) 0.37 LTFU = Lost to follow-up

13. Effect of High Rate of Loss to Follow-up on Results • If early results differ from later results, could create bias when comparing one study to another • Even a “random” (therefore unbiased) loss to follow-up can affect results • Clearly most losses to follow-up are not unbiased so more reasons for concern

14. Effect of High Rate of Loss to Follow-up on Results • More generally, people lost to follow-up may be different than those who remain • Could be differential in two treatment groups • Due to treatment (e.g. estrogen) • Advantages of randomization may be lost • Groups may no longer be comparable • Could bias results

15. Measuring adherence • Medication-taking • Just ask! (self report) • Pill counts • Biochemical assays for some drugs • High tech pill bottles • Can count number of times bottle actually opened • Visit schedule • N missed visits • Visits within schedule • etc.

16. Hi Tech approach to Medication Adherence

17. Adherence goals • Ideal: all participants continue to take medication (perfectly) throughout the trial and attend all follow-up visits until the very end • Why might participants stop medication? • Side effects (real or perceived) • Complex regimens • Want to take true active medication • New info on old medication • New competing medication • Want to stop active medication • New info on old medication (e.g, ERT increases BC risk)

18. Some Examples of “Bad Adherence Days” • Women’s Health Initiative • After first year, letter sent to all participants “observed a small increase in cardiovascular disease among ppts on HRT”… • Many stopped medications • PROOF trial (effect of Calcitonin on osteoporosis) • 1994 to 1999 • 1997: Alendronate approved with significant marketing and excellent results

19. Effect of Stopping Medication: Classical interpretation • Placebo’s start active medication==>become more like actives • Actives stop active medication and start “inactive”==>become more like placebo • Two groups become more similar • Treatment effect is underestimated/conservative • Comforting • “Classical interpretation” may not hold: • Example: patients stop study meds to take a medication that is better than active study medication

20. Strategies to enhance compliance • Warm and fuzzy stuff • Participants to feel appreciated • Staff in clinic spend enough time • Sensitive to ppts. scheduling needs • Ease of logistics/transportation to clinics • Ex. Van for transportation: encourage group solidarity • Parties/events with all participants • Birthday cards • Gifts (e.g. gift cards for moderate amounts) • Information, Newsletters, other

21. Strategies to enhance compliance II • Most drop outs occur in early study period • FIT (4 years total); 2/3 of drop outs occurred in first year, most of those in first 6 months • Care in recruitment strategies • Make certain that ppt’s understand study requirements • Run-in period • Trial run of drug/treatment • Typically 2-4 weeks, usually of placebo (not always) • Value controversial

22. Adherence to Follow-up visits • Goal: visits all on time (within window) • Set appointments flexibly • Reminders prior to appt. • Give study calendar • Listen to concerns/problems

23. Study Management Methods to Enhance Adherence • Monitor adherence during study • Important to assess as go along • Spot systematic or individual problems (and make corrections) • Can manage via study website

24. Example of Visit Adherence Monitoring • Horizon study • IV dosing of zoledronic acid • 1 dose per year • Endpoint: hip and vertebral fractures • 7400 women in 23 countries, 269 sites • Adherence to visit schedule monitored via IVRS • Interactive Voice Response System (telephone “punch 1, punch 2”) system • Reports on web

25. Adherence to medication is not the same as adherence to visit schedule • “Drop out” is very vague term • Can have perfect visit adherence (come to all visits on time) but-- • Not take a single study med pill • Take only 60% of pills • If miss visits or stop coming to visits, then generally don’t take study medication • Exceptions do occur: Trial of once-yearly infusion treatment. May have perfect medication compliance but poor visit compliance

26. Follow-up visits for those who have stopped study medications? • Practice varies dramatically across studies • Option 1: Stop follow-up as soon as drug stops • Option 2: Continue to collect follow-up info • Advantages of each • ??

27. Follow-up visits for those who have stopped study medications? • Practice varies dramatically across studies • Option 1: Stop follow-up as soon as drug stops • Option 2: Continue to collect follow-up info Saves money Won’t see long term negative effects Have full follow-up on all participants Can do all sorts of subsets and per protocol analyses Increase costs

28. Intention to Treat Analysis (ITT) • ITT coined by AB Hill in textbook on Stat (1961) • One of the main Commandments of RCT bible • Original definition “All subjects will be analyzed according to the treatment group they were originally intended by the randomization process” • “generally interpreted as including all patients, regardless of whether they satisfied entry criteria, treatment was actually received or withdrawal or deviation from protocol”-- Hollis BMJ article

29. True Meaning of ITT is Ambiguous • Most rigorous interpretation includes the following principles (Hollis, discussion): • All patients included in analysis even if they didn’t start intervention • Patients who were randomized but did not meet inclusion criterion should be included in ITT • All patients included regardless of compliance • Ascertainment for primary outcome on all randomized

30. Two Purposes of ITT (from Hollis) • 1. Maintain validity of original randomization • Groups can only differ by chance • Exclusion of some subjects post-randomization (e.g. didn’t take any pills) could create bias • Medical/surgery example in Hollis (table 1) • 2. Makes clinical trial more like real-world situation • Clinicians in real world have been known to deviate from protocols • (Probably a bit naive to compare any trial to the real clinical world)

31. ITT is considered sacred but….

32. Beware of “we did an ITT analysis” • Generally considered sacred, almost god-like virtue • The term “ITT” used differently in different studies • Probably does mean that all randomized (or most) were included in analysis • Examples of differences in meanings: • In practice, ITT does NOT always mean that participants were followed beyond stopping study medications

33. How are ITT Principles Applied in Real Trials? • Hollis (1999) evaluated application of ITT in 249 trials in BMJ, JAMA, Lancet, NEJM • About 50% (119 trials) reported using ITT • Found that most trials with (ITT claimed) violated one or more of the 4 ITT principles

34. Per protocol analyses as alternatives to ITT If all ppts. are followed regardless of adherence to medications, several types of options Various flavors of per protocol: • Include only those patients who took all study medications and completed all visits • Include all patients but only for the time that they remained on study medications If obtain complete follow-up on all ppts., can run several different types of analyses and any discrepancies could be informative.

35. Analysis based on post-randomization variables • Per-protocol limits analysis to adherers • Per-protocol is one example of analysis which stratifies based on post-randomization experience • Other examples? • More generally, subgroup analyses by post-rand. factors are biased, sometimes extremely biased--BEWARE

36. Can analysis be restricted to those who adhere? Coronary Drug Project (CDP, NEJM 1980) 5 year mortality Overall Adherence > 80% (2/3)< 80% (1/3) Clofibrate (n=1065) 18% 15% 25%

37. Can analysis be restricted to those who adhere? Coronary Drug Project (CDP, NEJM 1980) 5 year mortality Overall Adherence > 80% (2/3)< 80% (1/3) Clofibrate (n=1065) 18% 15% 25% Placebo (n=2695) 19% 15% 28% Lessons • Unknown/unmeasured confounders associated with compliance • Differ in placebo and active groups • Be wary of post-randomization groupings

38. Problems with ITT/full follow-up approach • ITT/full follow-up not holy grail • Does not estimate full biologic efficacy of drug/intervention • Advising individual patients may depend on efficacy • Utility underestimated • May be anti-conservative for adverse effects • per-protocol may be preferred

39. CONSORT Guidelines for Reporting Results of Trials • http://www.consort-statement.org/ • “The CONSORT statement is an important research tool that takes an evidence-based approach to improve the quality of reports of randomized trials. The statement is available in several languages and has been endorsed by prominent medical journals such as The Lancet, Annals of Internal Medicine, and the Journal of the American Medical Association” • Includes guidelines for reporting trials and the “Consort Flowchart”

40. CONSORT checklist

41. Example of CONSORT Diagram: Trial of Zoledronic acid vs. placebo Publication revision to be submitted ~ 2/8/07

42. Summary of ITT from Hollis in BMJ: • The Brits: they don’t know how to cook nor run a monarchy but…. • They sure do talk gooder than us Americans

43. ITT: Key Messages (Hollis) • ITT gives a pragmatic estimate of benefit of treatment policy rather benefit in patients who receive treatment exactly as planned • Full application of ITT possible only when complete outcome available on all randomized • Many trials that claim ITT varied in handling of missing data, deviations from protocol, etc. • ITT often inadequately described and applied

44. Recommendations for ITT (Hollis) • ITT best regarded as complete trial strategy (design, conduct, analysis) and not simply analysis • Design • Justify in advance any inclusion which if violated merit exclusion from ITT • Conduct • Minimise (sic) missing response on primary outcome • Follow up subjects who withdraw from treatment • Analysis • Investigate potential effects of missing response

45. Recommendations for ITT • Reporting • Specify how ITT used (explicitly describe handling of deviations from randomization and missing responses) • Report deviations and missingness • Discuss potential effects of missing reponse • Base conclusions on ITT analyses

46. Follow-up and Adherence: summary • Best trial: • All participants remain on medication • All participants are followed until end of study • Pre-planned analysis and handling of deviations from protocol • Design and implemention of study can help move toward that goal • Important to report details about adherence and how it could affect results