From Screening to Ascertainment of the Primary Outcome using EHR, Challenges in the STRIDE Trial

1. From Screening to Ascertainment of the Primary Outcome using EHR, Challenges in the STRIDE Trial Denise Esserman, PhD Department of Biostatistics denise.Esserman@yale.edu April 17, 2019

2. Outline • Introduction of STRIDE (Strategies to Reduce Injuries and Develop Confidence in Edlers1) • Screening and Recruitment • Ascertainment of Primary Outcome • Adjudication • Statistical Challenges • Concluding Remarks

3. STRIDE: Strategies to Reduce Injuries and Develop Confidence in Elders1 • Pragmatic, multisite cluster-randomized, parallel group superiority trial 10 US health care systems; 86 practices (patients nested within practices) • Testing a multifactorial, individually-tailored intervention implemented by nurse Falls Care Managers (FCM) in primary care setting • Used stratified, covariate constrained randomization • Primary outcome: Time to first serious fall-related injury

4. 8.Reliant Medical Group (Worcester, MA) 9.University of Pittsburg Medical Group (Pittsburg, PA) 10.University of Texas Medical Branch at Galveston (Galveston, TX) 1.Essentia Health (Duluth, MN) 2.HealthCare Partners (El Segundo, CA) 3.Johns Hopkins Medicine (Baltimore, MD) 4.Mount Sinai Health System (New York, NY) 5. Mercy Health Network (Des Moines, IA) and University of Iowa (Iowa City, IA) 6. Michigan Medicine, University of Michigan (Ann Arbor, MI) 7.Partners Healthcare (Boston, MA)

5. Inclusion and Exclusion Criteria INCLUSION CRITERIA • Identified as being at increased risk of falls • Answered yes to one of the following: • Have you fallen and hurt yourself in the past year? • Have you fallen two or more times in the past year? • Are you afraid that you might fall because of balance or walking problems? • Age 70 or older. • Able to provide telephone consent or proxy consent with patient assent. EXCLUSION CRITERIA • Patient enrolled in hospice. • Patient resides in nursing home. • Patient not capable of providing informed consent and does not have an available proxy. • Patient does not speak English or Spanish.

6. Screening and Recruitment • Primary recruitment strategy was centralized screening through Yale Recruitment and Assessment Center (RAC) (9 out of 10 sites) • Clinics provided RAC with names and addresses of age-eligible patients at each practice • Updated biannually • Clinic Screening was used by other site • Clinic provided names and addresses of those screened positive • Regularly transmitted to RAC

7. Screening and Recruitment2

8. Primary Outcome: Serious falls injury Initial plan: • Use self-report or EHR or CMS to detect serious fall injury CHALLENGES • Quality of EHR is not consistent across health care systems • Not all individuals have CMS data • ~40% Medicare Advantage • Variability in receipt of the data across sites and from data different sources • Independent sources of data for adjudication

9. EHR data • EHR data dumps serve 2 purposes • Adjudication • Serious Adverse Events (SAEs) • Hospitalizations > 24 hour stay • MEDRA coding (some cannot be coded e.g. some out of system) • Send specifications every time • Includes requirements of data dump • Standard coding • Validation (Manual) • Includes basic format checking, data types, column names • LESSON LEARNED: Online validation

10. Quality of EHR data • Not all EHR systems are the same • 8 EPIC; 2 Cerner • Conversion issues at the sites • Most default to Excel, problematic for data manipulation • Examples • ID’s with leading 0’s • Date fields with wrong formats/ odd numbers • Differing data • Encounter only • May have 20 lines for 1 visit • Claims only • Can only get hospitalization data through claims (ex. UTMB) • Claims & Encounter • May get duplicates, and need to filter (ex. Reliant)

11. Variability in receipt of the data • EHR data dumps • Obtained biannually • Real time as of data lock date • Centers for Medicare and Medicare Services (CMS) • Obtained ~5 months after close of quarter, through data enclave (Virtual Research Data Center) • Medicare provider and review (MedPAR) data • Some Medicare Advantage hospitalizations • ~18 month delay New Plan Primary Outcome: Self report as primary source, only

12. Adjudication Data Elements: Self-report Initiation of Process • Self-Report: verbal report of event • 4-monthly RAC interviews, informed by monthly fall calendars

13. Adjudication Data Elements: Administrative First line approach • Administrative data: used for healthcare operations, not direct patient care • MAIN SOURCE: Clinical Trial Sites (data dumps) • Mostly encounter (tracking; internal) • Some claims data (billing insurance) • CMS • Those with valid identifier

14. Adjudication Data Elements: Medical Record Second line approach • Medical Record Data: used for provision of patient care • Used when • Data not otherwise obtainable from administrative data • Example: notes, discharge summaries, radiographic reports • Administrative data and self-report inconsistent

15. Statistical Challenges Faced • Ascertainment Bias • Initiate events using only self-report • Changed the primary outcome definition • Estimating ICC/Design effect for time-to-event data • No standard methods for survival data • Impact of timing of practices and capturing of events

16. VIF Example: Impact of timing Cutoff based on event date Cutoff based on interview date

17. More Statistical Issues • Monitoring of the trial • Huge lag in adjudication process • Issues with defining and measuring the design effect and impact on trial for time to event outcome • Difficult to make decisions about trial extensions without knowing information fraction • Analysis • Models that take into account multiple levels of clustering, and semi-competing risks • Model convergence • Impact of the randomization (used covariate constrained randomization in the design)

18. Summarization • Challenging: What is easy on paper is not easy in practice • Often required to make substantive changes to protocol to deal with data issues • Not all data are the same or can be obtained in the same time frame. • Serious concerns with ascertainment and the biases that can be introduced

19. Acknowledgements The STRIDE study was primarily funded by the Patient Centered Outcomes Research Institute (PCORI) with additional support from the National Institute on Aging (NIA) at NIH (U01AG048270). Additional support at Yale University was provided by the NIH/National Center for Advanced Translational Sciences Clinical and Translational Science Awards program (UL1TR000142). PI’s: ShalenderBhasin, David Reuben, Thomas Gill Biostatistics and Data Management Core: Peter Peduzzi, James Dziura, Erich Greene, Thomas Travison, Michael Miller, Charles Lu, Peter Charpentier, Katy Araujo

20. References 1. Bhasin S, Gill TM, Reuben DB, et al. Strategies to reduce injuries and develop confidence in elders (STRIDE): a cluster-randomized pragmatic trial of a multifactorial fall injury prevention strategy: design and methods. Journals of Gerontology: Medical Sciences, 2018; 73(8): 1053-1061. 2. Gill TM, McGloin JM, Latham NK, et al. Screening, recruitment, and baseline characteristics for the strategies to reduce injuries and develop confidence in elders (STRIDE) study. Journals of Gerontology: Medical Sciences, 2018; 73(11): 1495-1501.

21. Workflow for the adjudication process