Alan S. Go, M.D. Division of Research, Kaiser Permanente Northern California

1. People and Measurements—The Nuts & Bolts of ResearchOptimizing Subjects & Variables and Introduction to Kaiser Permanente Northern California Division of Research Alan S. Go, M.D. Division of Research, Kaiser Permanente Northern California Depts. of Epidemiology, Biostatistics, and Medicine, UCSF August 7, 2012

2. Today’s Objectives • Brief Introduction to Research in Kaiser Permanente of Northern California • Gain a better understanding of the Kaiser Permanente Division of Research, population, and databases • Selecting the People • Develop systematic approach to optimize subject selection • Choosing the Measurements • Understand the implications of exposure & outcome variable/measurement choices • Application to a Real Research Question: The ATRIA Study

3. Kaiser Permanente Division of Research and Population Characteristics • Regional division focused on epidemiology, health services and clinical trial research • Kaiser Northern California population: • >3.2 million members (>2 million adults); ~53% women *2010 Kaiser Permanente Members Health Survey (N. Gordon, personal communication)

4. Kaiser Permanente Electronic Databases • Unique lifetime medical record number • Demographic, membership, insurance characteristics • Physician, clinic and medical center characteristics • Inpatient diagnoses/procedures • Ambulatory diagnoses/procedures • Outpatient pharmacy prescriptions • Inpatient and outpatient laboratory tests • Pathology findings • Selected ECG and imaging findings • Examples of Kaiser Permanente disease registries: • CKD/ESRD/AKI, Heart Failure, CHD, Diabetes, GDM, Cancer, HIV/AIDS, IBD, etc. • At end of 2008, regional EMR based on EpiCare

5. Subjects and Variables: The Nuts and Bolts of the Research Question • After deciding a great research question, figuring out WHO you want to study and WHAT you want to measure are the next key steps…

6. Selecting Your Subjects…

7. Optimizing Subject Selection: A Delicate Balancing Act Feasibility Accessibility Cost Time/Efficiency Generalizability Accuracy Diversity Adequate Size

8. Subject Selection: The Nitty Gritty • Explicitly Define Inclusion Criteria • Demographic features (age, gender, race/ethnicity) • Clinical criteria • Geographic/administrative characteristics • Sampling time frame • Explicitly Define Exclusion Criteria • Minimum number needed to be feasible with acceptable generalizability to target population

9. Subject Sampling Techniques:How to Get the “People?” (1) • Convenience Samples • True convenience (e.g., 25 clinic patients I know well) • Consecutive (e.g., next 100 patients undergoing PCI) • Probability Samples • Simple random (e.g., using random number table) • Stratified or weighted random (e.g., by gender) • Cluster (e.g., by clinic, neighborhood, health system, state)

10. Subject Recruitment:How to Get the “People?” (2) • Successful Recruitment Generally Means… •  response, generalizable, adequate size, on time • For database only studies • For hands-on studies (e.g., surveys, cohorts, trials) • Generally harder than you think! • Reasonable inclusion/exclusion criteria • Acceptable subject burden/potential benefits • Minimize subject non-response

11. Applying These Principles to Answer My Research Question: What is the association between use of the blood thinner, warfarin, and the risk of ischemic stroke & bleeding in patients with atrial fibrillation treated in a usual clinical care setting?

12. Warfarin for Stroke Prevention in AF • Atrial fibrillation (AF) is most common clinically significant arrhythmia1 and ↑ stroke risk ~5-fold2,3 • RCTs in selected nonvalvular AF (NVAF) patients showed warfarin  stroke by 68% but  bleeding3 • Aspirin much less effective (RRR ~20%) • Warfarin recommended for many NVAF patients, but concerns about whether trial results can be applied to the “real world” 1 Go AS et al. JAMA. 2001;285:2370-75. 2 Wolf PA et al. Stroke 1991;22:983-88. 3 Atrial Fibrillation Investigators. Arch Intern Med 1994;154:1449-57

13. AnTicoagulation and Risk Factors In Atrial Fibrillation The ATRIA Study

14. ATRIA Study Atrial Fibrillation Warfarin  TE/Bleeds

15. ATRIA Study: Subjects Ambulatory adults with diagnosed nonvalvular AF in Kaiser Permanente All adults with nonvalvular AF in U.S.

16. ATRIA Study: Inclusion Criteria • Sampling Frame Goal: Identify all ambulatory adults with diagnosed chronic nonvalvular AF • Inclusion criteria: • Demography: >18 years, M/F, all race/ethnicities • Clinical Criteria: Diagnosed AF from outpatient & ECG databases (1 outpatient AF dx + 1 ECG with AF or 2 outpatient AF dx only) • Geography/Administrative: Received care in Kaiser Permanente of Northern California • Time Period: AF diagnosis found in 1996-1997

17. ATRIA Study: Exclusion Criteria • Goal: Assemble a cohort with chronic, nonvalvular AF with comprehensive data on predictors and outcomes • Exclusion criteria • No health plan membership • Transient perioperative atrial fibrillation • Concomitant hyperthyroidism • Diagnosed valvular heart disease • No outpatient care during 12 months after index date • No drug benefit surrounding index date

18. 1 Outpatient AF Dx N=15,570 1 ECG with AF N=13,052 1 outpatient AF dx only Identified by ECG only No membership after AF dx <18 years old Transient AF after cardiac surgery Concomitant hyperthyroidism Valvular heart disease No outpatient care after index date or drug benefit ATRIA Cohort Assembly Suspected AF 13,559 Ambulatory Adults with Diagnosed Chronic Nonvalvular AF* and Known Warfarin Status *Validation studies suggest ≥87% of cohort w/ECG-confirmed AF

19. ATRIA: Baseline Characteristics Mean age ± SD 71 ± 12 yr Women 43 % Prior ischemic stroke 9 % Chronic heart failure 29 % Hypertension 50 % Diabetes mellitus 18 % Prior coronary disease 28 % Older, more women, and greater comorbid burden than RCT samplesgeneralizable to AF patients in typical clinical practice

20. Making the Measurements:Implications for Exposure & OutcomeVariable Choices

21. “The most elegant design of a clinical study will not overcome the damage caused by unreliable or imprecise measurement.” J.L. Fleiss (1986) Fleiss, JL. The design and analysis of clinical experiments. pp. 1-5. 1986. John Wiley and Sons, New York.

22. Confounding Variables* Effect Modifiers* Planning the MeasurementsRelationship of Key Exposures Predictor* Outcome *Often generally categorized as “exposures”

23. Dose Issues Cumulative exposure Exposure rate Time Issues Start of exposure When it ended Exposure distribution Alcohol Use Total # of drinks # Drinks/day Date of first Anchor Steam Date of last margarita Daily vs. binge drinking Additional “Exposure” Considerations

24. Continuous Quantitative intervals with typical ranking Examples: Cholesterol level Number of drinks Day supply of drug Waist size Time Categorical Dichotomous (yes/no) (e.g., death, diabetes) Nominal (no order) (e.g., ethnicity, occupation) Ordinal (ordered rank) (e.g., NYHA HF Class I-IV) General Variable Types

25. Survey/questionnaire Interviews Diaries Direct observation Environmental measurements Databases/registries Medical records Physiologic measures Biomarkers (e.g., DNA, sera) Imaging tests Pathology Typical Data Sources Goal: choose the source that gives data closest to the “gold standard” while being feasible to collect

26. Measurement Goals… • Same result when measured repeatedly—within the same subject, between subjects, and over time maximize PRECISION • Represents what it’s really supposed to be maximize ACCURACY/VALIDITY + high sensitivity & specificity

27. The Measurement Spectrum • After deciding the exposure/outcome of interest, “measurement” includes: • Written instructions for applying measurement method for each variable • Doing the measurement method itself • Organizing collected data for analysis • Implementing quality control throughout

28. Standardize methods Pretest, pretest, pretest Automate instrument Train & evaluate staff Timely editing, coding & correcting of forms Multiple measurements Use or validate against “gold standard” Less obtrusive measures Blinding of outcome collection and adjudication Quality control Improving Precision and Accuracy of Variables & Reducing Bias

29. Applying These Principles to Answer My Research Question: What is the association between use of warfarin and the risk of ischemic stroke & bleeding in patients with atrial fibrillation treated in usual clinical care?

30. ATRIA Study: Measurements Ambulatory adults with diagnosed NVAF in KPNC All adults with NVAF in U.S. - Longitudinal warfarin use - Hospitalized ischemic stroke or other systemic embolism - Hospitalized bleeding event Warfarin  TE/Bleeds

31. Confounding Variables Effect Modifiers Planning ATRIA Measurements -Demographic features -Stroke risk factors -Warfarin contraindications Predictor Outcome (?)

32. Exposure Example: Warfarin • Warfarin use (main predictor) • Baseline “warfarin use”—At least one of the following within 3 months of index AF dx date: • 1 filled Rx for warfarin in pharmacy database • “Coumadin therapy” in outpatient db (ICD-9 V58.61) • >1 outpatient INR measurement in lab database • Longitudinal “warfarin use”—time-dependent exposure based on warfarin Rx and INR tests • Validation study of method for baseline use • Chart review of random sample of “users” & “non-users”: 96% raw agreement (=0.92)

33. Outcome Example: Ischemic Stroke • Ischemic stroke (main outcome) • Identification method: searched databases • Primary discharge ICD-9 codes for possible acute ischemic stroke (e.g., 433.x, 434.x, 436.0) found in hospital discharge and billing claims databases • Validation method: reviewed medical records • Obtain Kaiser/non-Kaiser hospital records • 3-physician review (+/- Neurology consultant) • Unable to blind warfarin status at time of event • “Valid stroke” required documented acute neurological deficit lasting >24 hours not due to other etiology

34. What Did We Find? • In 13,559 adults with atrial fibrillation, longitudinal use of warfarin therapy was associated with… • 49% adjusted decrease in risk of ischemic stroke • Modest absolute increase in risk of intracranial hemorrhage (0.51 vs. 0.33 per 100 person-years) • Net benefit of warfarin greatest among patients at the highest risk for ischemic stroke • RCT results of the efficacy and safety of warfarin for atrial fibrillation translate well into certain settings Go AS, Hylek EM, Chang Y, et al. Anticoagulation therapy for stroke prevention in atrial fibrillation: how well do randomized trials translate into clinical practice? JAMA 2003; 290:2685-92.