Alan S. Go, M.D. Division of Research, Kaiser Permanente of Northern California

1. People and Measurements—The Nuts & Bolts of ResearchOptimizing Subjects & Variables and Introduction to Kaiser Division of Research Alan S. Go, M.D. Division of Research, Kaiser Permanente of Northern California Depts. of Epidemiology, Biostatistics, and Medicine, UCSF August 3, 2010

2. Today’s Objectives • Brief Introduction to Research in Kaiser Permanente of Northern California • Gain a better understanding of the Kaiser Permanente Division of Research, population, and databases • Selecting the People • Develop systematic approach to optimize subject selection • Choosing the Measurements • Understand the implications of exposure & outcome variable/measurement choices • Application to a Real Research Question: The ATRIA Study

3. AHA Cardiovascular Outcomes Research Center Fellowship Opportunity http://www.americanheart.org/presenter.jhtml?identifier=9713 • Two-year fellowship sponsored through the AHA Pharmaceutical Roundtable focused on training the next generation of outcomes researchers • Kaiser Permanente-Stanford University AHA CV Outcomes Research Center • Application deadline: January 2011 • Contact: Alan S. Go, MD (alan.s.go@kp.org)

4. Kaiser Permanente Division of Research and Population Characteristics • Regional research division focused on epidemiology, health care effectiveness research • Kaiser Northern California population: • >3.2 million members (>2 million adults); ~52% women *2008 Kaiser Permanente Members Health Survey (N. Gordon, personal communication)

5. Kaiser Permanente Electronic Databases • Unique lifetime medical record number • Demographic, membership, insurance characteristics • Physician, clinic and medical center characteristics • Inpatient diagnoses/procedures • Ambulatory diagnoses/procedures • Outpatient pharmacy prescriptions • Inpatient and outpatient laboratory tests • Pathology findings • Examples of Kaiser disease registries: • Renal Registry, Chronic Heart Failure, Diabetes, GDM, Cancer, HIV/AIDS, IBD, etc. • At end of 2008, regional EMR based on EpiCare

6. Subjects and Variables: The Nuts and Bolts of the Research Question • After deciding a great research question, figuring out WHO you want to study and WHAT you want to measure are the next key steps…

7. Selecting Your Subjects…

8. Optimizing Subject Selection: A Delicate Balancing Act Feasibility Accessibility Cost Time/Efficiency Generalizability Accuracy Diversity Adequate Size

9. Subject Selection: The Nitty Gritty • Explicitly Define Inclusion Criteria • Demographic features (e.g., age, gender, race) • Clinical criteria • Geographic/administrative characteristics • Sampling time frame • Explicitly Define Exclusion Criteria • Minimum number needed to be feasible with acceptable generalizability to target population

10. Subject Sampling Techniques:How to Get the “People?” (1) • Convenience Samples • True convenience (e.g., 25 clinic patients I know well) • Consecutive (e.g., next 100 patients undergoing liposuction) • Probability Samples • Simple random (e.g., using random number table) • Stratified or weighted random (e.g., by gender) • Cluster (e.g., by clinic or neighborhood)

11. Subject Recruitment:How to Get the “People?” (2) • Successful Recruitment Generally Means… •  response, generalizable sample, adequate size, completed on time (or early!) • For database only studies—Not usually a big problem • For hands-on studies (e.g., surveys, cohorts, trials) • Expect that it will be harder than you think! • Use reasonable inclusion/exclusion criteria • Acceptable subject burden/potential benefits • Efforts to minimize subject non-response

12. Applying These Principles to Answer My Research Question: What is the association between use of the blood thinner, warfarin, and the risk of ischemic stroke & bleeding in patients with atrial fibrillation treated in a usual clinical care setting?

13. Warfarin for Stroke Prevention in AF • Atrial fibrillation (AF) is most common clinically significant arrhythmia1 and ↑ stroke risk ~5-fold2,3 • RCTs in selected nonvalvular AF (NVAF) patients showed warfarin  stroke by 68% but  bleeding3 • Aspirin much less effective (RRR ~20%) • Warfarin recommended for most NVAF patients, but concerns about whether trial results can be applied to the “real world” 1 Go AS et al. JAMA. 2001;285:2370-75. 2 Wolf PA et al. Stroke 1991;22:983-88. 3 Atrial Fibrillation Investigators. Arch Intern Med 1994;154:1449-57

14. AnTicoagulation and Risk Factors In Atrial Fibrillation The ATRIA Study

15. ATRIA Study Atrial Fibrillation Warfarin  TE/Bleeds

16. ATRIA Study: Subjects Ambulatory adults with diagnosed nonvalvular AF in Kaiser Permanente All adults with nonvalvular AF in U.S.

17. ATRIA Study: Inclusion Criteria • Sampling Frame Goal: Identify all ambulatory adults with diagnosed chronic nonvalvular AF • Inclusion criteria: • Demography: >18 years, M/F, all race/ethnicities • Clinical Criteria: Diagnosed AF from outpatient & ECG databases (1 outpatient AF dx + 1 ECG with AF or 2 outpatient AF dx only) • Geography/Administrative: Received care in Kaiser Permanente of Northern California • Time Period: AF diagnosis found in 1996-1997

18. ATRIA Study: Exclusion Criteria • Goal: Assemble a cohort with chronic, nonvalvular AF with comprehensive data on predictors and outcomes • Exclusion criteria • No health plan membership • Transient perioperative atrial fibrillation • Concomitant hyperthyroidism • Diagnosed valvular heart disease • No outpatient care during 12 months after index date • No drug benefit surrounding index date

19. 1 Outpatient AF Dx N=15,570 1 ECG with AF N=13,052 1 outpatient AF dx only Identified by ECG only No membership after AF dx <18 years old Transient AF after cardiac surgery Concomitant hyperthyroidism Valvular heart disease No outpatient care after index date or drug benefit ATRIA Cohort Assembly Suspected AF 13,559 Ambulatory Adults with Diagnosed Chronic Nonvalvular AF* and Known Warfarin Status *Validation studies suggest ≥87% of cohort w/ECG-confirmed AF

20. ATRIA: Baseline Characteristics Mean age ± SD 71 ± 12 yr Women 43 % Prior ischemic stroke 9 % Chronic heart failure 29 % Hypertension 50 % Diabetes mellitus 18 % Prior coronary disease 28 % The ATRIA cohort is older, has more women, and greater comorbid burden than RCT populationsgeneralizable to AF patients in typical clinical practice

21. Making the Measurements:Implications for Exposure & OutcomeVariable Choices

22. “The most elegant design of a clinical study will not overcome the damage caused by unreliable or imprecise measurement.” J.L. Fleiss (1986) Fleiss, JL. The design and analysis of clinical experiments. pp. 1-5. 1986. John Wiley and Sons, New York.

23. Confounding Variables* Effect Modifiers* Planning the MeasurementsRelationship of Key Exposures Predictor* Outcome *Often generally categorized as “exposures”

24. Dose Issues Cumulative exposure Exposure rate Time Issues Start of exposure When it ended Exposure distribution Alcohol Use Total # of drinks # Drinks/day Date of first Anchor Steam Date of last margarita Daily vs. binge drinking Additional “Exposure” Considerations

25. Continuous Quantitative intervals with typical ranking Examples: Cholesterol level Number of drinks Day supply of drug Waist size Time Categorical Dichotomous (yes/no) (e.g., death, diabetes) Nominal (no order) (e.g., ethnicity, occupation) Ordinal (ordered rank) (e.g., NYHA HF Class I-IV) General Variable Types

26. Survey/questionnaire Interviews Diaries Direct observation Environmental measurements Databases/registries Medical records Physiologic measures Biomarkers (e.g., DNA, sera) Imaging tests Pathology Typical Data Sources Goal: choose the source that gives data closest to the “gold standard” while being feasible to collect

27. General Measurement Goals… • You get the same result when measured repeatedly—within the same subject, between subjects, and over timemaximize PRECISION • It represents what it’s really supposed to be maximize ACCURACY/VALIDITY + high sensitivity & specificity

28. The Measurement Spectrum • After deciding the exposure/outcome of interest, “measurement” includes: • Written instructions for applying the method for measuring the variable • Doing the measurement method itself • Spelling out collected data for analysis • Implementing quality control procedures throughout (i.e., making sure you get what you meant to get)

29. Standardize methods Pretest, pretest, pretest Automate instrument Train & evaluate staff Timely editing, coding & correcting of forms Multiple measurements Use or validate against “gold standard” Less obtrusive measures Keep outcome collection and adjudication blinded to exposure Institute quality control during data collection, processing, and analysis Improving Precision and Accuracy of Variables & Reducing Bias

30. Applying These Principles to Answer My Research Question: What is the association between use of warfarin and the risk of ischemic stroke & bleeding in patients with atrial fibrillation treated in usual clinical care?

31. ATRIA Study: Measurements Ambulatory adults with diagnosed NVAF in KPNC All adults with NVAF in U.S. - Longitudinal warfarin use - Hospitalized ischemic stroke or other systemic embolism - Hospitalized bleeding event Warfarin  TE/Bleeds

32. Confounding Variables Effect Modifiers Planning ATRIA Measurements -Demographic features -Stroke risk factors -Warfarin contraindications Predictor Outcome (?)

33. Exposure Example: Warfarin • Warfarin use (main predictor) • Baseline “warfarin use”—At least one of the following within 3 months of index AF dx date: • 1 filled Rx for warfarin in pharmacy database • “Coumadin therapy” in outpatient db (ICD-9 V58.61) • >1 outpatient INR measurement in lab database • Longitudinal “warfarin use”—time-dependent exposure based on warfarin Rx and INR tests • Validation study of method for baseline use • Chart review of random sample of “users” & “non-users”: 96% raw agreement (=0.92)

34. Outcome Example: Ischemic Stroke • Ischemic stroke (main outcome) • Identification method: searched databases • Primary discharge ICD-9 codes for possible acute ischemic stroke (e.g., 433.x, 434.x, 436.0) found in hospital discharge and billing claims databases • Validation method: reviewed medical records • Obtain Kaiser/non-Kaiser hospital records • 3-physician review (+/- Neurology consultant) • Unable to blind warfarin status at time of event • “Valid stroke” required documented acute neurological deficit lasting >24 hours not due to other etiology

35. What Did We Find? • In 13,559 adults with atrial fibrillation, longitudinal use of warfarin therapy was associated with… • 49% adjusted decrease in risk of ischemic stroke • Modest absolute increase in risk of intracranial hemorrhage (0.51 vs. 0.33 per 100 person-years) • Net benefit of warfarin greatest among patients at the highest risk for ischemic stroke • RCT results of the efficacy and safety of warfarin for atrial fibrillation translate well into certain settings Go AS, Hylek EM, Chang Y, et al. Anticoagulation therapy for stroke prevention in atrial fibrillation: how well do randomized trials translate into clinical practice? JAMA 2003; 290:2685-92.

36. The End and Good Luck!