Observational Study Designs and Studies of Medical Tests

1. Observational Study Designs andStudies of Medical Tests Tom Newman August 17, 2010 Thanks to Michael Kohn

2. Outline • Conceptual overview • Review common observational study designs • Cohort, Double Cohort • Case-Control • Cross-sectional • Studies of Medical Tests • Diagnostic Test Accuracy • Prognostic Test Accuracy • Examples • “Name that study”

3. Caveats • Nomenclature is confusing and used inconsistently • “Cross-sectional” can refer to timing or sampling • “Retrospective” does not always mean retrospective • Getting the name right is helpful, but it is more important to be able to explain what you want to do and have it make sense for your RQ • If you can’t name your study it’s worth making sure it makes sense

4. Key elements of study design • Timing of the study • Timing of variable occurrence and measurement • How the subjects will be sampled

5. Timing of the study • Prospective: investigator enrolls subjects and makes measurements in the present and future • Historical: investigator relates predictor variables that have already been measured to outcomes that have already occurred • Retrospective: can mean historical, but best reserved for case-control studies

6. Prospective studies • Control over subject selection and variable measurements • Have to wait for outcomes to occur • Take longer • More expensive

7. Historical studies • Less control over subject selection and variable measurements • Outcomes have already occurred • Done sooner • Less expensive

8. Timing of measurements • Longitudinal: measurements in subjects made at more than one time • Cross-sectional: predictor and outcome measured at the same time

9. Longitudinal timing of measurements • Predictor variable precedes outcome • Better for causality (reduces likelihood of “effect-cause”) • Measurement of predictor precedes measurement of outcome • No need for blinding of measurement of predictor variable • Needed to measure incidence = new cases/population at risk/time • Risk of getting the disease

10. Cross-sectional timing of measurements • Measurement of predictor and outcome at about the same time • Causality may be more difficult to infer • No loss to follow-up • Can only measure prevalence = existing cases at one point in time/population at risk • Prevalence = incidence x duration • Risk of having the disease • Not as good for causality

11. Example: “Incidence-Prevalence Bias” • In asymptomatic adults, prevalence of coronary calcium is lower in blacks than in whites* • Does this mean blacks get less heart disease? • No, incidence is greater, but duration is shorter** *Doherty TM et al J Am Coll Cardiol. 1999;34:787–794 **Nieto FJ, Blumenthal RS. J Am Coll Cardiol, 2000; 36:308-309

12. Sampling of subjects • By predictor variable • By outcome variable • By other (e.g., demographic) factors that define the population of interest • Sometimes called “cross-sectional” sampling Usually best

13. Study designs • Descriptive • Many studies of medical tests • Hint variables must VARY • If either the predictor or outcome variable does not vary in your study (e.g., because one value is an inclusion criterion) your study is descriptive • Analytical

15. Analytical study designs • Experimental -- Randomized trial • Observational (today’s topic) -- Cohort -- Double Cohort (exposed-unexposed) -- Case-control -- Cross-sectional

16. Observational analytic studies • Causality is important • May be only ethical option for studying risk factors for disease • Often more efficient • Populations may be more representative • More intellectually interesting than RCTs?

17. Note on Figures Following schematics of observational study designs assume: • Predictor = Risk Factor • Outcome = Disease • Both dichotomous

18. Cohort Study

19. Prospective Cohort Study

20. Historical Cohort Study THE PAST

21. Cohort Studies 1) Measure predictor variables on a sample from a population (defined by something other than the variables you are studying). 2) Exclude any subjects who already have the outcome. 3) Follow the subjects over time and attempt to determine outcome on all subjects.

22. Cohort Studies are longitudinal • Can identify individuals lost to follow up • Can estimate the incidence of the outcome in the population (e.g., cases/person-year) • Measure of disease association is the relative risk (RR) or relative hazard (RH)

23. Double Cohort Study

24. Double Cohort (Exposed-Unexposed) Studies • Sample study subjects separately based on predictor variable • Exclude potential subjects in whom outcome has already occurred. • Attempt to determine outcome in all subjects in both samples over time.

25. Double Cohort (Exposed-Unexposed) Studies • Can identify individuals lost to follow up • Can measure incidence in each cohort, but not overall incidence in the population* • Measure of disease association is the relative risk (RR) or relative hazard (RH) *Unless one of the cohorts is a sample of everyone not in the other cohort

26. Cohort Studies: Summary • Timing of the STUDY • Prospective • Historical • Timing of the MEASUREMENTS: • All cohort studies are longitudinal (follow patients over time) • SAMPLING • Cohort study – sample based on other (e.g., demographic) characteristics • Double cohort study -- sample on predictor variable

27. Case-Control Study

28. Case-Control Study 1) Separately sample subjects with the outcome (cases) and without the outcome (controls) 2) Attempt to determine predictor status on all subjects in both outcome groups

29. Case-Control Study • Cannot identify individuals lost to follow up (no such thing as “lost to follow up”, since by definition outcome status is known) • Cannot calculate prevalence (or incidence) of outcome • Measure of disease association is the Odds Ratio (OR) • Try to replicate a nested case control study in which the cases and controls arise from the same cohort.

30. Nested Case-Control Study

31. Cross-Sectional Study

32. Cross-Sectional Study Attempt to determine predictor and outcome status on all patients in a single population (defined by something other than predictor or outcome).

33. Cross-Sectional Study • No loss to follow-up • Can calculate prevalence but not incidence • Measure of disease association is the Relative Prevalence (RP). • Can be prospective or historical

34. Cohort Studies Start with a Cross-Sectional Study Eliminate subjects who already have disease

35. Studies of Medical Tests • Causality often irrelevant. • Not enough to show that test result is associated with disease status or outcome*. • Need to estimate parameters (e.g., sensitivity and specificity) describing test performance. *Although if it isn’t, you can stop.

36. Studies of Diagnostic Test Accuracy for Prevalent Disease Predictor = Test Result Outcome = Disease status as determined by Gold Standard Designs: Case-control (sample separately from disease positive and disease negative groups) Cross-sectional (sample from the whole population of interest) Double-cohort-like sampling (sample separately from test-positive and test-negative groups)

37. Dichotomous Tests Sensitivity = a/(a + c) Specificity = d/(b + d)

38. Studies of Dx Tests Importance of Sampling Scheme If sampling separately from Disease+ and Disease– groups (case-control sampling), cannot calculate prevalence, positive predictive value, or negative predictive value.

39. Dx Test: Case-Control Sampling Sensitivity = a/(a + c) Specificity = d/(b + d)

40. Dx Test: Cross-sectional Sampling PPV = a/(a + b) NPV = d/(c + d) Prevalence = (a + c)/N

41. Studies of Prognostic Tests for Incident Outcomes Predictor = Test Result Development of outcome or time to development of outcome. Design: Cohort study

42. Examples Name that observational study design

43. Babies born at Kaiser with severe neonatal hyperbilirubinemia (Bili  25) were compared with randomly selected “controls” from the same birth cohort. • Outcomes: (blinded) IQ test and neurologic examination at age 5 years. • Results: No difference in IQ or fraction with neurologic disability between the “case” and “control” groups. Newman, T. B., P. Liljestrand, et al. (2006). N Engl J Med354(18): 1889-900.

44. Jaundice and Infant Feeding Study Design? (Be Careful)

45. JIFee Double Cohort (Exposed-Unexposed) Study* The subjects are divided by predictor (Bili 25+), not outcome (neurologic disability). The “cases” are actually the exposed group and the “controls” are actually the unexposed group *Actually a nested triple cohort study, since “cases” and “controls” came from the same birth cohort and we also studied dehydration. See Hulley page 104.

46. HIV Tropism and Rapid Progression* Is HIV CXCR4 (as opposed to CCR5) tropism a predictor of rapid progression in acutely infected HIV patients? Molecular tropism assay is expensive. Have funding to perform a total of 80 assays. UCSF OPTIONS cohort follows patients acutely infected with HIV. Has banked serum from near time of acute infection. * Vivek Jain’s Project

47. Identify the 40 patients with the most rapid progression (Group 1) and randomly select 40 others from the UCSF Options cohort (Group 2). Run the tropism assay on banked serum for these 80 patients and compare results between Group 1 and Group 2. HIV Tropism and Rapid Progression (continued)

48. HIV Tropism and Rapid Progression Design?

49. HIV Tropism and Rapid Progression Nested Case-Control Study

50. RRISK(Reproductive Risk Factors for Incontinence at Kaiser) • Random sample of 2100 women aged 40-69 years old • Interview, self report, diaries to determine whether they have the outcome, urinary incontinence. • Chart abstraction of obstetrical and surgical records to establish predictor status