Maximizing Validity in Epidemiologic Studies

Module 2:Fundamentals of Epidemiology Issues of Interpretation in Epidemiologic Studies Developed through the APTR Initiative to Enhance Prevention and Population Health Education in collaboration with the Brody School of Medicine at East Carolina University with funding from the Centers for Disease Control and Prevention

Acknowledgments This education module is made possible through the Centers for Disease Control and Prevention (CDC) and the Association for Prevention Teaching and Research (APTR) Cooperative Agreement, No. 5U50CD300860. The module represents the opinions of the author(s) and does not necessarily represent the views of the Centers for Disease Control and Prevention or the Association for Prevention Teaching and Research. APTR wishes to acknowledge the following individual that developed this module: Jeffrey Bethel, PhD Department of Public Health Brody School of Medicine at East Carolina University

Presentation Objectives • Describe the key features of selection and information bias • Identify the ways selection and information bias can be minimized or avoided • Implement the methods for assessing and controlling confounding • Identify uses of the Surgeon General’s Guidelines for establishing causality

Smith, AH. The Epidemiologic Research Sequence. 1984

Exposure or Characteristic Observed Association Disease or Outcome

Exposure or Characteristic Observed Association Is it: biased, confounded, or causal? Disease or Outcome

Bias Any systematic error in the design, conduct, or analysis of a study that results in a mistaken estimate of an exposure’s effect on the risk of disease

Bias Can create spurious association when there really is not one (bias away from the null) Can mask an association when there really is one (bias towards the null) Primarily introduced by the investigators or study participants Selection and information bias

Selection Bias Results from procedures used to study participants that lead to a result different from what would have been obtained from the entire population targeted for study Systematic error made in selecting one or more of the study groups to be compared

Selection Bias Examples Control selection bias Self-selection bias Differential referral, surveillance, or diagnosis bias Loss to follow-up

Selection Bias in a Case-Control StudyControl Selection Bias Question: Do Pap smears prevent cervical cancer? Cases diagnosed at a city hospital. Controls randomly sampled from household in same city by canvassing the neighborhood on foot. Here is the observed relationship: OR = (100 x 150) / (100 x 50) = 1.0 There is no association between Pap smears and risk of cervical cancer (40% of cases and 40% of controls had Pap smears)

Selection Bias in a Case-Control StudyControl Selection Bias Recall: Cases from the hospital and controls come from the neighborhood around the hospital Now for the bias: Only controls who were at home during recruitment for the study were actually included in the study. Women at home were less likely to work and less likely to have regular checkups and Pap smears. Therefore, being included in the study as a control is not independent of the exposure.

Selection Bias in a Case-Control Study Control Selection Bias Question: Do Pap smears prevent cervical cancer? Cases diagnosed at a city hospital. Controls randomly sampled from household in same city by canvassing the neighborhood on foot. Here is the true relationship: OR = (100)(100) / (150)(150) = .44 56% reduced risk of cervical cancer among women who had Pap smears as compared to women who did not (40% of cases had Pap smears versus 60% of controls)

Selection Bias in a Case-Control StudySelf-Selection Bias • Refusal or nonresponse by participants that is related to both exposure and disease • e.g. if exposed cases are more/less likely to participate than participants in other categories • Best way to avoid is to obtain high participation rates

Selection Bias in a Case-Control StudyDifferential Surveillance, Diagnosis or Referral Example related to exposure CC study: venous thromboembolism (VT) and oral contraceptive (OC) use Cases: 20-44 yo, hospitalized for VT Controls: 20-44 yo, hospitalized for acute illness or elective surgery at same hospitals Result: OR = 10.2

Selection Bias in a Case-Control StudyDifferential Surveillance, Diagnosis or Referral Authors acknowledged high OR might be due to “bias in the criteria for hospital admission” Previous studies linked VT to OC Health care provider more likely to hospitalize women with VT symptoms who were taking OC than symptomatic women who were not taking OC

Selection Bias in a Cohort StudyLoss to Follow-up Compared HIV incidence rates among IVDU in NYC from 1992-97 through 10 incidence studies to previous years HIV incidence rates (IR) range from 0 to 2.96 per 100 person-years (py) Well below IR in NYC from late 70s and early 80s (13 per 100 py) to mid 80s and early 90s (4.4 per 100 py) Resulted in funding cuts to drug treatment and prevention programs

Selection Bias in a Cohort StudyLoss to Follow-up • Was decline real? • Follow-up rates in 10 cohorts ranged from 36% to 95% • Only 2 reported >80% • Sample size ranged from 96 to 1,671 Solution: minimize loss to follow-up

Selection Bias – Can We Fix It? • No – need to avoid it when you design and conduct the study • For example • Use the same criteria for selecting cases and controls • Obtaining all relevant participant records • Obtaining high participation rates • Taking in account diagnostic and referral patterns of disease

Information Bias Arises from a systematic difference in the way that exposure or outcome is measured between groups Can bias towards or away from the null Occurs in prospective and retrospective studies Includes recall bias and interviewer bias

Recall Bias in a Case-Control Study Case-control study of birth defects Controls: healthy infants Cases: malformed infants Exposure data collected at postpartum interviews with infants’ mothers Controls or cases may have underreported exposure, depending on nature of exposure

Methods to Minimize Recall Bias Select diseased control group Design structured questionnaire Use self-administered questionnaire Use biological measurements Mask participants to study hypotheses

Interviewer Bias Systematic difference in soliciting, recording, interpreting information Case-control study: exposure information is sought when outcome is known Cohort study: outcome information is sought when exposure is known Solutions:Mask interviewers, use standardized questionnaires or standardized methods of outcome (or exposure) ascertainment

Confounding • A mixing of effects – association between exposure and disease is distorted by the effect of a third variable that is associated with the disease • Alternate explanation for observed association between an exposure and disease X A B

Criteria for Confounding • In order for a factor (X) to be a confounder, all of the following must be TRUE: • Factor X is associated with Disease B (risk factor or preventive factor) • Factor X is associated with Factor A (exposure) • Factor X is not a result of Factor A (not on causal pathway) X A B

Example of Confounding Smoking Coffee Consumption Pancreatic Cancer • Smoking is associated with pancreatic cancer • Smoking is associated with coffee drinking • Smoking is not a result of coffee drinking

Impact of Confounding • Pulls the observed association away from the true association • Positive confounding • Exaggerates the true association • True relative risk (RR) = 1.0 and confounded RR = 2.0 • Negative confounding • Hides the true association • True RR = 2.0 and confounded RR = 1.0

Hypothetical Cohort Study of Obesity and Dementia Relative Risk = = 4.0 (crude measure) 400/1,000 100/1,000

Is age confounding the association between obesity and dementia?

Dementia and Diabetes Cohort StudyCriterion 1: Is Age Associated with Dementia? Relative Risk = = 4.0 400/1,000 100/1,000

Dementia and Diabetes Cohort StudyCriterion 2: Is Age Associated with Obesity? Relative Risk = = 9.0 Odds ratio = = 81.0 900/1000 100/1000 900 x 900 100 x 100

Were Criteria for Confounding Satisfied? • Age is associated with dementia (RR=4.0) • Age is associated with obesity (OR=81.0) • Age is not a result of obesity (not from data) Age Obesity Dementia

Controlling Confounding • Design phase • Group or individual matching on the suspected confounding factor • e.g. matching on age in case-control study • Analysis phase • Stratification • Standardization • Adjustment (multivariate analysis)

Thoughts on Confounding Not an error in the study Valid finding of relationships between factors and disease Failure to take into account confounding IS an error and can bias the results!

Epidemiologic Reasoning • Determine whether a statistical association exists between characteristics or exposures and disease • Study of group characteristics (ecologic studies) • Study of individual characteristics (case-control and cohort studies) • Derive inferences regarding possible causal relationship using pre-determined criteria or guidelines

Causation Association is not equal to causation Consider the following statement: If the rooster crows at the break of dawn, then the rooster caused the sun to rise Causation implies there is a true mechanism from exposure to disease

Koch-Henle Postulates (1880s) • The organism is always found with the disease (regular) • The organism is not found with any other disease (exclusive) • The organism, isolated from one who has the disease, and cultured through several generations, produces the disease (in experimental animals)

Koch-Henle Postulates (1880s) Koch added that “Even when an infectious disease cannot be transmitted to animals, the ‘regular’ and ‘exclusive’ presence of the organism [postulates 1 and 2] proves a causal relationship Unknown at the time of Koch-Henle (1840-1880) • Carrier state • Asymptomatic infection • Multifactorial causation • Biologic spectrum of disease

Understanding Causality • Let’s say you have determined: • There is a real association • You believe it to be causal (ruled out confounding) • NOW have you proven CAUSALITY?

Surgeon General’s Guidelines for Establishing Causality • Temporal relationship • Strength of the association • Dose-response relationship • Replication of the findings • Biologic plausibility • Consideration of alternate explanations • Cessation of exposure • Consistency with other knowledge • Specificity of the association

Temporal Relationship • Exposure to factor must have occurred before disease developed • Easiest to establish in a prospective cohort study • Length of interval between exposure and disease very important • e.g. asbestos and lung cancer • Lung cancer followed exposure by 3 or 20 years?

Strength of the Association The stronger the association, the more likely the exposure is causing the disease Example: RR of lung cancer in smokers vs. non-smokers = 9; RR of lung cancer in heavy vs. non-smokers = 20

Strength of the Association Which odds ratio (OR) would you be more likely to infer causation from? OR#1: OR = 1.4 95% CI = (1.2 - 1.7) OR#2: OR = 9.8 95% CI = (1.8 - 12.3) OR#3: OR = 6.6 95% CI = (5.9 - 8.1)

Dose-Response Relationship Persons who have increasingly higher exposure levels have increasingly higher risks of disease Example: Lung cancer death rates rise with the number of cigarettes smoked

Age-adjusted mortality rates of bronchogenic carcinoma by current amount of smoking

Replication of Findings • The association is observed repeatedly in different persons, places, times, and circumstances • Replicating the association in different samples, with different study designs, and different investigators gives evidence of causation • Example: Smoking has been associated with lung cancer in dozens of retrospective and prospective studies

Biologic Plausibility • Biological or social model exists to explain the association • Does not conflict with current knowledge of natural history and biology of disease • Example: Cigarettes contain many carcinogenic substances • Many epidemiologic studies have identified causal relationships before biological mechanisms were identified

Consideration of Alternate Explanations • Did the investigators consider bias and confounding? • Investigators must consider other possible explanations • Example: Did the investigators consider the associations between smoking, coffee consumption and pancreatic cancer?

Maximizing Validity in Epidemiologic Studies