Epidemiology Module 3: Systematic and Random Error (Biases and Statistical Precision)

1. Epidemiology Module 3:Systematic and Random Error (Biases and Statistical Precision) Tuhina Neogi, MD, PhD, FRCPC Steven Vlad, MD Clinical Epidemiology Research and Training Unit Boston University School of Medicine

2. Goals • Understand the difference between systematic error and random error • Review types of systematic error (bias) • confounding, information bias (measurement error), selection bias • Review random error (statistical precision) • correct interpretations of confidence intervals and p-values • Type 1 and Type 2 errors

3. Bias

4. How to interpret results of a study • The result of a study (the stated effect measure) can arise because: • It is the truth • It is due to bias (systematic error) • It is due to chance (random error)

5. Bias/Systematic Error • Once we know the study result, we need to focus on whether the result is the truth, or whether it could have been the result of bias • If a study’s result is RR=1.2, could the true (unbiased) result be higher or lower than that value? • How could possible biases influence the reported result?

6. What are biases (systematic errors)? • Biases can either move the observed result (as opposed to the true result) away from the null, or toward toward the null • Null = 0 for difference measures (e.g. risk difference) • Null = 1 for ratio measures (e.g. risk ratio) • Away = • if true result > null, effect appears larger • if true result < null, effect appears smaller • Toward = • if true result > null, effect appears smaller • if true result < null, effect appears larger • Bias has nothing to do with sample size • effect of study design

7. Bias Away from Null 1 Observed Value A Observed Value B 0 + ∞ - ∞ 0 + ∞ True Value A True Value B Null 1 0 + ∞ - ∞ 0 + ∞ Observed Value A Observed Value B Bias Toward Null

8. Direction of Bias • Often, the result of bias is unpredictable • Either toward or away from null • In some circumstances we can predict the effect a bias • Usually toward the null 8

9. Types of Bias • Only 3 types of bias! • all studies • regardless of study design • The different study designs have ways of dealing with these biases – whether it’s done effectively determines how valid the study is

10. The 3 different types of biases • Confounding • Information Bias (Measurement Error) • Selection Bias

11. Confounding

12. Confounding • Question: • Are there other factors that may account for the apparent relationship between exposure (intervention) and disease (outcome)? • or • Is there a common cause of both exposure and disease?

13. Two ways to look at Confounding Confounder Observed Value 2 Observed Value 2 Exposure Disease Confounder Exposure Disease

14. Confounding • Note • Confounder not caused by exposure or disease • Confounder not on causal path from exposure to disease • i.e. not an intermediate Confounder Exposure Disease

15. Examples Smoking Yellow Finger Lung Cancer Diabetes CRP CAD Depression SSRI Suicide

16. Why Does Confounding Occur? • As an imbalance in proportions of the confounder between the two comparison groups

20. Why Does Confounding Occur? Confounder Exposure Disease

21. Why Does Confounding Occur? Smoking Yellow Fingers Lung Cancer Crude RR = (4,600/1M) / (1,400/1M) = 3.3

22. Confounding cont’d • Does hyperlipidemia cause MIs? • Do higher lipid levels cause MIs? BP, age, gender, BMI, smoking, DM ? High LipidsMI

23. Does lowering lipid levels lower the risk of MI? • Does statin use lower lipids? • If so, does that have an effect on lowering MI events? BP, age, gender, BMI, smoking, DM, other healthy lifestyle factors, adherence ? statinslower lipid levels, MI

24. Does a diet high in cholesterol increase the risk of MI? • Does a ‘heart healthy diet’ lower lipids? • If so, does that have an effect on lowering MI events? BP, age, gender, BMI, smoking, DM, other healthy lifestyle factors, diet, exercise, adherence ? Healthy heart dietlower lipid levels, MI

25. Confounding by Indication • Particularly common and difficult to deal with in (observational) pharmacoepidemiology studies RA disease severity Lymphoma TNF-antagonist use Depression SSRI Suicide

26. What Confounding is NOT • Confounding IS NOT • A factor on the causal pathway (intermediate) • high fat diet  LDL  CAD • smoking adenomatous polyp  colon CA • A factor that modifies the relationship between an exposure and a disease • Effect of anti-HTN drug is different in Blacks vs Whites • Effect of blood levels of X on risk of Y in men vs women

27. Effect Modification(aka Interaction)

28. How does a RCT addressConfounding? • Randomization • evenly distribute both known and unknown confounders (by chance) between the two (or more) exposure groups (i.e. active treatment and placebo) • Can use specific inclusion/exclusion criteria to ensure everyone is the same for a particular confounder (e.g., all males in the study) [also known as restriction]

29. Control of Confounding by Randomization • Since potential confounders are balanced between exposure groups, they can’t confound the association Depression Suicide SSRI 1,000,000 get SSRI 1,000,000 get placebo Depression distributed equally by chance, so: Crude RR = (350/1M) / (350/1) = 1

30. Control of Confounding in an RCT • Check Table 1 for imbalances between treatment arms • Are any of the differences clinically meaningful? (not the same as statistically significant!) • How could those differences affect the results? • What other potential confounders are missing from Table 1? • Could they affect the results if they were imbalanced between the treatment arms? • How likely is it that unknown confounders are imbalanced? (with large RCTs, unlikely)

31. Control of Confounding in a RCT • Intention-to-treat analysis • Maintains the balance of potential confounders given by randomization, thereby continuing to (theoretically) address confounding • May need to ‘control’ for very unbalanced factors in the analyses

32. Control of Confounding in Observational Studies • Study design level: Restriction • Inclusion/exclusion criteria to limit study population to a more homogeneous group • E.g. study of alcohol and MI may exclude smokers since smoking is an important confounder • Data analysis level: Stratification • Analyze data stratified by an important confounder • E.g. evaluate effect of alcohol on MI among smokers and among non-smokers separately • Added advantage: identifies effect modification • Data analysis level: Regression • ‘Control’ for potential confounders in regression models • can also identify effect modification if planned for by investigators • Matching • Match on potential confounding variables

33. Control of Confounding in Observational Studies • Have the investigators identified all the important potential confounders in the study? • What factors could be common causes of both exposure and disease? • Have the investigators accounted for these potential confounders either in the design of the study or in the analysis? • If they haven’t, how might that affect the results they found? 33

34. Confounding: Examples

35. In large RCTs, confounding is not usually an important issue • Randomization distributes confounders equally between trial arms • Table 1 appears to confirm this • Probable confounders seem to be well balanced 35

36. Is this difference important? Previous fractures are a risk for future fractures ...

37. Questions to ask: • What are the potential sources of confounding? • Have the authors identified these potential confounders • How have the authors addressed potential confounders? • Was this sufficient?

38. 1. What are the potential sources of confounding? • Risk factors for exposure • i.e. for NSAID use • RA? (chronic NSAIDs) • CAD prevention? (ASA) • prior GI bleeding? (avoid NSAIDs, use coxib) • age? (avoid NSAIDs) confounder NSAID or type of NSAID CV event • Are any of these also related to the likelihood of having an event? 38

39. 1. What are the potential sources of confounding? • Risk factors for outcome? • i.e. having a CV event • smoking? • prior CV event? • general health? • ASA use? (preventative) • age? confounder NSAID or type of NSAID CV event • Are any of these also related to the likelihood of being exposed to an NSAID or type of NSAID? 39

40. 1. What are the potential sources of confounding? • Confounding by indication • Almost always a concern in observational studies of drugs • Here, could the reason that NSAIDs are prescribed (the ‘indication’) be related to both exposure (NSAID prescription) and disease (CV outcomes)? • What about reasons to avoid NSAIDs? • Chronic kidney disease? • Other health problems? Poor health CKD pain NSAID or type of NSAID NSAID or type of NSAID CV event CV event NSAID or type of NSAID CV event

41. 2. Have the authors identified these potential confounders? • In this study, seems like yes • they even have a section talking about ‘covariates’

42. 3. How have the authors addressed potential confounders? • Statistical analysis, • ‘Advanced methods’

43. 4. Was this sufficient? • Often the hardest of these questions to answer, especially if the authors have done a good job addressing the first three questions • Requires experience, judgement, maybe further analysis • My take on this study: probably sufficient • They’ve acknowledged the difficulties and done something about them; • What they’ve done is appropriate; • The methods are reasonable and go beyond what most studies do; • However, I recognize what an insidious problem confounding, especially confounding by indication, is, and therefore I’m ready to change my opinion if further evidence comes to light 43

44. Questions to ask: • What are the potential sources of confounding? • Have the authors identified these potential confounders? • How have the authors addressed potential confounders? • Was this sufficient?

45. 1. What are the potential sources of confounding? • This is another pharmacoepidemiologic study (kind of), so let’s jump right to confounding by indication again • Here, could the reason that DMARDs were prescribed (the ‘indication’) be related to both exposure (DMARD use) and disease (lymphoma)? • -> Disease severity, i.e. chronic inflammation disease severity DMARD or type of DMARD lymphoma

46. 2. Have the authors identified these potential confounders? • Clearly - that is what this paper is all about

47. 3. How have the authors addressed potential confounders? • Here: • detailed assessment of disease severity for each subject • mutual control of drug use and disease severity etc.

48. 4. Was this sufficient? • Again, this is often the hardest of these questions to answer, • My take on this study: probably sufficient • They’ve acknowledged the difficulties and done something about them; • What they’ve done is appropriate; • The methods are reasonable; • Again though, I recognize what an insidious problem confounding, especially confounding by indication, is, and therefore I’m ready to change my opinion if further evidence comes to light 48

49. Information Bias(Misclassification)

50. Information Bias • Could exposure (intervention) or disease (outcome) be inaccurately reported or measured • i.e. misclassified?