Study Design Bias Case study: Leptospirosis in meat workers

1. Study Design & BiasCase study: Leptospirosis in meat workers Anou Dreyfus, Jackie Benschop & Cord Heuer EpiCentre, IVABS, Massey University

2. Agenda Epidemiological studies Study design Bias & Confounding Break Case study: Leptospirosis in meat workers Group work Results & Discussion

3. What determines disease in an individual?

4. Study Objectives Find associations between host factors, agent, environment and disease Test causal hypotheses Predict outcome

5. Exposures and outcomes In an epidemiological study there is: the outcome of interest the primary exposure (or risk factor) of interest other exposures that may influence the outcome (potential confounders)

6. Epidemiological studies Define: Study question(s) Study objective(s) Case definition Hypothesis for testing

7. Epidemiological Studies Study design Study population Sampling Method Study size (statistical power) Data collection Data measurement / observation Data analysis Data presentation Message to scientists and stakeholders

8. Epidemiological studies

9. Epidemiological studies

10. Study design: classification Descriptive: describe rates. Ie mortality in a population. In analytical studies you compare groups and try to find a difference between them, find risk factors for an outcome. Group with more cancer has more smokers than group without cancer. In experimental studies you allocate the exposure. One group gets the treatment, the other does not. More people healed in the group with the drug? Descriptive: describe rates. Ie mortality in a population. In analytical studies you compare groups and try to find a difference between them, find risk factors for an outcome. Group with more cancer has more smokers than group without cancer. In experimental studies you allocate the exposure. One group gets the treatment, the other does not. More people healed in the group with the drug?

11. Study design Observational, analytical studies Cross – sectional Cohort Case – control

12. Prevalence vs Incidence Prevalence (disease or antibodies) The number of people ill in a given population at one point in time Incidence (disease or antibodies) The number of people falling ill in a given population during a given time period

14. Cross-sectional study: descriptive 14

15. The starting point in cohort studies is the definition of a group of disease free people by their exposure status. These are then followed up over time to see which ones develop a Disease or condition.The starting point in cohort studies is the definition of a group of disease free people by their exposure status. These are then followed up over time to see which ones develop a Disease or condition.

16. Cohort study Closest to clinical trial (causal hypothesis) Sampling disease free individuals Exposure based sampling Follow group over time Prospective (‘retrospective’, ‘historical’) Exposed and unexposed both sampled from the ‘same population’

17. Relative Risk or Risk Ratio

18. Cohort Study

19. Cohort study Rare exposures – multiple diseases (outcomes) Measurement of time relationship betw exposure & outcome High costs, high loss to follow up

20. Assumptions cohort study Cohorts equal except for exposure Constant risk

21. Analysis Rate, risk, odds, risk ratio, rate ratio, odds ratio, vaccine efficacy.... Need to account for confounding Stratified, multivariable methods. Survival analysis time to event censoring

22. Summary

23. Different types of error

24. Random error If measured effect of exposure occurs by chance Consequence of: Variability (random variation) Sample size? A measured effect of an exposure may occur just by chance and is called random error. This is usually due to an insufficient sample size and/or random variation. A measured effect of an exposure may occur just by chance and is called random error. This is usually due to an insufficient sample size and/or random variation.

25. Random error To quantify degree to which chance accounts for observed results: Test significance (p-value) Establish precision (confidence intervals) Establish power To see whether random error may account for the observed results, significance testing and establishing precision and power is needed. So if the power of a study is between 80 and 90, the p-value < 0.05 and the confidence intervals narrow and not including 1, there is a good probability that random error was excluded from the study. However, there are more errors that can occur even if the association is statistically significant, these are summarized in the expression ? To see whether random error may account for the observed results, significance testing and establishing precision and power is needed. So if the power of a study is between 80 and 90, the p-value < 0.05 and the confidence intervals narrow and not including 1, there is a good probability that random error was excluded from the study. However, there are more errors that can occur even if the association is statistically significant, these are summarized in the expression ?

26. Systematic error = Bias = Deviation of the truth ? Over- or underestimation of the measure of association, e.g. relative risk Consequence of: Study design problems / limitations Study implementation Systematic error or bias. Bias means deviation of the truth. The reason we are concerned about bias, is because measured health problems may be over- or underestimated. So our study results do not represent reality, they are biased. Bias or systematic error is a consequence of study design problems or how the study was carried out. Systematic error or bias. Bias means deviation of the truth. The reason we are concerned about bias, is because measured health problems may be over- or underestimated. So our study results do not represent reality, they are biased. Bias or systematic error is a consequence of study design problems or how the study was carried out.

27. 02.07.08 Errors in epidemiological studies One slide to show you the influence of study size on random and systematic error. The larger the study size the lower the chance of random error. Study size has no influence on bias!!! Let me quickly revise the different forms of Bias One slide to show you the influence of study size on random and systematic error. The larger the study size the lower the chance of random error. Study size has no influence on bias!!! Let me quickly revise the different forms of Bias

28. Systematic error = Bias Deal with at: Design / implementation stage Analysis stage Some bias can be dealt with at the design stage others at the analysis stage (Confounding).. ? Some bias can be dealt with at the design stage others at the analysis stage (Confounding).. ?

29. Different types of error

30. Selection bias Defined as: Error that arises in the process of identifying the study populations Selection bias can be defined as error that arises in the process of identifying the study populations? Selection bias can be defined as error that arises in the process of identifying the study populations?

31. Selection Bias in Cohort studies Study: exposed: factory workers unexposed: sample from population ? different „loss to follow-up“ in exposed and unexposed group not comparable In Cohort studies selection bias is less of a problem because the study population is selected before the outcome of the study (or without knowing the outcome of the study). Here an example for selection bias in cohort studies: a study investigates a disease typical for factory workers. So in the exposed group there are many factory workers and in the unexposed group a sample of the general population of the same region. There are two possible Bias: 1. if the loss to follow-up is different in the exposed and unexposed group. This could be the case if many of the factory workers were foreigners. They might have left the place after a few years (especially if they got sick) and cant be tracked down. Therefore the exposed group has a higher Loss-to-follow-up und the groups are uncomparable. In Cohort studies selection bias is less of a problem because the study population is selected before the outcome of the study (or without knowing the outcome of the study). Here an example for selection bias in cohort studies: a study investigates a disease typical for factory workers. So in the exposed group there are many factory workers and in the unexposed group a sample of the general population of the same region. There are two possible Bias: 1. if the loss to follow-up is different in the exposed and unexposed group. This could be the case if many of the factory workers were foreigners. They might have left the place after a few years (especially if they got sick) and cant be tracked down. Therefore the exposed group has a higher Loss-to-follow-up und the groups are uncomparable.

32. Selection Bias in Cohort studies “Healthy worker effect”: Diseased stay at home ? Underestimation of relative risk The 2nd bias may be the „Healthy worker effect“: The factory workers in the exposed group have a better health than the general population in the unexposed group. This can lead to an underestimation of the relative risk. The 2nd bias may be the „Healthy worker effect“: The factory workers in the exposed group have a better health than the general population in the unexposed group. This can lead to an underestimation of the relative risk.

33. Control for Selection Bias in Cohort study Examine proportion of individuals excluded from analysis: Refused to participate Lost to follow up Had missing data Voluntary sample Are excluded individuals different from study population? So how can we control for selection bias we should examine the proportion of individuals which were excluded from the analysis, either because they: refused to participate or were lost to follow up (this means they stepped out in the middle of the study) or because they had missing data. We should investigate whether these excluded individuals differed from the study population in characteristics investigated in the study. This would mean that the study results are not generalizable for the whole population ?So how can we control for selection bias we should examine the proportion of individuals which were excluded from the analysis, either because they: refused to participate or were lost to follow up (this means they stepped out in the middle of the study) or because they had missing data. We should investigate whether these excluded individuals differed from the study population in characteristics investigated in the study. This would mean that the study results are not generalizable for the whole population ?

34. Selection Bias in Cross-sectional study Non-response Voluntary sample Exposed more interested in taking part in the study Overestimation of risk Are excluded individuals different from study population? Adjust with standardisation method

35. Different types of error We looked now at selection bias. Lets revise information bias, which can be divided into observer and recall bias?We looked now at selection bias. Lets revise information bias, which can be divided into observer and recall bias?

36. Information Bias = Differential misclassification It is a systematic error in the measurement of information on exposure and outcome ? Biased estimate of strength of association between risk factor and disease information bias is also called differential misclassification. It is a systematic error in the measurement of information on exposure and outcome and the result is a biased estimate of strength of association between the risk factor and disease. let me show you an example of observation bias? information bias is also called differential misclassification. It is a systematic error in the measurement of information on exposure and outcome and the result is a biased estimate of strength of association between the risk factor and disease. let me show you an example of observation bias?

37. Observer bias Cohort study: strength of association between pleural mesothelioma and asbestos Difficult to diagnose by histology (cut-off unclear) Investigators think asbestos is important cause of pleural mesothelioma Pathologists more inclined to diagnose mesothelioma from exposed than unexposed person It is known that asbestos is an important cause of pleural mesothelioma (cancer of the mesothel), which is difficult to diagnose by histology. In a cohort study pathologists who look at histology slides to determine whether a study participant has mesothelioma or not, might be more inclined to diagnose mesothelioma from an exposed than an unexposed person. The dry definition of observer bias is: ?. It is known that asbestos is an important cause of pleural mesothelioma (cancer of the mesothel), which is difficult to diagnose by histology. In a cohort study pathologists who look at histology slides to determine whether a study participant has mesothelioma or not, might be more inclined to diagnose mesothelioma from an exposed than an unexposed person. The dry definition of observer bias is: ?.

38. Observer bias Occurs if information on exposure status influences classification of disease status, or vice versa Problem in case-control studies, cohort study studies and non-blinded clinical trials Information on exposure status influences classification of disease status, or vice versa. Problem in case-control, cohort study and non-blinded clinical trial. Now let me show you an example for the other form of information bias, recall bias ? Information on exposure status influences classification of disease status, or vice versa. Problem in case-control, cohort study and non-blinded clinical trial. Now let me show you an example for the other form of information bias, recall bias ?

39. Control for Information Bias Blind measurements Use placebo for study participants and/or observers

40. Different types of error

41. Confounding bias happens if observed effect is not because of an investigated association between exposure and outcome, but because of a third variable which interferes with the association So: Confounding happens if an observed effect is not because of the investigated association between an exposure and outcome, but because of a third variable which interferes with the association. So: Confounding happens if an observed effect is not because of the investigated association between an exposure and outcome, but because of a third variable which interferes with the association.

42. 02.07.08 Confounding = alternative explanations! Study result: risk of bone fracture higher in skiers than in snowboarders Skiers older than snowboarders Age associated with increased risk of bone fracture Age = alternative explanation for higher risk of bone fracture in skiers ? Age = confounder the result of a study was that risk of bone fracture is higher in skiers than in snowboarders. But Skiers are older than snowboarders and Age is associated with increased risk of bone fracture. Age is an alternative explanation for higher risk of bone fracture in skiers and so Age is a confounder. Once you look at young skiers and old snowboarders, you wont find a difference in the risk of breaking bones.? Extra example:.the outcome in a study was: Coffee drinking causes cancer. Is that really true? No, but: coffee drinking is associated with smoking and smoking causes cancer So, smoking is a confounder! If you look at non-smoking coffee drinkers they dont have more cancer than the normal population!)the result of a study was that risk of bone fracture is higher in skiers than in snowboarders. But Skiers are older than snowboarders and Age is associated with increased risk of bone fracture. Age is an alternative explanation for higher risk of bone fracture in skiers and so Age is a confounder. Once you look at young skiers and old snowboarders, you wont find a difference in the risk of breaking bones.? Extra example:.the outcome in a study was: Coffee drinking causes cancer. Is that really true? No, but: coffee drinking is associated with smoking and smoking causes cancer So, smoking is a confounder! If you look at non-smoking coffee drinkers they dont have more cancer than the normal population!)

43. Confounding Bias A confounder must be associated with the exposure and independent of that exposure be a risk factor for the disease (outcome) In other words: a confounder must be associated with the exposure and independent of that exposure be a risk factor for the disease (outcome). The same graphically:? In other words: a confounder must be associated with the exposure and independent of that exposure be a risk factor for the disease (outcome). The same graphically:?

44. Confounding Bias A confounder must be associated with the exposure and independent of that exposure be a risk factor for the disease (outcome)? A confounder must be associated with the exposure and independent of that exposure be a risk factor for the disease (outcome)?

45. Control for Confounding Once you identified the hypothesis think about independent risk factors for the outcome (potential confounders) deal with those potential confounders, either at the study design or analysis stage How to check for confounding: Once you identified the hypothesis think about independent risk factors for the outcome (potential confounders). Example: you test the association between smoking and coronary heart disease. What other risk factors would you consider? Stress, dislipidemia, genetical predisposition, age, gender, they are all potential confounders in the association between smoking and heart attack. Check whether authors have dealt with these potential confounders, either at the study design or analysis stage.? How to check for confounding: Once you identified the hypothesis think about independent risk factors for the outcome (potential confounders). Example: you test the association between smoking and coronary heart disease. What other risk factors would you consider? Stress, dislipidemia, genetical predisposition, age, gender, they are all potential confounders in the association between smoking and heart attack. Check whether authors have dealt with these potential confounders, either at the study design or analysis stage.?

46. Control Confounding at the stage of Very quickly the different possibilities to control confounding. Confounding can be controlled at the stage of Study design and analysis (remember bias cannot be controlled at the stage of analysis!). At the stage of study design you can use restriction, randomisation and matching. At the analysis stage confounding can be controlled by statistical modelling, comparability and stratified analysis..? Very quickly the different possibilities to control confounding. Confounding can be controlled at the stage of Study design and analysis (remember bias cannot be controlled at the stage of analysis!). At the stage of study design you can use restriction, randomisation and matching. At the analysis stage confounding can be controlled by statistical modelling, comparability and stratified analysis..?

47. Control for Confounding At study design stage: Restriction: only one age group or gender etc. Randomisation in intervention study: comparability of intervention and control groups Matching: compare exposed and unexposed persons with similar characteristics for potential confounder I will quickly run through the possibilities on how to control confounding at the study design stage. You will learn about this tomorrow and you dont need to know more than what I said on the former slide to critically asses the paper I will give you. You can control confounding by restriction: study only includes one age group or gender etc. , so you compare Young skiers to young snowboarders. Randomisation and a big enough sample size increases the chance that the unknown confounding factors are distributed evenly in the control and intervention group. and by Matching: compare exposed and unexposed persons with similar characteristics for the potential confounder. So again with the former example match skiers and snowboarders by age.? I will quickly run through the possibilities on how to control confounding at the study design stage. You will learn about this tomorrow and you dont need to know more than what I said on the former slide to critically asses the paper I will give you. You can control confounding by restriction: study only includes one age group or gender etc. , so you compare Young skiers to young snowboarders. Randomisation and a big enough sample size increases the chance that the unknown confounding factors are distributed evenly in the control and intervention group. and by Matching: compare exposed and unexposed persons with similar characteristics for the potential confounder. So again with the former example match skiers and snowboarders by age.?

48. Control for Confounding At analysis stage: Demonstrate comparability Stratified analysis (Maentel Haensel) Statistical modelling (Regression analysis) At the analysis stage one can check for confounding by stratifying (dividing) the skiers and snowboarders by age thus by the confounding factor and investigating the risk of bone fracture within these groups, the risk of bone fracture independantly of age can be analysed. Comparability means that you look whether confounders are equally distributed in both groups.statistische Modelling is more complex and there is not enough time to explain ist. It is important you know that it can be used to control confounding ? At the analysis stage one can check for confounding by stratifying (dividing) the skiers and snowboarders by age thus by the confounding factor and investigating the risk of bone fracture within these groups, the risk of bone fracture independantly of age can be analysed. Comparability means that you look whether confounders are equally distributed in both groups.statistische Modelling is more complex and there is not enough time to explain ist. It is important you know that it can be used to control confounding ?

49. Different types of error

50. Measurement error = Non-differential misclassification Exposure or outcome measured in study is not true exposure or outcome Same mistake in compared groups Measurement error occurs if the exposure or outcome measured in the study does not reflect the true exposure or outcome. Check if: Instruments and observers were assigned randomly or equivalently to compared groups . Methods of measurements conform to standards If not, was systematic measurement error checked for? Measurement error occurs if the exposure or outcome measured in the study does not reflect the true exposure or outcome. Check if: Instruments and observers were assigned randomly or equivalently to compared groups . Methods of measurements conform to standards If not, was systematic measurement error checked for?

51. Control for measurement error Assign instruments and observers randomly or equivalently to compared groups Use methods of measurements conform to standards Check for systematic measurement error

53. 10 minutes break 53

54. Leptospirosis in meat workers -Case Study

55. Leptospirosis Endemic bacterial disease of livestock in New Zealand Reservoirs in wild and domestic mammals Survives in environment Animal are often “silent” shedders (urine) Enter through skin abrasions or mucosae Symptoms range from mild flu haemorrhagic disease death

56. What determines disease in an individual?

58. Leptospirosis Currently most important occupationally-acquired zoonotic disease in meat-workers and farmers in NZ Suffering & time away from work & ACC costs Same serovars in livestock and humans Low vaccination in sheep, deer, beef

59. Occupational exposures NZ 2008 121 cases notified Meat processing = 34% Farming = 39% 45% hospitalised Under-reported?

60. Prevalence in animals in NZ Deer farms also grazing cattle were 17 times as likely to be infected with pomona than deer-only farmsDeer farms also grazing cattle were 17 times as likely to be infected with pomona than deer-only farms

61. Cross sectional & cohort study in abattoir Voluntary sampling of meat workers 1 abattoir slaughtering sheep blood sampling & interviewing meat workers (2 times in one year)

62. Interview by questionnaire Personal data name, address, age, gender, ethnicity, previous disease Risk factors for infection Work related: time in industry, position, species, PPE* use Off-work activities: home slaughter, contact with animals, hunting, water sports

63. Diagnostics Test blood for antibodies against Lepto Microscopic Agglutination Test (MAT) is standard reference test Indirect method to measure amount of antibodies in blood = titre High sensitivity & specificity Serovar specific

64. Objectives to determine in meat workers sero-prevalence and annual sero-conversion (incidence) Leptospira serovars Hardjo, Pomona Risk factors for leptospirosis infection work position use of PPE home slaughter hunting farm work etc.

65. Objectives: to investigate Association between flu-like symptoms and sero-conversion and calculate economic costs from time off work due to disease Options for control

66. Implementation: 1st year one plant Acceptance by Human Ethics Committee Recruit meat plants Recruit, interview & blood sample meat workers Test blood samples in lab Build database & enter data Descriptive, univariable & multivariable statistical analysis Communication of results: scientific report for abattoir management letters to participants

67. Results 2008 242 meat workers blood sampled at a sheep-only plant 9.5% sero-positive 60% of sero-positives had previous disease Same types in people as in deer, sheep, cattle

69. Results of repeated testing in 2009 135 workers again tested 12 % of formerly antibody negative participants developed antibodies against Pomona or Hardjo ? a worker has a 12% risk of infection for each season from sheep carcasses None of the study participants described severe clinical leptospirosis

70. Results 2009

71. Wake up – your turn 71

73. 1. Bias Sampling Bias (= selection bias) due to voluntary sampling Are people at higher risk more inclined to take part in the study? Overestimation of prevalence By choosing negatives for cohort study, we select people at lower risk Underestimation of true incidence “Healthy Worker effect”

74. Control for sampling bias I Study: Persuade as many as possible from all working areas to participate, try to reduce loss-to follow up Analysis: Adjust for sampling bias Get info on demographics and risk factors from entire work force

75. Control for sampling bias II Analysis: Adjust for sampling bias (cont.) Evaluate associations between population characteristics and observed prevalence Females have lower prevalence Compare significant population parameters between sample and data of entire workforce gender ratio Sampling bias with respect to each significant parameter is defined as the relative difference between sample and population

76. Control for sampling bias III Analysis: Adjust for sampling bias (cont.) The fraction of a parameter in the population is used to indirectly standardise the sample prevalence for each plant to obtain comparable plant specific prevalence Healthy worker effect Model incidence from prevalence if you know antibody titre duration and compare to measured incidence

77. 2. Confounding factors Hunting Home slaughter Working on a farm Work time Age Gender Ethnicity

78. Confounding factors

79. Control for confounding Ask about other risk factors in interview Bivariable analysis: stratify by confounder Multivariable analysis: ad risk factors to logistic regression model

80. 3. Outcome Determination of the prevalence and incidence quantifies public health hazard gives measure of underreporting and subclinical infection of human Leptospirosis Raise awareness of practitioners, stakeholders Estimates of risk factors for sero-conversion enable recommendations for control measures (PPE)

81. 4. Measurement error Depending on chosen MAT cut-off Different prevalence and incidence

82. Study 2009 Serovar distribution by antibody titre

83. 5. External Validity No, meat workers have much more exposure than general population No, because 1 sheep plant is not representative for all plants Sample more meat plants from different parts of the country slaughtering different animal species

84. Thank you for your attention!