Interpreting Epidemiological Evidence: Bias in Association Studies

1. Module 4 Overview Context Content Area: Interpretation of Epidemiological Evidence Essential Question (Generic): Is the association causal? Essential Question (Drug Abuse Specific): Is an association with drug abuse causal? Enduring Epidemiological Understanding: Causation is only one explanation for finding an association between an exposure and a disease. Because observational studies are complicated by factors not controlled by the observer, other explanations must also be considered. Synopsis: In Module 4, students explore the rationale and methods of interpreting epidemiological studies. Students develop skills to assess possible explanations for an association found in a study, with consideration of explanations of causality, chance, confounding, reversed time order, and bias. Module 4 concludes with consideration of methods for weighing the overall evidence for an association. Lessons: Lesson 4-1: Introduction to Interpreting Associations Lesson 4-2: Causality Lesson 4-3: Chance Lesson 4-4: Bias Lesson 4-5: Confounding Lesson 4-6: Reverse Time Order Lesson 4-7: Weighing the Evidence

2. Module 4 - Interpretation of Epidemiological Evidence • Lesson 4-4 Bias • Content • How a study can be in error • Bias in selection of study subjects and how it can produce an erroneous result, using an example of student drug testing • Bias in measurement during a study and how it can produce an erroneous result, using an example of measurement of cocaine use • Big Ideas • Association does not necessarily mean causation • When an association is found, several possible explanations must be considered, including the possibility that the association is observed because of bias in the study. • Systematic error in selection of study participants or in measurement of exposure or outcomes will result in study results that are not accurate. This project is supported by a Science Education Drug Abuse Partnership Award, Grant Number 1R24DA016357-01, from the National Institute on Drug Abuse, National Institutes of Health.

3. Where Are We? Essential Questions Enduring Understandings

4. Bias Is the association due to bias? 1. Cause 2. Chance 3. Bias

5. Explanations for Finding an Association Bias

6. Definition Bias Deviation from truthful estimates of risk or inference from a study because of systematic error in the design or conduct of the study A faulty assumption in a study that produces a result or interpretation that is not true

7. Definition Selection Bias Error in a study because of systematic differences in characteristics between those who are selected to be in a study and those who are not selected.

8. Definition Measurement Bias Error in a study because of systematic, inaccuracies in measurement of exposures or outcomes

9. Explanations for Finding an Association Is the association due to bias?

10. Evaluating A Drug-Related Program • Real Scenario - in an effort to curb drug abuse among students, some schools have implemented random mandatory drug-testing programs • These activities have generated controversy about the ethics and the effectiveness of such programs

11. Evaluating A Drug-Related Program • Should high school drug-testing programs be evaluated? “… the efficacy of this means for addressing the problem (is) self-evident.” Justice Antonin Scalia, 1995 “Schools should not implement a drug testing program until they’re proven to work. …. They are too expensive. It’s like having experimental surgery that’s never been shown to work.” Linn Goldberg, Oregon Health and Science University

12. Background on Testing Controversies “(W)e find that testing students who participate in extracurricular activities isa reasonably effective meansof addressing the School District’s legitimate concerns in preventing, deterring, and detecting drug use.” Justice Clarence Thomas (June 27, 2002)

13. Evaluating Strategies • Should high school drug-testing programs be evaluated? • What are the advantages of evaluating high school drug testing programs?

14. Evaluating Strategies • Should high school drug-testing programs be evaluated? • What are the advantages of evaluating high school drug testing programs? • How should high school drug-testing programs be evaluated?

15. Evaluating Strategies % or a b c d % or

16. Evaluating Strategies No Drug Use Drug Use Drug Testing No Drug Testing Risks of Drug Use Relative Risk Total % or a b 1 c d % or

17. No Drug Use Drug Use Drug Testing No Drug Testing Evaluating Strategies Risks of Drug Use Relative Risk Total % or a b < 1 c d % or

18. No Drug Use Drug Use Drug Testing No Drug Testing Evaluating Strategies Risks of Drug Use Relative Risk Total % or a b > 1 c d % or

19. DZ DZ E DZ E DZ E DZ DZ Random Assignment DZ DZ E E DZ DZ DZ DZ E E E Review - The Four Study Designs Controlled Trial Case-Control Study Healthy People Healthy People Cohort Study Cross-Sectional Study E Healthy People Healthy People E

20. E E Time Review - Cross-Sectional Study Flow Diagram DZ DZ Cross-SectionalStudy Flow Diagram Observational Study

21. Drug Testing Program Drug Testing Program Drug Use Drug Use Cross-Sectional Study Time

22. DZ DZ E DZ E Healthy People DZ E DZ DZ Random Assignment Healthy People Healthy People DZ DZ E E - - - - DZ E DZ Healthy People DZ DZ E E E E ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Time Time Time Time Should high school students be drug tested? Case-Control Study Controlled Trial Cohort Study Cross-Sectional Study

23. Cross-Sectional Study R Yamaguchi, LD Johnston, O’Malley. Relationship between student illicit drug use and school drug-testing policies. Journal of School Health 2003;73(4):159-164.

24. Cross-Sectional Study R Yamaguchi, LD Johnston, O’Malley. Relationship between student illicit drug use and school drug-testing policies. Journal of School Health 2003;73(4):159-164. Exposure Outcome

25. Cross-Sectional Study - Exposure R Yamaguchi, LD Johnston, O’Malley. Relationship between student illicit drug use and school drug-testing policies. Journal of School Health 2003;73(4):159-164. Youth, Education and Society Exposure

26. Cross-Sectional Study - Exposure Youth, Education, and Society Study School-Level Information • Mailed Survey • Answered by a school administrator, usually a principal • 82% response rate

27. Exposure Measures Youth, Education, and Society Study Questions about school policy In the school year, were there . . . ? Any random searches of students for illicit drugs ? Any searches of students based on cause or suspicion for illicit drugs ? Any tests of students for illicit drugs

28. Cross-Sectional Study - Outcome R Yamaguchi, LD Johnston, O’Malley. Relationship between student illicit drug use and school drug-testing policies. Journal of School Health 2003;73(4):159-164. Monitoring the Future survey Outcome

29. Cross-Sectional Study - Outcome Monitoring the Future Survey Self-Administered Questionnaires 30,000 8th grade students in 260 schools 23,000 10th grade students in 227 high schools 23,000 12th grade students in 235 high schools

30. Outcome Measures Monitoring the Future Survey On how many occasions (if any) have you used marijuana? - In your lifetime - During the last 12 months - During the last 30 days?On how many occasions (if any) have you used drugs other than marijuana (separate questions for LSD, crack, amphetamines, heroin and several others - In your lifetime - During the last 12 months - During the last 30 days?

31. Cross-Sectional Study R Yamaguchi, LD Johnston, O’Malley. Relationship between student illicit drug use and school drug-testing policies. Journal of School Health 2003;73(4):159-164. • What percentage of schools employ a drug testing policy? • Which students are tested for drugs in these schools? • On what bases are students tested for drugs in school • What relationship exists between school drug testing and student drug use?

32. Descriptive Epidemiology R Yamaguchi, LD Johnston, O’Malley. Relationship between student illicit drug use and school drug-testing policies. Journal of School Health 2003;73(4):159-164. • What percentage of schools employ a drug testing policy? • Which students are tested for drugs in these schools? • On what bases are students tested for drugs in school • What relationship exists between school drug testing and student drug use?

33. Prevalence of Testing, Groups Tested, Reasons for Testing

34. How do you interpret the difference between the % of schools and the % of students? Descriptive Epidemiology - Prevalence of Testing

35. What do you think happened to the % of schools that did “drug testing of any kind” between 1998 and 2001? Descriptive Epidemiology - Prevalence of Testing

36. Descriptive Epidemiology - Prevalence of Testing What do you think happened to the % of schools that did “drug testing of any kind” between 1998 and 2001?

37. Descriptive Epidemiology - Prevalence of Testing What group of students do you think was most frequently drug tested?

38. Descriptive Epidemiology - Groups Tested

39. Descriptive Epidemiology - Reasons for Testing What do you think was the most frequent bases for drug testing?

40. Descriptive Epidemiology - Reasons for Testing

41. Descriptive Epidemiology R Yamaguchi, LD Johnston, O’Malley. Relationship between student illicit drug use and school drug-testing policies. Journal of School Health 2003;73(4):159-164. • What percentage of schools employ a drug testing policy? • Which students are tested for drugs in these schools? • On what bases are students tested for drugs in schools?

42. Analytical Epidemiology - Associations R Yamaguchi, LD Johnston, O’Malley. Relationship between student illicit drug use and school drug-testing policies. Journal of School Health 2003;73(4):159-164. • What percentage of schools employ a drug testing policy? • Which students are tested for drugs in these schools? • On what bases are students tested for drugs in schools? • What relationship exists between school drug testing and student drug use?

43. Review - Things That Turn Up Together Tied Related Associated Linked What do we mean when we say that there is anassociationbetween two things? Things that are associatedare linked in some way that makes themturn up together.

44. No Drug Use Drug Use Drug Testing No Drug Testing Relationship between Drug Testing and Drug Use Risks of Drug Use Relative Prevalence Total % or a b ? c d % or In this study, percent of drug use is calculated for the drug testing schools versus the schools that did not drug test

45. No Drug Use Drug Use Drug Testing No Drug Testing Relationship between Drug Testing and Drug Use Risks of Drug Use Relative Prevalence Total % or a b c d % or The study report provides the percentages If we divide one percentage by the other percentage, what does this give us? The relative prevalence

46. Relationship between Drug Testing and Drug Use

47. 1 = 0 occasions 2 = 1-2 occasions 3 = 3-5 occasions 4 = 6-9 occasions 5 = 10-19 occasions 6 = 20-39 occasions 7 = 40 or more occasions 1 – 7 Scale Prevalence The number of events in a given population at a designated time Standard Deviation (SD) A measure of dispersion or variation. A summary of how widely dispersed the values are around the mean. Relationship between Drug Testing and Drug Use

48. ? ? Relationship between Drug Testing and Drug Use

49. Risk of Over-weight Relative Risk Drug Use Drug Use No Drug Use No Drug Use Prevalence of Over-weight Relative Prevalence Total Total Drug Testing Drug Testing 16% 16% a a b b ~1 ~1 c c d d No Drug Testing No Drug Testing 15% 15% Relationship between Drug Testing and Drug Use

50. Relationship between Drug Testing and Drug Use Optional - Math teachers may want to point out the great variability in these data, as shown by the large standard deviations relative to the means