1 / 55

Epidemiologic Methods - Fall 2004

Epidemiologic Methods - Fall 2004. Bias in Clinical Research: Selection Bias. Framework for understanding error in clinical research systematic error: threats to internal validity (bias) random error: sampling error (chance) Selection bias by study design: descriptive case-control

sema
Download Presentation

Epidemiologic Methods - Fall 2004

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Epidemiologic Methods - Fall 2004

  2. Bias in Clinical Research: Selection Bias • Framework for understanding error in clinical research • systematic error: threats to internal validity (bias) • random error: sampling error (chance) • Selection bias • by study design: • descriptive • case-control • cross-sectional • longitudinal studies (observational or experimental)

  3. Clinical Research: Sample Measure (Intervene) Analyze Infer • Inference • Websters: the act of passing from sample data to generalizations, usually with calculated degrees of certainty

  4. OTHER POPULATIONS Disease + - + - Exposure REFERENCE/ TARGET/ SOURCE POPULATION aka STUDY BASE STUDY SAMPLE

  5. 20 to 65 year olds, in Europe >65 years old in U.S. Disease + - + - Exposure 20 to 65 year olds, in U.S., outside of San Francisco San Franciscans, 20 to 65 years old SAMPLE of San Franciscans, 20 to 65 yrs old

  6. Disease + - + - Exposure REFERENCE/ TARGET/ SOURCE POPULATION aka STUDY BASE STUDY SAMPLE

  7. Error in Clinical Research • The goal of any study is to find the truth, i.e.: • measure of disease occurrence in a descriptive study • measure of association between exposure and disease in an analytic study • Ways of getting the wrong answer: • systematic error; aka bias • any systematic process in the conduct of a study that causes a distortion from the truth in a predictable direction • captured in the validity of the inference • random error; aka chance • occurs because we cannot study everyone (we must sample) • captured in the precision of the inference (e.g., confidence interval)

  8. Validity and Precision Good Validity Good Precision Poor Validity Poor Precision

  9. Validity and Precision Poor Validity Good Precision Good Validity Poor Precision

  10. Random error (chance) Validity and Precision Random error (chance) No Systematic error Systematic error (bias) Poor Validity Good Precision Good Validity Poor Precision

  11. Performing an Actual Study You only have one shot Only judgment can tell you about systematic error (validity) Field of “statistics” can tell you the random error (precision)

  12. Internal vs External Validity • Internal validity • Do the results obtained from the actual subjects accurately represent the target/reference/source population? • External validity (generalizability) • Do the results obtained from the actual subjects pertain to persons outside of the source population? • Internal validity is a prerequisite for external validity • “Validity” to us typically means internal validity

  13. OTHER POPULATIONS Disease + - ? INTERNAL VALIDITY + - Exposure REFERENCE/ TARGET/ SOURCE POPULATION ? EXTERNAL VALIDITY (generalizability) STUDY SAMPLE

  14. MetLife Is Settling Bias Lawsuit BUSINESS/FINANCIAL DESK | August 30, 2002, Friday MetLife said yesterday that it had reached a preliminary settlement of a class-action lawsuit accusing it of charging blacks more than whites for life insurance from 1901 to 1972. MetLife, based in New York, did not say how much the settlement was worth but said it should be covered by the $250 million, before tax, that it set aside for the case in February.

  15. “Bias” in Webster’s Dictionary 1: a line diagonal to the grain of a fabric; especially: a line at a 45° angle to the selvage often utilized in the cutting of garments for smoother fit2 a: a peculiarity in the shape of a bowl that causes it to swerve when rolled on the green b: the tendency of a bowl to swerve; also: the impulse causing this tendency c: the swerve of the bowl3 a: bent or tendencyb: an inclination of temperament or outlook; especially: a personal and sometimes unreasoned judgment : prejudice c: an instance of such prejudice d (1) : deviation of the expected value of a statistical estimate from the quantity it estimates (2) : systematic error introduced into sampling or testing 4 a: a voltage applied to a device (as a transistor control electrode) to establish a reference level for operation b: a high-frequency voltage combined with an audio signal to reduce distortion in tape recording

  16. Bias of Priene (600 - 540 BC) • One of the 7 sages of classical antiquity • Consulted by Croesus, king of Lydia, about the bestway to deploy warships against the Ionians • Bias wished to avoid bloodshed, so he misled Croesus, falselyadvising him that the Ionians were buying horses • Bias later confessed to Croesusthat he had lied and that the Ionians were also building warships. • Croesus was pleased with the way that he had been deceived byBias and made peace with the Ionians. BMJ 2002;324:1071

  17. Classification Schemes for Error • Szklo and Nieto • Bias • Selection Bias • Information/Measurement Bias • Confounding • Chance • Other Common Approach • Bias • Selection Bias • Information/Measurement Bias • Confounding Bias • Chance

  18. Selection Bias • Technical definition • Bias that is caused when individuals have different probabilities of being included in the study according to relevant study characteristics: namely, the exposure and the outcome of interest • Plain definition • Bias that is caused by some kind of problem in the process of selecting subjects initially or - in a longitudinal study - in the process that determines how long subjects stay in the study

  19. Selection Bias in a Descriptive Study • Pre-election surveys re: 1948 Presidential Election • various methods used to find subjects • largest % favored Dewey • General election results • Truman beat Dewey • Ushered in realization of the importance of representative (random) sampling

  20. N= 894 sample Actual Yes 4,717,006 (55%) No 3,809,090 (45%)

  21. Leukemia Incidence Among Observers of a Nuclear Bomb Test Caldwell et al. JAMA 1980 • Smoky Atomic Test in Nevada • Outcome of 76% of troops at site was later found; occurrence of leukemia determined 82% contacted by the investigators 18% contacted the investigators on their own 4.4 greater risk of leukemia than those contacted by the investigators

  22. Descriptive Study: Unbiased Sampling An even dispersion of arrows SOURCE POPULATION STUDY SAMPLE

  23. Descriptive Study: Selection Bias SOURCE POPULATION STUDY SAMPLE

  24. Analytic Study: Unbiased Sampling Disease + - + - Exposure SOURCE POPULATION STUDY SAMPLE

  25. Analytic Study: Selection Bias Diseased + - + - Exposed SOURCE POPULATION STUDY SAMPLE

  26. Selection Bias in Case-Control Studies Coffee and cancer of the pancreas MacMahon et al. N Eng J Med 1981; 304:630-3 Cases: patients with histologic diagnosis of pancreatic cancer in any of 11 large hospitals in Boston and Rhode Island between October 1974 and August 1979 What study base gave rise to these cases? How should controls be selected?

  27. Selection Bias in a Case-Control Study • Coffee and cancer of the pancreas • MacMahon et al. N Eng J Med 1981; 304:630-3 • Controls: • Other patients under the care of the same physician of the cases with pancreatic cancer. • Patients with diseases known to be associated with smoking or alcohol consumption were excluded

  28. Coffee and cancer of the pancreas MacMahon et al., (N Eng J Med 1981; 304:630-3) Case Control Coffee: > 1 cup day No coffee 216 307 OR= (207/9) / (275/32) = 2.7 (95% CI, 1.2-6.5)

  29. Relative to the study base that gave rise to the cases, the: • Controls were: • Other patients under the care of the same physician at the time of an interview with a patient with pancreatic cancer • Most of the MDs were gastroenterologists whose other patients were likely advised to stop using coffee • Patients with diseases known to be associated with smoking or alcohol consumption were excluded • Smoking and alcohol use are correlated with coffee use; therefore, sample is relatively depleted of coffee users

  30. Case-control Study of Coffee and Pancreatic Cancer: Selection Bias Cancer No cancer coffee no coffee SOURCE POPULATION STUDY SAMPLE

  31. 84 82 10 14 • Coffee and cancer of the pancreas: • Use of population-based controls • Gold et al. Cancer 1985 Case Control Coffee: > 1 cup day No coffee OR= (84/10) / (82/14) = 1.4 (95% CI, 0.55 - 3.8)

  32. Selection Bias in a Cross-sectional Study • Inclusion of prevalent cases causes all sorts of problems • Finding a diseased person in a cross-sectional study requires 2 things: • the disease occurred in the first place • the case survived long enough to be sampled • Any factor associated with a prevalent case of disease might be associated with disease development, survival with disease, or both • Assuming goal is to find factors associated with disease development, bias in prevalence ratio occurs any time that exposure under study is associated with survival with disease

  33. Cross-Sectional Study Design

  34. Selection Bias in a Cross-sectional Study e.g., Smoking and emphysema • Smoking is a cause of emphysema, but persons with emphysema who continue to smoke have shorter survival • Hence, in any cross-section of persons with emphysema, those who smoke less are apt to be more greatly represented (because of the survival disadvantage of those who continue to smoke) • Therefore, cross-sectional study of current smoking and emphysema will result in a prevalence ratio that underestimates the entity you are presumably interested in: the incidence ratio

  35. Cross-sectional study of smoking and emphysema Emphysema + - + - Smoke SOURCE POPULATION STUDY SAMPLE

  36. Selection Bias in a Cross-Sectional Study • Is glutathione S-transferase class  deletion (GSTM1-null) polymorphism associated with increased risk of breast cancer? • With prevalent breast cancer cases, an association with GSTM1-null is seen depending upon the number of years since diagnosis • But not with incident cases Kelsey et al. Canc Epi Bio Prev 1997

  37. Selection Bias: Cohort Studies/RCTs • Among initially selected subjects, selection bias less likely to occur compared to case-control or cross-sectional studies • Reason: study participants (exposed or unexposed; treatment vs placebo) are selected (theoretically) before the outcome occurs

  38. Cohort Study/RCT Since disease has not occurred yet among initially selected subjects, there is no opportunity for disproportionate sampling with respect to exposure and disease Disease + - + - Exposure SOURCE POPULATION STUDY SAMPLE

  39. Cohort Study/RCT All that is sampled is exposure status Even if disproportionate sampling of exposed or unexposed groups occurs, it will not result in selection bias when forming measures of association Disease + - + - Exposure SOURCE POPULATION STUDY SAMPLE

  40. Selection Bias: Cohort Studies • Selection bias can occur on the “front-end” of the cohort if diseased individuals are unknowingly entered into the cohort • e.g.: • Consider a cohort study of effect of exercise on all-cause mortality in persons initially thought to be completely healthy. • If some participants were enrolled had undiagnosed cardiovascular disease and as a consequence were more likely to exercise less, what would the effect be on the measure of association?

  41. Cohort Study of Exercise and Survival Selection bias will lead to spurious protective effect of exercise (assuming truly no effect) Death No death exercise no exercise SOURCE POPULATION STUDY SAMPLE

  42. Selection Bias: Cohort Studies/RCTs • Most common form of selection bias does not occur with the process of initial selection of subjects • Instead, selection bias most commonly caused by forces that determine length of participation (who ultimately stays in the analysis) i.e. loss to follow-up • When those lost to follow-up have a different probability of the outcome than those who remain (i.e. informative censoring) AND • Frequency of lost to follow-up is different across exposure groups, or degree of informative censoring differs across exposure groups (i.e. when loss to follow-up is associated with both outcome and exposure) • selection bias results

  43. Selection Bias: Cohort Studies/RCTs e.g., Cohort study of progression to AIDS: IDU vs homosexual men • In general, getting sicker is a common reason for loss to follow-up • Therefore, persons who are lost to follow-up have different AIDS incidence than those who remain (i.e., informative censoring) • In general, IDU more likely to become lost to follow-up - at any given level of feeling sick • Therefore, the frequency of informative censoring differs across exposure groups (IDU vs homosexual men) • Results in selection bias: underestimates the incidence of AIDS in IDU relative to homosexual men

  44. Effect of Selection Bias in a Cohort Study Effect of informative censoring in homosexual male group Effect of informative censoring in IDU group Probability of being AIDS-free Survival assuming no informative censoring and no difference between IDU and homosexual men Time

  45. Cohort Study of HIV Risk Group and AIDS Progression Selection bias will lead to spurious underestimation of AIDS incidence in both exposure groups, more so in IDU group AIDS No AIDS IDU Homo-sexual men SOURCE POPULATION STUDY SAMPLE

  46. Selection Bias in a Clinical Trial • If randomization is performed correctly, then selection bias on the “front-end” of the study (i.e., differential inclusion of diseased individuals between arms) is not possible • even if diseased individuals are unknowingly included, the randomization process ensures that this occurs evenly across treatment groups • However, beware of imposters to randomization, such as “every other participant was assigned”, that can be deciphered by participants or staff • can lead to differential distribution of diseased or high-risk participants across treatment groups

  47. Selection Bias in a Clinical Trial • Losses to follow-up are the big unknown in clinical trials and the major potential for selection bias • e.g., • a symptomatic side effect of a drug is more common in persons “sick” from disease • occurrence of the side effect is associated with more losses to follow-up • Then: • drug treatment group would be selectively depleted of the sickest persons (i.e., informative censoring) • drug treatment group looks better overall

More Related