Understanding Selection Bias in Data Analysis: Concepts and Examples

1. Selection Bias Concepts Hein Stigum Presentation, data and programs at: http://folk.uio.no/heins/talks H.S.

2. Questions Given measured appropriate variables: Can you adjust for confounding? Yes Can you adjust for selection bias? Depends on the definition H.S.

3. Contents • Background • Define bias • Selection bias • as effect modification (old concept) • as collider stratification bias (new concept) • DAG structure • Examples • Size and direction of bias H.S.

4. Bias definition • Bias • Frequency: expected risk ≠ true risk • Effect: association ≠ causal effect H.S.

5. Selection bias concepts Concept DAG structure Effect modification Collider stratification bias Effect responders ≠ Effect non responders Differential response bias Differential loss to follow up Healthy worker bias Berkson’s bias (case control) H.S.

6. Selection bias as effect modification H.S.

7. Selection bias: Risk • Selection of responders  • The prevalence is different among • the responders compared to the full population • the responders compared to the non responders R0 Non responders Rp Population R1 Responders Rp is the weighted mean of R0 and R1 H.S.

8. Effect modification • Selection of responders  • The effect of E on D is different among • the responders compared to the full population • the responders compared to the non responders RR0 Non responders RRp Population RR1 Responders H.S.

9. Problems • Is not a bias, RR0 and RR1 are the true effects • Is effect modification by selection variable S • Leads to the conclusion that: • Biolocical effects are protected from bias • The bias can not be adjusted for • RRp is the average of RR0 and RR1 Not true for collider stratification bias “DAG” structure: S E D H.S.

10. Selection bias as collider stratification bias H.S.

11. Example with paths • Study • Milk on bone density • Exclude Calcium supplements S calcium supp. C family history S calcium supp. Lessons learned: Biological effect not protected May adjust for selection bias Structure: Collider stratification E milk E milk D bone density D bone density H.S.

12. Examples S respond C education • Differential response • Survey: Alcohol and CHD • Differential loss to follow up • Randomized trial: drug and disease • Healthy worker effect • Cross-section: Melt hall dust and lung disease E alcohol D CHD S working C health S loss to follow up C smoking E dust D lung disease E drug D disease Note: no confounding H.S.

13. Selection bias structure January 20 H.S. 13

14. Paths 1. Causal • 2. Confounding • An open non-causal path without colliders • 3. Selection bias • A non-causal path that is open due to conditioning on a collider BCVs? C C A B A B E D E D E A B D Causal Confounding Selection bias

15. Collider stratification bias • Selection bias = Collider stratification bias • Selection bias, Path definition • A non causal path that is open due to conditioning on a collider S S S C A B E D E D E D H.S.

16. Selection bias examples H.S.

17. Folic acid and cardiac malformation Selection: Study only live born Bias? Selection: Non grieving parents volonteer Bias? C Live born Yes, E[C]D is open E Folic acid D Card. Mal. S Grief Yes, E[C]D is (partially) open C Live born E Folic acid D Card. Mal. H.S.

18. Education and unfaithfulness • Study the effect among couples in a relationship (not divorced)? R divorced S sensation seeking E education D unfaithful Selection bias H.S.

19. Size and Direction of bias January 20 H.S. 19

20. Example 1, full table (Adjusted) RRs True and biased RRs Proportion responding in 1,1 group

21. Example 2 Pattern: Only D influence response Result: RR (and RD) biased, OR unbiased ODS, Case-Control

22. Example 3 Pattern: Both E and D influence response Result: Surprise: responders are unbiased Theory: bias in at least one stratum

23. Example 4 Pattern: Both E and D influence response Result: Surprise: both strata biased upwards True RR is not a weighted average

24. Example 5 Pattern: Both E and D influence response Result: Same DAG, different results The DAG does not fully determine the selection!

25. Summing up • Selection bias as “effect modification”: • Is not a bias, should not be called selection bias • Has properties different from proper selection bias • Selection bias as “collider stratification”: • Structure defined in DAG, • Distinct from confounding • Consistent with • Differential response bias • Differential loss to follow up • Healthy worker bias • Berkson’s bias (case control) H.S.

26. Litterature • Hernan and Robins, Causal Inference H.S.