Observational Studies Can’t Alone Determine Causation

1. Observational Studies Can’t Alone Determine Causation References: Robins, J. and Wasserman, L. 1999. On the Impossibility of Inferring Causation from Association without Background Knowledge. Computation, Causation and Discovery, Eds. Glymour, P. and Cooper, G.

2. The problem with observational studies: lack of randomization • If one has a treatment, or risk factor, with two levels (A and B), no guarantee that study populations (those getting A and B) will be roughly equivalent (in risk of the disease of interest). • In a perfect world can given everyone, every in study level A, record outcome, reset clock and then give level B. • Randomization means one can interpret estimates as if this is precisely what was done.

3. The Point of Debate • Some (Sprites, et al., 1993; Pearl and Verma, 1991) that given a causal graph and: • Large Sample Size • Distribution of random variables is faithful to the causal graph • One can infer causal relationships between some variables X  Y from associations measured using observational data

4. A Specific Example • Consider the graph below: Sprites et al. assert that, if one estimates statistical independence of X and Y (e.g., correlation 0), then, given their assumptions, one can assume that: • No arrow (cause) goes from X to Y, and • There is no confounding as well.

5. Faithfulness Assumption • The rely on a faithfulness assumption that basically states, given the graph, the effect of X is not exactly cancelled by the effect of U giving the appearance of independence.

6. Using the Bayes Factor to analyze their claim • Robins, Wasserman look at the asymptotics of the Bayes Factor: where A is the event X causes Y, Ac is X does not cause Y and Zn is the data. • In their set-up, Bn is the posterior odds of the event Ac. • They examine the behavior of Bn as sample size increases and under different assumptions regarding the potential confounders.

7. Bayes Factor • Note that if Bn is near 0, one would infer a causal relationship, and as Bn , no causal relationship between X and Y.

8. Their Asymptotic Results • In their specific case if A is true, then they show that Bn 1, meaning no conclusion regarding causation can be made • In addition, if Ac is true, they show that if the number of confounders, k, is relatively small compared to the sample size, then Bn  and their claim is vindicated. • However, if the number of confounders is large relative to log(n), and Ac is true, then Bn 1. • In section 6, generalize this basic result to more general set-up.

9. Conclusions • Match the beliefs of practicing professionals. • The world contains many potentially unmeasured confounders for most variables of interest. • Highly unlikely that just a single one causes both X and Y. • Thus, in observational studies small causal effects can never be either reliably ruled in or ruled out. • One should not leap from even relatively large empirical associations to causation without substantive subject-matter-specific background information.