Judea Pearl Computer Science Department UCLA cs.ucla/~judea

1. Judea Pearl Computer Science Department UCLA www.cs.ucla.edu/~judea DIRECT AND INDIRECT EFFECTS

2. QUESTIONS ASKED • Why decompose effects? • What is the semantics of direct and indirect effects? • What are the policy implications of direct and indirect effects? • Can path-analytic techniques be extended to nonlinear and nonparametric models? • When can direct and indirect effect be estimatedconsistently from experimental and nonexperimental data.

3. WHY DECOMPOSE EFFECTS? • Direct (or indirect) effect may be more transportable. • Indirect effects may be prevented or controlled. • Direct (or indirect) effect may be forbidden  Pill Pregnancy + + Thrombosis Gender Qualification Hiring

4. b X Z a c Y a bc EFFECT-DECOMPOSITION IN LINEAR MODELS Definition:

5. COUNTERFACTUALS: STRUCTURAL SEMANTICS u u W W Z X X=x Z Y Yx(u)=y Notation: Yx(u) = yAbbreviation:yx Formal:Y has the value y in the solution to a mutilated system of equations, where the equation for X is replaced by a constant X=x. Functional Bayes Net Probability of Counterfactuals:

6. TOTAL, DIRECT, AND INDIRECT EFFECTS HAVE SIMPLE SEMANTICS IN LINEAR MODELS b X Z z = bx + 1 y = ax + cz + 2 a c Y a+bc a bc

7. SEMANTICS BECOMES NONTRIVIAL IN NONLINEAR MODELS (even when the model is completely specified) X Z z = f (x, 1) y = g (x, z, 2) Y Dependent on z? Void of operational meaning?

8. NEED OF FORMALIZATION What is the direct effect of X on Y? X Z = AND Y Indirect Effect?

9. TWO CONCEPTIONS OF DIRECT AND INDIRECT EFFECTS: Controlled vs. Natural X Z = AND Y Starting from X=0, (and Z=0 and Y=0) Total Effect: Change X from 0 to 1, and test the change in Y. Controlled DE: Keep Z constant at Z=0, or Z=1, and change X=0 to X=1. Controlled IE: None. Natural DE: Keep Z constant at its current value, and change X to 1. Natural IE: Keep X at 0, but set Z to what it would be if X were 1.

10. LEGAL DEFINITIONS TAKE THE NATURAL CONCEPTION (FORMALIZING DISCRIMINATION) ``The central question in any employment-discrimination case is whether the employer would have taken the same action had the employee been of different race (age, sex, religion, national origin etc.) and everything else had been the same’’ [Carson versus Bethlehem Steel Corp. (70 FEP Cases 921, 7th Cir. (1996))] x = male, x = female y = hire, y = not hire z = applicant’s qualifications NO DIRECT EFFECT

11. TWO CONCEPTIONS OF AVERAGE DIRECT AND INDIRECT EFFECTS: POPULATION-LEVEL DEFINITIONS u3 u2 X Z (all other parents of Y) Probabilistic causal model: y = f (x,z,u) á ñ u1 M, P(u) Y Starting from X=x*, (and Z=Zx*(u) and Y= Yx*(u)) Total Effect: TE(x,x*;Y) = E(Yx) – E(Yx*) Controlled DE: CDEZ(x,x*;Y) = E(Yxz) – E(Yx*z) Controlled IE: None. Natural DE: NDE(x,x*;Y) = E(YxZx*) – E(Yx*) Natural IE: NIE(x,x*;Y) = E(Yx*Zx) – E(Yx*)

12. THE OPERATIONAL MEANING OF AVERAGE DIRECT EFFECTS X Z z = f (x, 1) y = g (x, z, 2) Y “Natural” Direct Effect of X on Y: The expected change in Y per unit change of X, when we keep Z constant at whatever value it attains before the change. In linear models, NDE = Controlled Direct Effect

13. GENDER QUALIFICATION HIRING POLICY IMPLICATIONS (Who cares?) indirect What is the direct effect of X on Y? The effect of Gender on Hiring if sex discrimination is eliminated. X Z IGNORE f Y

14. THE OPERATIONAL MEANING OF INDIRECT EFFECTS X Z z = f (x, 1) y = g (x, z, 2) Y “Natural” Indirect Effect of X on Y: The expected change in Y when we keep X constant, say at x0, and let Z change to whatever value it would have under a unit change in X. In linear models, NIE = TE - DE

15. GRAPHICAL CONDITION FOR EXPERIMENTAL IDENTIFICATION OF AVERAGE NATURAL DIRECT EFFECTS Theorem: If there exists a set W such that Example:

16. HOW THE PROOF GOES? Proof: Each factor is identifiable by experimentation.

17. U3 U2 Z X U1 Y GRAPHICAL CRITERION FOR COUNTERFACTUAL INDEPENDENCE U3 U3 U2 U2 X Z Z X U1 Y U1 Y

18. GRAPHICAL CONDITION FOR NONEXPERIMENTAL IDENTIFICATION OF AVERAGE NATURAL DIRECT EFFECTS • Identification conditions • There exists a W such that (YZ | W)GXZ • There exist additional covariates that render all • counterfactual terms identifiable.

19. IDENTIFICATION IN MARKOVIAN MODELS Corollary 3: The average natural direct effect in Markovian models is identifiable from nonexperimental data, and it is given by where S stands for all parents of X (or another sufficient set). Example: S =  X Z Y

20. POLICY QUESTION ANSWERED BY NATURAL DIRECT EFFECT Drug X W Headache Z Aspirin Y Outcome • How effective would the drug be if we eliminate its • side-effect (Headache)?

21. POLICY-BASED INTERPRETATION OF INDIRECT EFFECTS (Advertisement Budget) X Z (Competitor’s Budget) Y (Sales) • NIE(x,x*;Y) = Expected increase in sales, if we bluff the competitor into believing that X is about to change from x* to x. • For Markovian models:

22. RELATIONS BETWEEN TOTAL, DIRECT, AND INDIRECT EFFECTS Theorem 5: The total, direct and indirect effects obey The following equality In words, the total effect (on Y) associated with the transition from x* to x is equal to the difference between the direct effect associated with this transition and the indirect effect associated with the reverse transition, from x to x*.

23. x* z* = Zx*(u) GENERAL PATH-SPECIFIC EFFECTS (Def.) X X W Z W Z Y Y Form a new model, , specific to active subgraph g Definition: g-specific effect Nonidentifiable even in Markovian models

24. SUMMARY OF RESULTS • New formulation of path-specific effects, based on signal blocking, instead of value fixing. • Path-analytic techniques extended to nonlinear and nonparametric models. • Conditions for estimating direct and indirect effects from experimental and nonexperimental data. • Estimability conditions hold in Markovian models. • Graphical techniques of inferring effects of nonstandard policies, involving signal blocking.