Making Causal Inferences and Ruling out Rival Explanations

1. Making Causal Inferences and Ruling out Rival Explanations 29 February

2. Questions? • How do we know that X is causing Y? • Did X have any effect on Y? • If X had not happened would Y have changed anyway?

3. Hypothesized relationship: %Women elected in National Parliaments Party rules gender quotas

4. Questions? • How do we know that party quotas causing changes in %women elected? • Standard Design • Party adopts quotas  % women elected X O Where X = treatment and O = observation

5. Establishing Causation: • Co-variation • Time – (x occurs before y) • Consistent with other evidence • Rule out rival explanations • Example – spurious relationship

6. Spurious Relationship a relationship in which two variables that are not causally linked appear to be so because a third variables in influencing both of them

7. + # of fire engines responding to call Fire damage in $ + + Intensity of fire Spurious Relationship (the third variable problem)

8. Alternative explanations: Electoral System Political Culture %Women elected in National Parliaments Women’s Labor Force Participation Party rules - quotas Access to educational opportunities Women’s Political Resources % of women candidates standing for election

9. + Party quotas %women elected + + Political culture Spurious Relationship

10. When choosing a research design? • When and how to make observations: • Internal Validity • Ability to establish causality • External Validity • Ability to generalize

11. Types of Designs: • Experimental designs • Control groups • Quasi-experimental • Non-experimental designs • Statistical controls

12. Experiments come in a wide variety of apparent types but all share three basic characteristics: • Random assignment • Manipulation of an independent variable • Control over other potential sources of systematic variance X O1 R O2

13. These basic characteristics effectively solve the two basic problems in nonexperimental (correlational) research: • The directionality problem • The third variable problem

14. Random Assignment Random assignment means that assignment to experimental conditions is determined by chance. Participants have a equal probabilities of being assigned to a treatment or control group. This insures that any pre-existing characteristics that participants bring with them to the study are distributed equally among the experimental groups . . . in the long run. Treatment group = (equivalent to) Control group

15. Think about example of party quotas a % women elected: Randomly assign countries to two groups: treatment and control Theoretically should end up with two groups that have equivalent distributions on all other “third variables” (i.e. culture, % women in labour force, etc.) Have one group adopt quotas Observe % women elected, treatment group expected to have higher average for % women elected.

16. Problems? • Random assignment might be difficult in this case. • Turn to quasi-experiments when randomization not possible

17. To Review - One Group Post-Test Only Design • X O • The simplest and the weakest possible design: • Lack of a pretest prevents assessment of change • Lack of a control group prevents threats from being ruled out.

18. Threats to Internal Validity • Selection Threats • Maturation • History • Testing • Instrumentation • Regression • Note: Experimental designs control for these Party adopts quotas  % women elected X O

19. One Group Post-Test Only Design X O Without changing the basic nature of this design, it can be improved considerably by adding additional outcome measures: O1 X O2 O3Compared to norms or expectations, only O2 should be unusual.

20. Post-Test Only Design with Nonequivalent Groups • X O • O • Threats: • Selection

21. One-Group Pretest Post-Test Design • O X O • This very common applied design is susceptible to all threats to within-groups comparisons: • History • Maturation • Testing • Regression • Instrumentation

22. One-Group Pretest Post-Test Design O X O One powerful modification is to add pretests: O O O OO X O Maturation threats can now be examined and their influence separated from treatment effects.

23. O O O O O O O O O X O

24. Untreated Control Group Design with Pretest and Posttest O1 X O2 O1 O2 Can compare change within groups and across groups Expect change in treatment group to be greater Selection still a threat

25. Conclusions: • Experiments best for internal validity • May not be good on external validity • In non-experimental designs, use statistical controls (hold constant all possible “third” variables.