1 / 18

Experimental Approaches

Experimental Approaches. Lecture 24. Today’s plan. Experiments in economics. A rarity, but providing important ‘thought’ experiments for quasi-experiments. How to estimate with experimental data for full and partial compliance. Quasi-experiments

malha
Download Presentation

Experimental Approaches

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Experimental Approaches Lecture 24

  2. Today’s plan • Experiments in economics. • A rarity, but providing important ‘thought’ experiments for quasi-experiments. • How to estimate with experimental data for full and partial compliance. • Quasi-experiments • Heterogeneous populations and the local average treatment effect (LATE) • LATE and random variation in the number of kids you may have.

  3. Experiments • Random controlled experiments: • select subjects from population of interest • random assignment to either treatment or control group • either receive or not receive treatment • causal effect is expected effect on outcome of interest. • Treatment has to be assigned randomly • Treatment is then distributed independently of any other determinants of the outcome. Eliminated omitted variable bias.

  4. Experiments (2) • Estimate effect of some random assignment (e.g. training program): Yi = a + bXi + ui • Where Xi is a binary variable: • Xi = 1 if treatment • Xi = 0 if no treatment • Causal effect of treatment level x is difference of conditional expectations: E(Y|X = x) - E(Y|X = 0)

  5. Experiments (3) • Because of random assignment: E(ui|Xi) = 0 • b^ is the difference estimator. • Problems with experiments: two types: 1) internal validity; 2) external validity. • Internal validity examines the threat to inference and understanding the causal effects for the population under study • External validity concerns the generalizing of the experimental result.

  6. Problems with Experiments • Internal Validity: • Failure to randomize • Failure to follow treatment • Attrition • Experimental (Hawthorne) effects • Small samples • External Validity: • Non-representative sample • Non-representative program or policy • General equilibrium effects • Treatment vs eligibility

  7. Regression estimators • Simple estimator (Yi = a + bXi + ui) might not be the most consistent, unbiased, or efficient. • Differences estimator with additional regressors: Yi = a + bXi + g1W1i + … + gjWji + ui • If all OLS assumptions hold, then b^ is BLUE. • Can still have a consistent estimate of b^ if u and W’s are correlated, but providing no correlation with X. Called conditional weak dependence. • Estimator is: • more efficient • provides a check on randomization • adjusts for conditional randomization

  8. Regression estimators (2) • Difference in difference estimator • Experimental data often has before and after (panel) nature. • Difference in difference exploits this property to allow for differences in pre-treatment differences! • T = treatment; C = control; difference in difference estimator: b^(diff-in-diff) = (meanYT,after - meanYT,before) - (meanYC,after - meanYC,before) also: = DmeanYT - DmeanYC

  9. Regression estimators (3) • As a regression: DYi = a + bXi + ui • Where DYi (for each i before and after the treatment). • Reasons for using this approach: • Removes time invariant effects • Eliminates pre-treatment differences in Y. • Can use difference in difference with additional regressors: DYi = a + bXi + g1W1i + … + gjWji + ui • Same arguments apply as before. • Useful estimation tool

  10. Regression estimators (4) • Useful estimation tool. Has a number of possibilities: • if multiple time period observations exist, can use panel methods • estimate causal effects for different groups • estimate when there is only partial compliance • Partial compliance: use assigned treatment level as an instrumental variable for the actual (observed). • Providing assigned treatment level is truly random, it can act as an instrument.

  11. Regression estimators (5) • Test for Random Assignment: • X (variable indicating treatment) should be uncorrelated with other observables. Run regression of X on W’s and undertake F - test. • Alternative: test initial assignment for treatment (Z) on observables. Once again an F - test.

  12. Quasi-experiments • Economic experiments expensive and unethical (?) (e.g. SIME/DIME and STAR) . • Economics has taken to using quasi experiments (also called natural experiments). • Quasi - experiments provide an ‘as if’ random assignment of individuals into a treatment and control groupings. No experiment actually took place; BUT some event, law, policy, or rule (social, behaviorial) places some individuals in a ‘treatment’ set and some in a ‘control’ set. • Need random variation of some description. • Relies heavily on the idea that the partial compliance can be instrumented with initial assignment.

  13. Quasi-experiments (2) • Same problems as before. • Internal Validity: • Failure to randomize • Failure to follow treatment • Attrition • Experimental (Hawthorne) effects • Instrument validity • External Validity: more of a judgmental consideration: are we examining a special case?

  14. Heterogeneous populations • In a situation where there is unobserved variation in the population (heterogeneous), there will be a response to treatment for each i. • Hence estimation becomes: Yi = a + biXi + ui. • Note that b depends on i. • If individuals selected at random, it follows that bi should also be a random variable. • When population is heterogeneous, then the causal effect observed is the average causal effect, or local average treatment effect (LATE).

  15. LATE • LATE (OLS estimated) identifies only the average effect for individuals under ‘as if’ assignment. • LATE (IV estimated) identifies only the weighted average of the causal effects for those who reacted greatest to the treatment. • In other words, the instrument identifies a weighted average of the causal effect, where those for whom the instrument is most influential receive the most weight.

  16. LATE and Angrist & Evans • Understanding instrumental variables is important in economics because of recent use of local average treatment effects (LATE) in applied literature • Example: Angrist and Evans AER (1998) • tested for the effect of having children on hours worked • looked at random instruments for a treatment and control group • Number of kids and propensity to have another child • testing to see if sex of the first two affects the likelihood of having a third

  17. LATE and Angrist & Evans (2) • Random assignment of treatment and control groups • can’t determine sex of children • assumption: number of kids influences hours of work because the more kids you have, the less time time you have to work • Identification comes off small part of sample: need very large sample size

  18. LATE and Angrist & Evans (3) • Estimated the following equation:Hi = a1 +b1Xi + b2NKi + I • here they identifying on the number of kids (NKi) • Why can we expect NKi to be endogenous? • Might choose combination of number of kids and hours of work, but can’t choose sex of children • So Angrist and Evans used the same sex variable as the identifying variable: NKi = g1 + g2Xi + g3samesexi + vi • samesexiis a dummy variable equal to 1 if the children are the same sex and 0 otherwise • Negative result: more kids leads to less hours of work • L24.xls has a worked example

More Related