Download
1 / 37
380 likes | 534 Views
Treatment Evaluation. Identification. Graduate and professional economics mainly concerned with identification in empirical work. Concept of understanding what is the causal relationship behind empirical results. Selection Bias. Example 1: Do hospitals make people healthier?. Selection Bias.
E N D
Identification • Graduate and professional economics mainly concerned with identification in empirical work. • Concept of understanding what is the causal relationship behind empirical results.
Selection Bias Example 1: • Do hospitals make people healthier?
Selection Bias National Health Interview Survey (NHIS) • “During the past 12 months, was the respondent a patient in a hospital overnight?” • “Would you say your health in general is excellent, very good, good, fair, poor?”(1 is excellent; 5 is poor)
Selection Bias • Going to the hospital makes people sicker • It’s not impossible: hospitals are full of other sick people who might infect us, and dangerous machines and chemicals that might hurt us.
Selection Bias • People who go to the hospital are probably less healthy to begin with. • Even after hospitalization, people who have sought medical care are not as healthy, on average, as those who never get hospitalized. • Though they may well be better after hospitalization than they otherwise would have been.
Selection Bias Example 2: • Does college education increase wage?
Selection Bias • College graduates earn 84% more than high school graduates.
Selection Bias • Selection into college: higher ability, smarter, work harder, etc. • College graduates would have earned more even without college education. • Simple comparison can not identify the causal impact of college education on wage.
Solution 1: Randomization • Random assignment makes the treatment independent of potential outcomes. • It eliminates selection bias and reveals true treatment effect. • Treatment effect: compare post-treatment outcome between those who get the treatment and those who don’t.
Solution 1: Randomization • Example 1: hormone replacement therapy (HRT) • Recommended for middle-aged women to reduce menopausal symptoms.
Solution 1: Randomization • Nurses Health Study (non-experimental survey of nurses): better health among the HRT users. • Randomized trial: few benefits; serious side effects(see, e.g., Women’s Health Initiative [WHI], Hsia, et al., 2006).
Solution 1: Randomization • Example 2: government-subsidized training programs. • Provide a combination of classroom instruction and on-the-job training for groups of disadvantaged workers such as the long-term unemployed, drug addicts, and ex-offenders. • Aim: increase employment and earning.
Solution 1: Randomization • Non-experimental studies: trainees earn less than comparison groups (see, e.g., Ashenfelter, 1978; Ashenfelter and Card, 1985; Lalonde 1995). • Evidence from randomized evaluations of training programs generate mostly positive effects (see, e.g., Lalonde, 1986; Orr, et al, 1996).
Solution 1: Randomization Problems of randomization: • Randomly offers, but people don’t want to be part of the game • High costs • Small sample size
Solution 2: Difference-in-difference • Panel data available
Solution 2: Difference-in-difference • New problem: time trend • Compare change in outcomes between treatment group and control group • Impact is the difference in the change in outcomeImpact = (Yt1-Yt0) - (Yc1-Yc0)
Solution 2: Difference-in-difference Pre Post
Solution 2: Difference-in-difference Effect of program using only pre- & post- data from T group (ignoring general time trend). Pre Post
Solution 2: Difference-in-difference Effect of program using only T & C comparison from post-intervention (ignoring pre-existing differences between T & C groups). Pre Post
Solution 2: Difference-in-difference • Whatever happened to the control group over time is what would have happened to the treatment group in the absence of the program. Effect of program difference-in-difference (taking into account pre-existing differences between T & C and general time trend). Pre Post
Solution 2: Difference-in-difference • Example: Schooling and labor market consequences of school construction in Indonesia: evidence from an unusual policy experiment Esther Duflo, MITAmerican Economic Review, Sept 2001
Solution 2: Difference-in-difference School infrastructure Educational achievement? Educational achievement Salary level?
Solution 2: Difference-in-difference • 1973-1978: The Indonesian government built 61,000 schools equivalent to one school per 500 children between 5 and 14 years old • The enrollment rate increased from 69% to 85% between 1973 and 1978 • The number of schools built in each region depended on the number of children out of school in those regions in 1972, before the start of the program.
Solution 2: Difference-in-difference • 2 sources of variations in the intensity of the program for a given individual • By region:simplify the intensity of the program: high or low • By age:Young cohort of children who benefittedOlder cohort of children who did not benefit
Solution 2: Difference-in-difference • Fundamental assumption that trends (slopes) are the same in treatments and controls (sometimes true, sometimes not)
EstimatedAverage Treatment Effect Treatment Group Control Group Outcome Average Treatment Effect Time Treatment
Solution 2: Difference-in-difference • Need a minimum of three points in time (age of cohort in the example) to verify this and estimate treatment (two pre-intervention)
Average Treatment Effect Outcome Treatment Group Third observation Control Group Second observation First observation Time Treatment
Solution 3: Matching • Panel data NOT available • Controls: non-participants with same characteristics as participants • The matches are selected on the basis of similarities in observed characteristics
Solution 3: Matching • Instead of aiming to ensure that the matched control for each participant has exactly the same value of X, same result can be achieved by matching on the probability of participation
Solution 3: Matching • For each participant find a sample of non-participants that have similar propensity scores (prob. of treatment) • Compare the outcome
Solution 3: Matching • Common support
Solution 3: Matching • Assumes no selection bias based on unobserved characteristics
