1 / 12

MEASURING IMPACT

MEASURING IMPACT. Impact Evaluation Methods for Policy Makers.

thu
Download Presentation

MEASURING IMPACT

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MEASURING IMPACT Impact Evaluation Methods for Policy Makers This material constitutes supporting material for the "Impact Evaluation in Practice" book. This additional material is made freely but please acknowledge its use as follows: Gertler, P. J.; Martinez, S., Premand, P., Rawlings, L. B. and Christel M. J. Vermeersch, 2010, Impact Evaluation in Practice: Ancillary Material, The World Bank, Washington DC (www.worldbank.org/ieinpractice). The content of this presentation reflects the views of the authors and not necessarily those of the World Bank.

  2. 1 Counterfactuals False Counterfactuals Before & After (Pre & Post) Enrolled & Not Enrolled (Apples & Oranges) Causal Inference

  3. 2 Randomized Assignment Randomized Offering/Promotion Discontinuity Design Difference-in-Differences Diff-in-Diff IE MethodsToolbox Matching P-Score matching

  4. 2 Randomized Assignment Randomized Offering/Promotion Discontinuity Design Difference-in-Differences Diff-in-Diff IE MethodsToolbox Matching P-Score matching

  5. Difference-in-differences (Diff-in-diff) Y=Girls exam score (percentage of correct answers) P= Tutoring - - - = 0.11 Diff-in-Diff: Impact=(Yt1-Yt0)-(Yc1-Yc0)

  6. Difference-in-differences (Diff-in-diff) Y=Girls exam score (percentage of correct answers) P= Tutoring - - - = 0.11 Diff-in-Diff: Impact=(Yt1-Yc1)-(Yt0-Yc0)

  7. Impact =(A-B)-(C-D)=(A-C)-(B-D) Not enrolled C=0.81 Exam score D=0.78 A=0.74 Enrolled Impact=0.11 B=0.60 Time T=0 Before T=1 After

  8. Impact =(A-B)-(C-D)=(A-C)-(B-D) Not enrolled C=0.81 Exam score D=0.78 A=0.74 Impact<0.11 Enrolled B=0.60 Time T=0 Before T=1 After

  9. Case 6: Difference-in-differences Note: If the effect is statistically significant at the 1% significance level, we label the estimated impact with 2 stars (**).

  10. Progresa Policy Recommendation? Note: If the effect is statistically significant at the 1% significance level, we label the estimated impact with 2 stars (**).

  11. ! Keep in Mind Difference-in-Differences • To test this, at least 3 observations in time are needed: • 2 observations before • 1 observation after. Difference-in-Differences combines Enrolled & Not Enrolled with Before & After. Slope: Generate counterfactual for change in outcome Trends –slopes- are the same in treatments and comparisons (Fundamental assumption).

More Related