1 / 24

The Future of Monitoring and Evaluation: Lessons from a Decade of Impact Evaluations

The Future of Monitoring and Evaluation: Lessons from a Decade of Impact Evaluations. Eric Foster-Moore The World Bank | Africa Water Resources efostermoore@worldbank.org. Our project will directly benefit 400,000 people … Our project will benefit the people of Mozambique …

eze
Download Presentation

The Future of Monitoring and Evaluation: Lessons from a Decade of Impact Evaluations

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Future of Monitoring and Evaluation: Lessons from a Decade of Impact Evaluations Eric Foster-Moore The World Bank | Africa Water Resources efostermoore@worldbank.org

  2. Our project will directly benefit 400,000 people… • Our project will benefit the people of Mozambique… • Number of women benefitted • Growth rate in country X increased by 2 percentage points Development works

  3. Jeffrey Sachs Big Push, Millennium Villages Project, The End of Poverty “$195 billion in aid per year for 20 yearswould eliminate poverty.” William Easterly “No it won’t.” The White Man’s Burden Dead Aid by DambisaMoyo Can the West Save Africa? The white band’s burden… Art credit: @ChrisPlanicka

  4. Is aid effective? • How do we know? • How do we apply this to our own work? Three questions

  5. Does aid increase growth? A macro perspective Boone (1996): Nope. Burnside and Dollar (2002). It depends. Effective when combined with “good policy.” Easterly, Levine, and Roodman (2003). Not really.

  6. Original Burnside-Dollar results

  7. Easterly, Levine, and Roodman

  8. Does aid increase growth? A macro perspective Boone (1996): Nope. Burnside and Dollar (2002). It depends. Effective when combined with “good policy.” Easterly, Levine, and Roodman (2003). Not really. Clemens, Radelet, Bhavnani, and Bazzi (2012). It still depends. (Chris Blattman)

  9. Often just make them up: “This project will directly benefit 300,000 people.” • Move beyond correlations. Establish causality. Do an experiment. Some are natural, some are contrived • Natural experiment: half of a community displaced by a reservoir • Contrived experiment: the Fiala (2013) paper What about at the micro level?

  10. Grants, loans, training, or a combo? Fiala (2013): Stimulating Microenterprise Growth: Results from a Loans, Grants, and Training Experiment in Uganda

  11. Loans • Loans plus training • Cash • Cash plus training • Control Treatment groups: Follow up surveys after 6 and 9 months.

  12. Loan only: effect is gone after nine months • Loans and training: 54 percent increase in profits for men • Grant only: no effect whatsoever. For men, Fiala speculates, “Knowing that the loan had to be repaid appears to have led them to use the money more effectively in the business.” • Grants and training: no effect • Women: no positive effects. “Family pressure on women appears to have significant negative effects on business investment decisions: married women with family nearby perform worse than the control group in a number of the interventions.” Results

  13. What is the effect of institutions on economic performance? Acemoglu, Johnson, and Robinson (2000): The Colonial Origins of Comparative Development

  14. Instrumental variable Strong economic performance Strong institutions Methods Other stuff

  15. Hypothesis

  16. “Differences in institutions account for roughly three quarters of the differences in income per capita.” • Not driven by outliers such as the United States, Canada, Australia, etc. • Not driven by simply being on the African continent. • Not driven by distance from the equator (i.e. geography). Results

  17. Compare program alternatives • Quantify effects of program X in place Z at time T • Force you to identify and test assumptions • Microfinance, cash transfers, education, health, job training, etc. What RCTs do really well:

  18. What you see: • Policy and legal frameworks strengthened • Cash collection ratio; debts repaid • Number of people in project area with access to “Improved Water Sources” • Percent of biological samples failing • Number of direct beneficiaries, percent of which are female • Number of staff trained What you don’t see: • Household income • Business profitability • Governance • Strength of political parties • Comparison to other interventions • Opportunity cost • Articulation of assumptions Monitoring and evaluation: some examples

  19. There’s an entire sector of projects that don’t get tested and where assumptions that are probably decades old dominate. Let’s bring the lessons of impact evaluation to these projects. • Need to be strategic about their application in order to learn something about human behavior. Two problems with RCTs

  20. Is aid effective? • How do we know? • How do we apply this to our own work? Three questions

  21. Backup

  22. Solow model of growth

  23. From E-L-R (2003)

  24. From AJR

More Related