1 / 17

AEA meetings 2010 Jennie Litvack and Victoria Monchuk IEG World Bank

Complementary Approaches to Evaluating Social Safety Nets at the World Bank: Evaluation questions and approaches. AEA meetings 2010 Jennie Litvack and Victoria Monchuk IEG World Bank. What are Social Safety Nets (SSNs)?.

imala
Download Presentation

AEA meetings 2010 Jennie Litvack and Victoria Monchuk IEG World Bank

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Complementary Approaches to Evaluating Social Safety Nets at the World Bank:Evaluation questions and approaches AEA meetings 2010 Jennie Litvack and Victoria Monchuk IEG World Bank

  2. What are Social Safety Nets (SSNs)? • Non-contributory transfers, targeted, in some way, to the poor and vulnerable • Such as: • Unconditional or conditional cash transfer programs • Food and in-kind transfers • Food and energy subsidies • School feeding programs • Scholarships • Public works programs • Non-contributory /social pensions

  3. What functions do Safety Nets serve? Protect the poor and vulnerable from shocks-- individual or systemic shocks Redistribute resources by increasing the consumption of the poorest in society and reduce chronic poverty and inequality Improve investments in human capital by the poor Protect the poor and vulnerable from negative effects that can accompany macro reforms

  4. Safety Nets and the World Bank 1980s: Bank started supporting SSNs – structural reform 1990s: Transition to market economies of former Soviet Union 1998: Asia Financial Crisis 1999-2000: Debt crisis in Latin America, Turkey 2007-10: Food, fuel and financial crisis

  5. Safety net lending has increased

  6. Safety nets in all regions

  7. Evaluation purpose Accountability: $13 billion spent • Assess how effectively the Bank has helped countries develop SSNs to protect the poor and vulnerable and mitigate risk Learning: New strategy in preparation • What lessons can be learned for informing future support

  8. Evaluation questions • How effective, relevant and sustainable have Bank supported projects been in LICs and MICs? • How relevant and effective has overall Bank assistance been in helping countries establish sound safety nets? • How can the effectiveness and relevance of Bank support be explained and understood? • What lessons can be derived for future support?

  9. Evaluation approach • Large and complex evaluation scope • World Bank thematic area • 2000-2010 • Multiple evaluation questions • Describe what Bank is doing • Assess how relevant and effective it is • Assess what final results have been produced • Understand why things work, why not, how improve? Need to develop evaluation building blocks

  10. Evaluation building blocks • SSN portfolio reviews Lending, Trust Funds, Analytical work and Technical Assistance • 244 lending projects coded and analyzed • Analyzed trends in Bank engagement over time, by: region, country income level, SSN instruments, SSN functions, project results framework, performance, etc. • Also for Trust Funds and analytical and technical work

  11. Evaluation building blocks • Country case studies • 30 case studies analyzed (25 randomly selected desk-based, 5 purposely selected field-based), 3 in-depth Project Performance Assessment Reviews • Looking beyond projects–Bank program in country context • Review CASs, lending projects, analytical work, as well as semi-structured interviews with staff, other donors and clients (in field-based cases) • Examined the relevance and nature of the Bank’s engagement; country’s application of social risk management framework (SP Strategy, 2001); and the results in countries

  12. Evaluation building blocks • Food, fuel and financial crisis (FFF) Staff Survey • On-line survey sent to World Bank staff working on social protection/SSNs in all client countries • Focused on countries' experiences with SSNs in the context of the FFF crises, as well as World Bank SSNs assistance to countries in response to FFF crises

  13. Evaluation building blocks • Impact evaluations of safety nets • Literature Review: 137 impact evaluations reviewed to examined evidence re what works and under which conditions; distribution of benefits; sustainability of benefits; and generalizability of results. Sub-sample of 25 programs supported by the Bank. • Two new rigorous impact evaluations–Colombia and Pakistan–examining longer term impacts of conditional cash transfers (CCTs)

  14. Evaluation building blocks • Background studies • Political economy of SSNs programs • Decentralization and SSNs: challenges and strategies for the Bank • SSNs in low-income countries • A review of Public Works Programs • Development Policy Loans for Economic Reforms: Bank experience with SSNs to mitigate distributional impacts on the poor • World Bank matrix structure and how it affects assistance for SSNs

  15. Matrix mapping each evaluation question with building blocks • Relevance • Portfolio reviews (lending, analytical and technical work) • Case studies • Background papers on political economy, public works programs, SSNs in low-income countries, etc. • Application of social risk management strategy, etc. • Effectiveness/understanding effectiveness • Portfolio review (design details, results frameworks, project ratings) • Case studies • Impact evaluations, etc.

  16. The “Matrix”

  17. Main take-aways: Develop building blocks Triangulate to answer questions Evaluation will be available spring 2011 worldbank.org/ieg

More Related