Improving Development Results Through Excellence in Evaluation

1. Improving Development Results Through Excellence in Evaluation Vinod Thomas, Director-General Evaluation Executive Directors’ Induction December 11, 2008

2. Origins of evaluation at the World Bank Group “All large organizations,…if they’re honest with themselves, will admit that they have successes and failures. We should learn from these experiences of success and failure.” -- Robert S. McNamara, World Bank Operations Evaluation Department, The First 30 Years

3. Results: Poverty trends Source: Chen, Shaohua and Martin Ravallion. (2008). The Developing World Is Poorer Than We Thought, But No Less Successful In The Fight Against Poverty, Policy Research Working Paper, Washington, DC: World Bank.

4. Current set up for evaluation and control Executive Directors Independent Evaluation Group Inspection Panel Self Evaluation Monitoring (QAG, DOTS, AS) World Bank Group Management Compliance Ombudsman (IFC, MIGA) Internal Audit Institutional Integrity

5. IEG’s clientele and stakeholders • Client • The Board and CODE • Stakeholders • World Bank Group Management • World Bank Group staff • Development partners • MDBs, ECG, DAC, UNEG, NONIE, Bilaterals partners (NORAD, SDC, DFID) • Countries • Broader development community (NGOs, Academia, think tanks) • Governments (Evaluation Capacity Development)

6. IEG addresses both accountability and learning • Accountability • Is the Bank Group doing what it says? Is it achieving its objectives? Contributing to country outcomes? With what results? • How is the institution building on project and CAS results frameworks • Learning • What projects and programs achieve results – relevance, efficacy, and efficiency – and why? • How can M&E be strengthened in the Bank and in client countries?

7. IEG is on the front line of a One Bank • DGE oversees evaluation work on public & private activities for the 3 entities • IEG’s Leadership Team is fully integrated • An increasing number of evaluations are done jointly across 3 units (e.g., Guarantees, Environment) • We are seeking a greater integration of budgets and programs • IEG moved to a common space (October 2008)

8. Products meet diverse client needs and purposes IEG Products

9. 1. Projects: World Bank project performance • Development outcomes of lending have improved • Over-optimism impairs managing for results • Securing strong development outcomes at the country level is tougher • Improved focus on results requires better M&E, simpler results frameworks, and stronger evaluation knowledge IEG Staff calculations, IEG Database as of December 5, 2008

10. IFC’s project performance • Improvements, with no apparent trade off between development results and IFC profitability • Better outcomes are distinguished by better work quality • Strong attention needed to work quality, portfolio risk and M&E as IFC continues to grow and decentralize • Additionality should be clearly mapped out in IFC’s strategies and measurement IEG staff calculations, XPSR database

11. MIGA’s project performance • MIGA has been effective in facilitating the flow of FDI to high risk sectors and countries • Constrained by a Convention which needs changing • MIGA needs to focus more on quality at entry and • MIGA needs to strengthen alignment of business with development impact • MIGA must scale up its ability to assess and report on results, with self-evaluation for learning and accountability IEG staff calculations

12. 2. Country evaluations: Parallel Country Evaluations • Ukraine • World Bank strategy should strive for greater selectivity—Priorities for governance and institutional development • IFC’s operations should increase coordination with the World Bank to address privatization challenges and replicate the design of successful advisory services • Indonesia • To address persisting poverty, the World Bank should help improve conditions for SMEs to grow and increase focus on good governance, infrastructure, and education. • IFC should play a stronger role in infrastructure and capital market development and strengthen the environmental supervision

13. 3. Thematic: Environment • Findings • The WBG has made progress since 1990 as an advocate for the environment • But treatment of environmental issues in many WBG country programs remains weak due to major external and internal constraints • Recommendations • Fully integrate environmental sustainability into strategy and assistance programs • Support rigorous M&E of environmental impacts • Improve coordination among the Bank, IFC, MIGA and external partners

14. Guarantees • Findings • Guarantees have facilitated the flow of investment to high-risk sectors and countries • Overlap and competition between WBG agencies confuses clients • Weakness in the delivery of political risk mitigation instruments constrains their deployment • Recommendations • Improve the current system or adopt an alternative system across the WBG • Introduce greater flexibility of guarantee instruments in responding to client needs and market developments

15. 4. Corporate: Doing Business Indicators • Findings • DBI have highlighted the issue of regulatory burden • They do not capture major aspects of business climate nor any benefits of regulation • Hence DBI ought not to be seen as a measure of country reform • Lessons & Recommendations • Don’t overstate the implications of the rankings • Improve the credibility of the indicators through data disclosure and transparency • Make reform analysis meaningful by making clear the partial and limited scope of coverage

16. 5. Global: Global Public Goods • Findings • Meeting the shared global challenges will be crucial to reducing poverty • The Bank’s role, particularly where global and local benefits diverge, is crucial • Recommendations • Provide better incentives to provide global public goods at the country level • Require clearer organizational arrangements to link country, regional, and global initiatives • Give greater voice to developing countries in the governance of global programs

17. 6. Quick Evaluation Brief: Past financial crises • Focus on growth and poverty from the outset • Emphasize not only speed & volume, but also quality of support • Seek selectivity, flexibility and adaptability of resources and response • Organize for early warning and preparedness • Stress coordination among partners • Focus on monitoring, evaluation and reporting • Going forward, build in the environment and climate change

18. Four challenges for evaluation’s effectiveness • Methodological: attribution and joint frameworks • Relevance under rapid change: forward looking results • Impact on results: follow-up by Board, Management • Timely knowledge sharing: internal and external

19. Forthcoming reports

20. Thank You! Improving Development Results Through Excellence in Evaluation http://www.worldbank.org/ieg/ http://www.ifc.org/ieg/ http://www.miga.org/ieg/