Development Effectiveness at the World Bank

1. Development Effectiveness at the World Bank “...we will not measure our performance by dollars lent or projects approved, but by our development impact -- results on the ground. I cannot overstate the importance of this change.” James D. WolfensohnAnnual Meeting SpeechOctober 1996 ODI 04/00

2. The Development Environment Has Evolved Before Now Ascent of GlobalCapital Market Money Bank Full Service Bank Planned Development ParticipatoryDevelopment Greater Pluralism Knowledge Managementand Partnerships End of Ideological Divide Conditionality Advent of Civil Society DevelopmentEffectiveness Resource Transfer

3. Fifties Sixties Seventies Eighties Nineties Reconstruction Growth Basic Needs Adjustment Capacity Building The Very Concept of Development Has Changed Physical Social Human Natural DevelopmentDecades Major Objective Main Instruments Dominant Discipline Technical Assistance Projects Sector Investment Adjustment Loans Country Assistance Strategy Engineering Finance Planning Neoclassical Multi-disciplinary

4. World Bank Performance Criteria Reflect the New Development Agenda: Environmentally Socially and Sustainable Growth Governance: social development, economic management, public sector reform, etc. Poverty/gender population, health, nutrition Private sector development Environment Rural development Finance Education Energy/mining Infrastructure Social protection Human economy Physical economy Market economy

5. The Meaning of Development Effectiveness Has Changed Too Development Effectiveness connotes a demonstrable and efficient contribution to economically sound, socially responsible and environmentally sustainable growth

6. OED methods are best practice According to two major reviews of the MDB’s evaluation methods and practices: • Bank implementation of evaluation criteria are closest to the agreed criteria of all MDBs (according to Wyss report Harmonization of Evaluation Criteria Among the MDBs, 1999) • The Bank is a leader in completion reporting and performance review among all MDBs (according to Eriksson report Comparison of MDB Completion Reporting and Performance Review, 1996)

7. How is Development Effectiveness Assessed? Results 1.Outcome 2. Sustainability 3. Institutional development impact At the project level, OED assesses five main criteria of development effectiveness: Country conditions, exogenous factors Inputs 4. Bank performance 5. Borrower performance

8. The outcome assessment Definition: the extent to which a project has achieved or is expected to achieve its major relevant objectives efficiently; rate of return threshold of 10% used when possible “DOING THE RIGHT THINGS” Relevance: Are objectives responsive to country needs and Bank priorities? “DOING THINGS RIGHT” Efficacy: Are we achieving objectives shared by the Bank and its partners? Efficiency: Are we using Bank and partner resources economically? • 6-point scale: • highly satisfactory • satisfactory • marginally satisfactory • marginally unsatisfactory • unsatisfactory • highly unsatisfactory

9. The sustainability assessment Definition: The resilience to risk of net benefits flows over the long term • Evaluation of 9 criteria on the risks to project net benefits over time: • technical • environmental • financial • government ownership • economic • stakeholder ownership • social conditions • institutional support • resilience to exogenous factors • 3-point scale: • likely • uncertain • unlikely

10. The institutional development impact assessment Definition: the improvement in a country’s ability to make effective use of its human, financial, and natural resources • Evaluation of 2 basic criteria: • capacity development of an organization • improving institutional arrangements (i.e., “rules of the game” such as the regulatory framework) • 3-point scale: • substantial • modest • negligible

11. What is the unit of account? • Investment projects • Sector investment programs • Adjustment loans • Non-lending services • Country assistance strategies (CAS) • Sector strategies

12. ARDE Regions AFR EAP SAS ECA MNA LAC Networks HDD ESSD FPSI PREM Country Assistance Strategy and Compacts Sector Assistance Strategies and Thematic Progress Reports Nonlending Services Nonlending Services Lending Services Lending Services Aid Coordination and Cofinancing Reviews Activity Completion Reports on Nonlending Services Performance Audit Reports and Impact Evaluations Portfolio Improvement Program Reviews Research Activity Completion Reports Imple. Completion Reports Sector Impact Eval. Trust Funds Completion Memos Supervision Reports Mid-term Reviews Networks/ OED Regions/ QAG/OED Regions/ OED DEC/ Networks Regions/ OPR Regions/ OPR Regions/ OPR Regions/Networks OED QAG OED

13. Evaluation in The World Bank • OED is independent: it reports to the Board of Executive Directors • 5,000 projects have been evaluated since OED’s creation in 1970 (644 operations, worth $52 billion in real terms, in FY98 -FY99 alone) • The function spans a variety of sectors (agriculture, energy, finance, transport, water supply, urban development) • It evaluates projects, programs, processes, and policies

14. Project Performance Measurement System OED • consistent • objective • continuous • professional • credibile • coherent(but need further harmonization) Quality at exit Monitoring during implementation Quality at entry QAG QAG

15. Real Time Evaluation Means Quality Assurance Two Complementary Components • Portfolio Improvement Program • Quality Assessment Program • Quality at Entry • Supervision • Economic and Sector Work

16. It also Means Quality Management • Target managerial attention on “high risk” clusters • Regular monitoring and feedback to staff and managers • Partnering to disseminate lessons of experience • Linking evaluation to staff/managerial evaluations and strategy (a long term goal)

17. QAG - Non-risky portfolio (active) OED - Outcome performance (exiting) 90 90 80 80 Percent non-risky 70 70 Percent satisfactory 60 60 50 50 1994 1995 1996 1997 1998-99* 1997 1998 1999 Exit fiscal year Active fiscal year Where are we? • Quality at exit have been improving over the 90s • Data on the active portfolio promises future improvement • However, borrower implementation performance stagnant

18. Project Outcome by Region (% satisfactory) Exit fiscal year Region 1990-93 1994-97 1998-99* AFR 54 55 60 (+) EAP 80 84 81 (-) ECA 73 72 83 (+) LAC 64 78 82 (+) MNA 70 66 70 (+) SAS 73 67 68 Bankwide 66 68 72 (+)

19. Sustainability has been improving... Sustainability Assessments 60 49 47 50 Exit FY94-97 Exit FY98-99* 37 40 33 30 20 20 14 10 0 % Likely % Uncertain % Unlikely

20. … as well as institutional development impact. Institutional Development Impact Assessments 60 48 46 Exit FY94-97 50 Exit FY98-99* 39 40 31 30 24 20 13 10 0 % Substantial % Modest % Negligible

21. Sustainability (percent likely) ID Impact (percent substantial) AFR AFR EAP EAP ECA ECA LCR LCR MNA MNA SAR SAR 0 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 Exit FY90–93 Exit FY94–97 Exit FY98–99* Regional Comparisons * Results are preliminary for FY99 exits (43 percent coverage).

22. Project Outcome by Network (% satisfactory) Exit fiscal year Network 1990-93 1994-97 1998-99* ESSD 59 68 63 (-) FPSI 68 65 72 (+) HDN 78 72 75 (+) PREM 66 73 78 (+) Bankwide 66 68 72 (+)

23. Sustainability (percent likely) ID Impact (percent substantial) ESSD ESSD FPSI FPSI HDN HDN PREM PREM 0 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 Exit FY90–93 Exit FY94–97 Exit FY98–99* Network Comparisons * Results are preliminary for FY99 exits (43 percent coverage).