1 / 28

Evaluation at the World Bank

Evaluation at the World Bank. “...we will not measure our performance by dollars lent or projects approved, but by our development impact -- results on the ground. I cannot overstate the importance of this change.” James D. Wolfensohn Annual Meeting Speech October 1996. Oslo 10/14/99.

zaza
Download Presentation

Evaluation at the World Bank

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluation at the World Bank “...we will not measure our performance by dollars lent or projects approved, but by our development impact -- results on the ground. I cannot overstate the importance of this change.” James D. WolfensohnAnnual Meeting SpeechOctober 1996 Oslo 10/14/99

  2. The World Bank and Development • A global development finance institution • 179 country members • $25-30 billion annual borrowings • The largest single provider of development assistance • over 1800 projects under implementation • (worth more than $140 billion) • 250-300 new loans/credits a year • (worth $20-25 billion)* * Excluding financial crisis loans (about $8–9 billion)

  3. The World Bank and Development • A far-flung development network • 83 field offices • 72 aid groups • 800 trust funds • A Bank of ideas and skills • A significant research and knowledge management budget ($99 million) • $126 million worth of advisory and aid coordination services • $34 million of development training (EDI)

  4. Evaluation in The World Bank • OED is independent: it reports to the Board of Executive Directors • 5,000 projects have been evaluated since OED’s creation in 1970 (325 operations, worth $26 billion, in FY96 -FY97 alone) • The function spans a variety of sectors (agriculture, energy, finance, transport, water supply, urban development) • It evaluates projects, programs, processes, and policies

  5. The Development Environment Has Evolved Before Now Ascent of GlobalCapital Market Money Bank Full Service Bank Planned Development ParticipatoryDevelopment Greater Pluralism Knowledge Managementand Partnerships End of Ideological Divide Conditionality Advent of Civil Society DevelopmentEffectiveness Resource Transfer

  6. Fifties Sixties Seventies Eighties Nineties Reconstruction Growth Basic Needs Adjustment Capacity Building The Very Concept of Development Has Changed Physical Social Human Natural Development Decades Major Objective Main Instruments Dominant Discipline Technical Assistance Projects Sector Investment Adjustment Loans Country Assistance Strategy Engineering Finance Planning Neoclassical Multi-disciplinary

  7. The Meaning of Development Effectiveness Has Changed Too Development Effectiveness connotes a demonstrable and efficient contribution to economically sound, socially responsible and environmentally sustainable growth

  8. DOING THE RIGHT THINGS Relevance: Are the objectives we seek responsive to country needs and Bank priorities? Institutional development: Are we helping to build the capacity of our partners? DOING THINGS RIGHT Efficacy: Are we achieving objectives shared by the Bank and its partners? Efficiency: Are we using Bank and partners’ resources economically? Development Effectiveness Means: Sustainability: Will the benefits of Bank intervention be sustained?

  9. World Bank Performance Criteria Reflect the New Development Agenda: Environmentally Socially and Sustainable Growth Governance: social development, economic management, public sector reform, etc. Poverty/gender population, health, nutrition Private sector development Environment Rural development Finance Education Energy/mining Infrastructure Social protection Human economy Physical economy Market economy

  10. Performance characteristics Development themes • ECONOMICMacro policySector policy • FINANCIALCorporate financePublic financeCommercial aspects • SOCIALIncome distributionSocial impactResettlement Indigenous groups • INSTITUTIONALOrganization and managementLegalGovernance aspects • ENVIRONMENTALTechnical optionsEngineering optionsChoice of technology • ECONOMIC MGMT. AdjustmentDebtTrade policyFiscal policyPublic expendituresManagement • POVERTY REDUCTION AND HUMAN RESOURCESEducation/trainingPopulationHealth and nutritionGender aspects • PSD/PSMFinancial intermediationEnabling policiesCivil service reform • ENVIRONMENTALNatural resource managementPollutionInfrastructureRegional planning As vehicles for policy and institutional reform, projects are evaluated in terms of the special emphases of the development agenda.

  11. The Evaluation System is Being Adjusted Characteristics of evaluation system Dominant concepts • Project focus • Investment-driven growth • Import substitution • Central planning • Country focus • Structural Adjustment • Outward-oriented policies • Decentralized decision making • Project evaluation • Cost/benefit analysis • Shadow pricing • Self-evaluation • Portfolio evaluation • Policy evaluation • Risk analysis • Participatory evaluation THEN NOW Evaluation methodologies reflect the dominant paradigms of development policy.

  12. Measuring Performance Matters • For accountability: Increasingly, shareholders want to know whether they are getting “value for money.” • For Bank management and staff: To enhance professional excellence and get results on the ground through organizational learning.

  13. Project Performance Measurement System OED • consistent • objective • continuous • professional • external credibility • work in process--e.g., NLS Quality at exit Monitoring during implementation Quality at entry QAG QAG

  14. Real Time Evaluation Means Quality Assurance Two Complementary Components • Portfolio Improvement Program • Quality Assessment Program • Quality at Entry • Supervision • Economic and Sector Work

  15. It also Means Quality Management • Target managerial attention on “high risk” clusters • Regular monitoring and feedback to staff and managers • Partnering to disseminate lessons of experience • Linking portfolio management indicators to staff/managerial evaluations

  16. The Other Main Challenge Is the “Scaling-up” of Evaluation • Country, sector, and thematic evaluations • Adding up project-level results and linking them to network and strategic goals. • Non-lending services, partnerships, and strategic alliances. • Moving towards a results-oriented framework for performance measurement.

  17. Evaluation Gaps Must Be Filled • Today’s evaluation system concentrates on lending • Activity completion reporting is also needed for research, grants, economic and sector work, trust funds, training, etc… • Evaluation must move to a “higher plane”

  18. Country as Unit of Account Country Assistance Agenda Country Assistance Strategy Performance Factors Development Effectiveness Results • Social • Poverty head count, change • Private consumption p.c. growth • Child mortality, change • School enrollment, by gender, change • Bank • Country Dialogue • ESW Quality • Quality at Entry • Supervision Quality • Portfolio Management • Cost Effectiveness • Partnerships • Economic • GDP p.c. growth • Export growth • Debt/GDP • Inflation Thematic Priorities • Country Assistance Strategy Achievements • Relevance • Efficacy • Efficiency • Sustainability • Institutional Development Assessment of Past Projects Development Priorities Development Objectives Operational Programs • Borrower • Fiscal • Monetary • Trade • Size of Government • Quality of PublicExpenditure • Commitment • Environmental • Country-specific measures(e.g. salinity, forest cover),change • Environmental Index (e.g.enforcement of environmentalregulations), change Country’s Policies/Capacities Partners’ Plans/Programs • Private Sector • Development • Private Investment/GDP • Foreign Investment/GDP • Privatizations • Exogenous • Terms of Trade • Interest Rates • Droughts/Floods • War • Capacity Building • Public Sector Management • Governance

  19. Sectors/Themes as Objects of Evaluation Thematic and country programs 3 year sector/thematic evaluation Environ. sustainable development Economic/social governance Human/ social capital Private sector development PSD Finance Industry Telecom. Urban dev. Rural dev. Environ- ment Economic mgmt. PSM Education PHN Soc. policy/ social security Energy Water Transport Annual business plans & retrospectives Operating services programs

  20. The Ultimate Objective Is Transparent Evaluation of Development Effectiveness at Corporate Level… ARDE Regions AFR EAP SAS ECA MNA LAC Networks HDD ESSD FPSI PREM Country Assistance Strategy and Compacts Sector Assistance Strategies and Thematic Progress Reports Nonlending Services Nonlending Services Lending Services Lending Services Aid Coordination and Cofinancing Reviews Activity Completion Reports on Nonlending Services Performance Audit Reports and Impact Evaluations Portfolio Improvement Program Reviews Research Activity Completion Reports Imple. Completion Reports Sector Impact Eval. Trust Funds Completion Memos Supervision Reports Mid-term Reviews Regions/ OPR Regions/ OPR Networks/ OED Regions/ QAG/OED Regions/ OED DEC/ Networks Regions/ OPR Regions/Networks OED QAG OED Proposed

  21. …Combined with Regular Monitoring of Bank Performance (Scorecard) Borrowing spreads Reserves/ loan ratio Investment return Admin. expenses as % of assets Interest coverage ratio • Quality at exit: • Outcomes • Sustainability • Institutional development • Bank performance TRE, RM, FPI TRE, FPI FPI TRE, FPI PBD Client feedback on staff performance OED Regions Financial strength • Quality at entry: • 10 ECON commandments Processing times OED/QAG • Portfolio management quality: • Projects at risk • Realism index • Proactivity index PBD Score-card on Bank performance Client responsiveness Development effectiveness Disbursement rates QAG PBD • Quality of non lending serv.: • Relevance • Efficacy • Efficiency Values and skills Front-line services as % of total admin. cost OED/QAG PBD • Cost effec. indicators: • Completion coefficients • Cost/SY Effectiveness of staff training Staff feedback from surveys Skills assessment PBD Networks LLC ICD

  22. This Vision Implies Partnerships in Evaluation Development effectiveness requires participatory evaluation • Locally: to support management conditions • Globally: to support strategic alliances Thus ensuring proactivity of the overall development system • Global networking among evaluators • Evaluation capacity development • beneficiary involvement in evaluation

  23. NGOs Multilateral Strategic alliances Bilateral Global and Regional PERFORMANCE PARTNERSHIPS Civil society Country Management coalitions Government Donors

  24. Operations Evaluation Department DGO (171/99) Director OED (OEDDR 175/05) Resource Management Operations Advisor Manager Country Evaluations & Regional Relations (OEDCR - 175/20) Manager Sector & Thematic Evaluations (OEDST - 175/10) Manager Partnership & Knowledge Programs (OEDPK - 175/40) Poverty, Gender, Social Development & Social Protection Rural Development Environment and Water Finance and PSD PSM Energy,Mining & Telecommunication Education Knowledge Management HNP Country Economics Urban & Infrastructure ECDP, Partnerships & Training Manager Corporate Evaluations and Methods (OEDCM - 175/50) Transport Evaluation Process and Methods Clusters are Interactive and Flexible ACS Staff assigned to managerial group and cluster = Thematic & Regional Coordinators

  25. Trends in OED Work ProgramFY96, FY99 - FY00–02

  26. Trends in OED Work ProgramFY96, FY99 - FY00–02

  27. Trends in OED Work ProgramFY96, FY99 - FY00–02

  28. Trends in OED Work ProgramFY96, FY99 - FY00–02

More Related