NORAD/OED Trust Fund for Evaluation Activities

1. NORAD/OED Trust Fund for Evaluation Activities Nils Fostvedt Senior Advisor to the Director-General, Operations Evaluation May 26, 2005

2. Financial Times May 20, 2005 • “Watchdog tells World Bank to focus on growth rather than health and education.” • “The hard-hitting report by the Bank’s Operations Evaluation Department criticized the emphasis placed on education and health spending.” • “Just getting spending up in the social sector is not enough for poverty reduction.”

3. The Bank Group has Three Independent Evaluation Units • Bank: Operations Evaluation Department (1973) IFC: Operations Evaluation Group (1996) MIGA: Operations Evaluation Unit (2002) • Overseen by the Director-General, Operations Evaluation (DGO), who reports directly to the Board through the Board’s Committee on Development Effectiveness (CODE) • Findings and Recommendations are not subject to approval by Bank Management • Budgets are approved by the Board • Work Programs are set by the DGO, with Board consultation and endorsement

4. Major OED Evaluation Products • Ex post project evaluations • Country Assistance Evaluations (over 70 CAEs) • New Country Assistance Strategy (CAS) Completion Report Reviews • Sector/thematic and global evaluations • Extractive Industries, Investment Climate, HIV/AIDS • Process evaluations • Global Programs, Africa Capacity Building, PRSs (Poverty Reduction Strategies) • Corporate reviews • Annual Review of Development Effectiveness (ARDE) • Annual Report on Operations Evaluation (AROE)

5. Project Evaluations • Self-Evaluation – Implementation Completion Reports (ICRs) followed by independent evaluation • Objectives-based methodology • Ratings • Ratings database

6. Independent Project Evaluations • All projects (ICRs) subject to OED desk reviews • Subsequent in-depth Project Performance Assessment Reports (PPARs) for 25% • Internal quality review process • Send drafts to Bank staff and to Borrowers for comments • All PPARs are now disclosed

7. Key Ratings: Project Outcome • Rating Scale • Highly Satisfactory • Satisfactory • Moderately Satisfactory • Moderately Unsatisfactory • Unsatisfactory • Highly Unsatisfactory • Based on three components • Efficacy: To what extent were objectives achieved? • Efficiency: Cost effectiveness of operation? • Relevance: Were these the right objectives?

8. Outcome Ratings

9. Interactions with Management • Get comments on evaluation reports from Bank management before sending to CODE; draft Management Response accompanies evaluation. • Review management’s actions committed in Management Response and report to the Board in the Management Action Record (MAR). • Respond to management comments on evaluation products at CODE/Board Meetings.

10. Reporting to the Board • Meet with Board Committee (CODE) and the full Board on major corporate reviews (ARDE, AROE) • Present sector, thematic and process reviews to CODE • Discuss CAEs and methods with CODE Subcommittee • Do not negotiate content of reports with CODE/Board • Provide comments to Board and participate in CODE discussion of selected Management reports and proposals

11. Part of the Bank’s Evaluation and Control System • Independent Evaluation (Reports to Board) • OED, OEG, OEU • Quality Assurance (Reports to Management) • QAG, QACU (Bank, at entry, safeguard compliance) • Quality Assurance Teams, Credit Review (Bank, IFC) • OPCS, CPR, CES (Bank, reviews; IFC, risk and compliance) • Financial Probity (Reports to Management) • Internal Audit Department (effectiveness of controls) • Institutional Integrity Department (fraud, corruption) • External claims of noncompliance • Inspection Panel (Bank, investigative, reports to Board) • Compliance Advisor Ombudsman (IFC, MIGA to management)

12. NORAD/OED Trust Fund for Evaluation Activities • NKr15 million over three years • Funding supports selected OED evaluation studies and training activities • The funds are used for consultants, dissemination activities, evaluation capacity development (ECD) • Funds are can be used for consultants from any member country • But we wish to strengthen evaluation partnership with Norway

13. OED’s Use of Consultants • About $5 million per annum, mostly for individual consultants • Consultants typically work in teams led by an OED Task Manager • They are identified based on specific requirements, which depend on work program • Also use senior individuals on advisory panels for major evaluations

14. Possible consulting requirements next three years include • Impact evaluations • Evaluation capacity development • Judicial reform, agriculture, environment, economic sector work, public sector reform, small & medium enterprises, fiduciary systems and infrastructure • Also work on evaluations of projects and country programs • Need specialists in these areas with an understanding of evaluation • Experience in developing countries an advantage

15. If you think you have something to offer: • Look at OED’s website for types of products http://oed.worldbank.org/index.shtml • Contact Patrick Grasso pgrasso@worldbank.orgor Nils Fostvedt nfostvedt@worldbank.org • And don’t hold your breath

16. Thank You