IMPROVING QUALITY AT ENTRY

1. IMPROVING QUALITY AT ENTRY Findings from QAG Review April 13, 2005

2. Entry quality matters for outcomes • The single greatest factor explaining why project outcomes at exit are not satisfactory • Especially so regarding institutional capacity, ownership/commitment • In a large number of cases, weak M&E at entry and in implementation • Projects rated less than satisfactory at entry are almost twice as likely to not have satisfactory outcomes as others.

3. Issues at entry put more of a burden on supervision • With issues at entry, supervision should correct them by seizing opportunities (MTRs, restructuring) for on-course correction. • Good quality at entry is an “inheritance” to be conserved and sustained in supervision.

4. Sixth Quality of Supervision Assessment (QSA6) • Supervision assessments were carried out for randomly selected 130 operations approved by the Board prior to FY04 and active on July 1, 2004. • Operations were rated on a scale of 1 to 6 – 1 being Highly Satisfactory and 6 being Highly Unsatisfactory. • 10 operations could be classified as CDD/Social Fund type of operations • Seven of the 10 operations were rated Satisfactory, 2 Moderately Satisfactory and 1 Unsatisfactory. This is consistent with the overall findings of 90% Moderately Satisfactory or Better.

5. Quality at entry Issues • About one-fourth of the sample in QSA6 had quality at entry issues and • 60% of these operations are considered unlikely or uncertain to reach DOs as against … • 13% of operations without problems in quality at entry. • The main issues at entry quality noted, lie in lack of: • an inadequate results framework; • realism of DO assumptions; • readiness for implementation; • realism of overall risk ratings of operations;

6. SD/CDD Operations at entry • Of the 10 operations, panelists assessed: • 6 likely to achieve the DO; • 4 had a sound results framework; • 6 were ready for implementation; • 6 had realistic assumptions about reaching the DOs; (cont’d)

7. SD/CDD Operations at entry • About 40% of the CDD/SF sample operations had deficient quality at entry. • About the same percentage of these are considered unlikely or uncertain to reach DOs.

8. Trailing areas in SD/Social Fund • Risk assessment, esp. Political, governance aspects and institutional capacity • Project approach and design alternatives • Institutional capacity assessments and capacity building • Readiness on TA, staffing & management arrangements • M&E (baselines & outcome indicators) for institutional and policy reforms, DOs, poverty and social aspects

9. What’s key in highly satisfactory quality at entry • Strategic relevance • Thoughtful choice of approach • Strong policy and institutional understanding and design • Readiness for implementation • Candid assessment of risks To get all these – excellent staff and managerial inputs and processes.

10. Essentials of Project Design:1 • Client/stakeholder buy-in: check it, build it • Anchor the operation clearly in the country and sector context and strategy (CAS, Transitional Strategy, PRSP) • Build on analytic work (AAA, OESW, analyses of other agencies, by clients) • Develop synergy with other donors, NGOs, other Bank operations, AAA • Be clear on WHY the operation is being done, and why a Bank involvement (value added?) • Clear links between objectives & components

11. Essentials of Project Design: 2 • Be clear on outcomes: what will be achieved? when? how would this be measured? by whom? against what? does the client have the capacity for M&E? can the outcomes be attributed to the project? what are reasonable proxies for intermediate outcomes? • Be realistic in scope, complexity, time-frame, political economy, institutional capacity – match to what has been/can be shown as feasible • Look at alternative approaches/project design – sequencing, phasing • Keep it simple wherever possible • If complexity can’t be avoided, face it head-on and explicitly in resource implications, staffing, sustained attention

12. Development Objectives • No more than one or maximum two • Number of DOs will at least equal number of components – careful! • Avoid numerous sub-components with sub-objectives • Make DOs realistic and easier to track • If a project proves too complex – do restructure

13. Economic and Financial Analysis • Economic rationale for public action (should government be doing this?) • Improving efficiency (cost/time) and equity (participation of the poor, women, other vulnerable groups) • Fiscal impact – recurrent costs, sustainability, absorptive capacity • Are Bank funds truly complementary to domestic resources?

14. Project Scope • Program/sector wide approaches (most complex designs, often weak environments) • Public vs. private sector – even in a post-conflict country (El Salvador), in rural education, the project successfully turned over school management to parents and community leaders • Focus on specific issue – access to basic services (education, health care, infrastructure) - can still mean complexity, to be addressed!

15. What to do about risks? • Know what they are – be clear-eyed, especially on institutional, governance areas • Be explicit about the risks and effects on outcomes • Connect analysis of risks with design • Manage risks where possible – how will their evolution be monitored? • Be clear on residual risks and course of action for client/Bank if they materialize

16. Examples of good practice Philippines: KALAHI-CDSS Project • Directly related to the country’s development agenda • Well conceived outcomes in terms of improved governance and communities’ capacity to design and implement community investment • Incorporates lessons from similar programs in and outside the country; • Design based on good analytical studies • High commitment as evidenced from fast processing and effectiveness • Design includes a pilot phase to allow the derivation of lessons and fine-tuning of the approach • Risk of elite capture addressed by putting in place arrangements for ensuring accountability and transparency • Appropriately designed arrangements for cooperation with NGOs and local government • Candor in identification of risks and introduced measures well designed training, organizational arrangements, monitoring and supervision systems.

17. Examples of good practice Albania: Second Community Works • Combines lessons of past experiences and blended well with global practice and adapts the combination to the macro-economic environment and to fiscal decentralization • Built on lessons from past experience and innovating for the future by piloting in new areas • High degree of coordination with donors in the developing targeting criteria to reach poor municipalities • Risk assessments were done with candor and rigor; for addressing social risks, developed adequate risk management strategies, including strong community participation in decision-making and project monitoring, • Good institutional capacity assessment carried out and TA and training arrangement well prepared. • Good and innovative M&E system in place.