Donor/Agency HQ Evaluations

1. Donor/Agency HQ Evaluations Presentation to International Reference Group 30 November, 2009

3. Eleven Phase 1 evaluations Anticipated seven new Phase 2 evaluations Updates of the Phase 1 evaluations Building blocks at Donor/Agency HQ level

4. Purpose of Donor/Agency HQ Evaluations • To deepen our understanding of the findings and results emerging from Monitoring Survey inputs; • To facilitate global learning on aid effectiveness through the evaluation processes and to facilitate more efficient implementation of the Paris Declaration; • To make specific recommendations to development agencies and to the global aid community for improving aid effectiveness; • To supplement and strengthen the basis for the main focus of the Phase 2 evaluation, a strong set of Country-level Evaluations.

5. Objectives of Donor/Agency HQ Evaluations • To enable donors/agencies to clarify, improve and strengthen policies and practice consistent with the Paris Declaration in pursuit of aid effectiveness and development effectiveness. • To highlight barriers and difficulties that may have limited the implementation of the Paris Declaration and its effects and impacts – and ways that these barriers and difficulties may be overcome. • To enable sharing and exchange of experience among stakeholders, countries and partnerships so as to facilitate reflection, lesson-learning and policy improvement.

6. Phase 1: Input, output and ”enabling conditions”: a) Commitment to the Paris Declaration principles, b) Capacity to implement it, and c) Incentives to do so. Phase 2: Same as Phase 1 Development outcomes and effectiveness will be examined through the country studies Focus of Donor/Agency HQ Evaluations

7. Benefitting from Experience • Interpreting the enabling conditions using ”best practices” • Improving understanding of context • Deepening understanding on MfDR and Mutual Accountability and use of country systems • Adding “Mirror questions”, including examining policy coherence

8. Interpreting the enabling conditions • Commitment: Internal and external factors • Capacities: Institutional capacity and systemic capacity • Incentives and disincentives: Individual level, agency level, and level of government

9. Understanding Context • Staff, budget, geographic spread of programme, multi/bilateral share, share provided to vertical funds, and aid modalities (project, programme, budget support, TA etc.) • Organizational structure including decentralization • Performance management • Action plans to implement Accra

10. Deepening Understanding • What is the main problem facing donors/agencies in fulfilling the Paris Declaration commitment to Manage for Development Results? • What arrangements or mechanisms for Mutual Accountability have been in place and how well are they working? • What explains the lack of use of country systems even where these are considered relatively strong?

11. From Approach Paper Have “transaction costs” of delivering aid actually been reduced and/or shifted because of implementation of the Paris Declaration? If so, how, and what are the effects of such changes? What would be the likely implications for development results if donors were to shift a majority of aid to either general or sector-specific budget support? “Mirror questions” • New Questions • What is the influence of the Paris Declaration on government policies with implications for developing countries? • Who are the key actors in the donor country (or constituency for multilateral agencies) involved in the aid arena, taking major decisions on aid flows, priorities, activities, programs and projects? • How are their perceptions and decisions influenced by the Paris Declaration and the Accra Agenda for Action?

12. Evaluation Methods • Literature and documentation review • Syntheses and meta-analyses of existing evidence • Semi-structured interviews and focus groups (key respondents at HQ level and relevant field staff, either by phone or video conference or through selected field visits) • Structured surveys and questionnaires focused on the embassies / country offices located in those of the countries which have volunteered to conduct a country level evaluation

13. Updates to Phase 1: Purpose The purpose of the updates is to further strengthen the robustness of the evaluation as such through an effort to: • Address certain weaknesses on the knowledge and evidence base at HQ level, identified in phase 1 that would strengthen the scope of coverage; • Increase potential for cross validation between the country studies and the HQ studies by examining a series of “mirror questions” that would add depth to the evaluation; and • If possible, throw light on what changes the Accra commitments have lead to at the HQ level.

14. Updates to Phase 1: Questions • Context: Staff, Budget, Geographic spread of programme, Multi/bilateral share, share provided to vertical funds, and aid modalities (project, programme, budget support, TA etc.), organizational structure including decentralization, performance management, action plans to implement Accra • “Mirror questions”: questions in country studies that can be “mirrored” on the donor/agency side, in areas where it would enhance the depth of the evaluation if there was evidence from both levels. Questions were suggested in the Approach Paper and new questions that emerged from the regional workshops • “Deepening questions”: questions elaborating on issues where there is a general recognition (expressed in the Monitoring Survey and in the London meeting with the Evaluation Coordinators) that there is insufficient knowledge: Managing for Development Results and Mutual Accountability, and use of country systems .

15. Support & Timeline • Support: refer to separate presentation • Timeline: • For Donor/Agency HQ Studies updates – end May 2010 • For Donor/Agency HQ Studies first draft report – 15th September 2010