GENERAL APPROACH FOR PHASE II OF THE EVALUATION OF THE PARIS DECLARATION ON AID EFFECTIVENESS

1. Phase II Approach Paper GENERAL APPROACH FOR PHASE II OF THE EVALUATION OF THE PARIS DECLARATION ON AID EFFECTIVENESS

2. Main Elements of the Approach • Building Blocks (Ch. II) • Scope and Focus (Ch. III) • Conceptual Frameworks (III-B) • The Evaluation Model (III-C) • Critical Substantive Topics (III-D) • Methodological Issues (III-E; Day 2) • Governance and Architecture (Ch. IV; Day 3) • Timetable and Budget (IV-F/G; Day 3)

3. Building Blocks (Ch. II) • Booth and Evans, Options Paper (2006). • Framework ToR for First Phase Evaluation (2007). • Wood, etal, Synthesis Report on First Phase of the Evaluation of Implementation of the PD (2008). • Stern, etal, The Paris Declaration, Aid Effectiveness, and Development Effectiveness (“linkages study”) (2008) • Jones and Kotoglu, Applicability of the Paris Declaration in Fragile and Conflict-affected Situations (2008)

4. Scope and Focus: Conceptual Frameworks (III-B) • Broad Evaluation Questions • Degree of adaptation of PD to country & aid circumstances • Value added to stakeholders of using PD partnership arrangements • Could other strategies achieve results more effectively and efficiently than the PD? • Indicative Evaluation Framework - 2006 (Figure 1) • Policy-focused Eval. Framework - 2008 (Figure 2) • Some comparisons between the two frameworks

6. PD Configuration Actors Intentions & Priorities Contexts Inputs and Outputs Country level Outcomes & Impacts Aid Management Analysis, planning, budgeting Aid-related Policies & Programmes Relevant Inclusive Targeted Country Level Government, Parliament, Private Sector, NGOs Poverty Reduction/ Achievement of MDGs Effective, efficient & sustainable Country Characteristics Poverty, human development growth governance Development Outcomes Human & economic Partnership Working Capacity Development State Building Public management inclusion & institutionalisation Donors Goals and priorities of donors & extent of coordination Aid Scenarios Donor engagement, aid volumes & dependency, aid composition & modalities Country Policies & Policy Making Policy making, governance & institutions Development-related policies International Outcomes & Impacts ODA Legitimation Donor ‘policy learning’ Figure 2. Policy-Focused Evaluation Framework. (from Fig. 3.2, Stern, etal, November 2008)

7. Some comparisons between the Two Frameworks • Both frameworks give importance to outcomes, impacts, & external factors • Fig. 1 “unpacks” outcomes and impacts • Feedback loops in Fig.2 suggest learning potential for countries and donors (e.g. “state-building”) • Outcomes and impacts on donor policy and ODA are delineated in Fig. 2 • Fig. 2 unpacks inputs and associated “mechanisms of change” (AP para 45 from Stern, etal. P. 47)

8. Phase 2 Evaluation Model Two complementary foci: • Implementation focus (expanded Phase 1) • Larger, diverse set of countries • Wider range of stakeholder groups • More attention to capacity building • Assess donor aid to analytic & advisory activities • Self-evaluation of PD implementation at HQ level by several additional donors not covered in Phase 1

9. Phase 2 Evaluation Model Two complementary foci (continued): • Results focus: identify PD influence on development effectiveness (outcomes & results): this should be a key value added of the Phase 2 Evaluation • Challenges • Brief time span since PD endorsed (March 05) • Demanding data requirements • External influences make attribution difficult • Rigorous methodology required

10. Phase 2 Evaluation Model Two complementary foci: • Results focus (continued) • Dealing with the challenges • More feasible to identify PD-related policy changes that “point” to likely changes in development effectiveness • Focus on only 1 or 2 sectors and a few MDGs • “Backward-track” from current results to “PD-like” actions in previous years • Identify “control” entities or areas (a methodological approach to be discussed on Day 2)

11. Evaluation “Propositions” • 21 propositions (tentative hypotheses) from “Linkages Study” (AP Box 1, p. 16) • About 10 relate directly to results (e.g. 2, 7, 9, 10, 11, 12, 13, 17, 19) • Do these propositions capture the key potential PD – results linkages? • E.g., have PD-like instruments, such as SWAps, yielded development results?

12. Critical Substantive Topics • Accra Agenda for Action – September 2008 • Capacity Strengthening • Fragile Situations

13. Critical Substantive Topics Accra Agenda for Action (AAA) • Proposed that Phase 2 assess AAA progress • The AAA makes 48 Commitments under 3 broad headings (and 12 sub-headings – AP para 47): • Strengthening Country Ownership over Development • Building More Effective and Inclusive Partnerships for Development • Delivering and Accounting for Development Results (2 of the 4 commitments relate directly to results)

14. Critical Substantive Topics Capacity Strengthening • A means to realize MDGs & other development goals • Possible evaluation questions: • What changes in capacity attributable to the PD? • What effects have these changes had on development outcomes and results? • How effective has donor assistance been in strengthening partner country capacity? • What factors explain effectiveness of this aid and what lessons can be learned? • Development results of some capacity strengthening will be observable only in long-run. • Issue: Should Phase 2 concentrate only on a few MDGs? Is including capacity strengthening too ambitious?

15. Critical Substantive Topics Fragile Situations – Phase 1 Thematic Study • Given wide range of such situations, this study needs to be taken into account • Among the study’s major conclusions are: • In post-conflict or improving settings, the PD can be applied incrementally, but in deteriorating settings, it may be of limited or declining relevance. • Harmonization is a “key entry point” for improving aid effectiveness in these situations. • Aid effectiveness in a fragile situation must include “state building,” taking into account the political and conflict-resolving nature of the process.

16. Main Elements of the Approach(“√” covered today; next 3 topics to be covered in subsequent sessions) • Building Blocks (Ch. II) • Scope and Focus (Ch. III) • Conceptual Frameworks (III-B) • The Evaluation Model (III-C) • Critical Substantive Topics (III-D) • Day 2: Methodological Issues (III-E) • Day 3: Governance and Architecture (Ch. IV) • Day 3: Timetable and Budget (IV-F/G)