Evaluation Network follow-up to the Paris Declaration: An options paper

1. Evaluation Network follow-up to the Paris Declaration:An options paper David Booth Alison Evans

2. Overview • What sort of challenge to evaluation? • What would be the value added? • What should be the focus? • Is it possible? • Tradeoffs and options • Recommended way forward • Timing and next steps

3. What sort of challenge? • One that can’t be refused– the Declaration itself calls for independent evaluation “to provide a more comprehensive understanding …” • An important one– the Declaration contains an unprecedented set of mutual commitments, leading to major efforts and investments • An exciting one– it is possible to articulate the “program logic” of the Declaration and interrogate the underlying theories of change

4. What would be the value added? • The monitoring scheme will tell us something about what is happening, so will peer reviews, • but we need to understand also how and why/why not • The monitoring has also made various pragmatic compromises to get standardised data quickly and without imposing on countries • an evaluation effort could be more balanced and wide ranging (and would be required to be so)

5. What should be the focus? • The illustrative results chain in the paper shows the conceptual scope, from inputs to impacts • There is no necessary implication that the program theory is agreed or that it is right • The principal focus would be on the top levels, and on formative evaluation questions • which does not imply results and impacts do not matter! • “Negative” findings – why things are not working, or are working for unexpected reasons – should be valued as much as “positive” findings

6. Is it possible? • So long as the formative purpose is kept firmly in mind • So long a ambitions regarding the results chain are well controlled, and the framework is used selectively • So long as it is possible to draw on a wide range of types of evidence to answer a selected sub-set of questions • These include • are we doing things right? • are we doing the right things? And • In what range of situations might these things work? • But probably not “is it cost-effective?”

7. Tradeoffs and options • Preparation of a common evaluation framework is widely supported – considered intrinsically useful • Otherwise, options involve significant tradeoffs: • between buy-in and “criticality” • between country- and cross-country learning • between low costs and adding value • No approach to design of an evaluation process will have all of the desired qualities • Needed: an optimal solution or good compromise

8. Recommended way forward Four loosely-connected activities, offering distinct strengths • Development of a common framework – an agile, consultative but fairly top-down process • Country-led country evaluation work – based on self-selection, building closely on existing monitoring initiatives and localising elements of the common framework • Thematic studiesfocused on donors– similar principles • A medium/long-term programme of synthesis and meta-evaluation – triangulating diverse data to address the big questions in collaboration with MT Monitoring Plan of JV

9. Timing and next steps Considering the time-line to the HLF of 2008: • Work on the framework should happen in 2006 • To be sufficiently thorough, promoting the idea of country-led evaluation also needs to start soon • Dialogue with the JV to ensure well-integrated processes is vital • If the general approach is agreed, a task team, preferably with some country representation, would be required to take the work forward

10. In summary • In some significant and useful ways, it can be done • It is very important that it should be done • Appropriate, realistic expectations are needed, especially before 2008 • “Walking on four legs” may look untidy, but it will do the job, recognising the constraints