Joint humanitarian impact evaluation: options paper Tony Beck

1. Joint humanitarian impact evaluation: options paper   Tony Beck 25th ALNAP Meeting, London, 18th November 2009 Commissioned by the Evaluation and Studies Section UNOCHA

2. Purpose of paper • To set out a process for facilitated consultation over six to nine months on options for joint humanitarian impact evaluation • To delineate options for discussion during the consultation process

3. Purpose of workshop session • To discuss the proposed consultation mechanisms and focus • To discuss whether the options presented are the right ones to discuss NOTE: the purpose of the session is not to have an in-depth discussion of the options, as this is planned to take place over the next 6-9 months

4. Background • Ongoing discussion on joint impact evaluation among agencies including OCHA, ECB and UNICEF • Evaluability study completed in June 2009 • Need for sustained consultation identified by respondents

5. Consultation in two countries with: • The affected population • Local NGOs • National and local government • International NGOs • Clusters • UNCT/HCT • Donors • Plus regional level consultation

6. Consultation at the international level with: • ALNAP members and observers, including the UN, IFRC, INGOs, academics and research bodies, independent evaluators • Evaluation bodies, Paris Declaration evaluation Phase 2 • Donors

7. Consultation coordination • OCHA forms and chairs a working group on joint impact evaluation • The working group guides the consultation process, synthesizes the consultation results, and makes recommendations on the future of joint impact evaluation

8. Consultation: process Working group should oversee development of: • a short background brief on each option • a set of key discussion points, and • a method for capturing and writing up the findings from each consultation process

9. Consultation – what would it look like? • Consultation with the affected population: How can we meaningfully discuss with the affected population their priorities and perspectives on joint impact evaluation? On what kinds of topics would affected people like a joint impact evaluation to focus? Who will lead the consultation process with the affected population; can it be fitted into ongoing consultation processes?

10. Consultation – what would it look like? • Donors Two donor forums in country hosted by e.g. donor coordination forum, focusing on options: what do donors want joint impact evaluation to look like? Similar forum e.g. in London, Geneva or New York for HQ staff

11. Options for consultation • Evaluation purpose: judgment, and/or internal lesson learning, and/or generalized knowledge generation? • Evaluation focus: institution, affected population, or both? The entire system, of parts of the system? • Method: mainly quantitative, mainly qualitative, or mixed (if feasible)?

12. Options for consultation • Capacity: national or international evaluators, or both?   • Method: pilots in which places?   • Co-ordination: a common framework approach?   • Optimal management arrangements for joint impact evaluation: a dual structure?

13. Areas for discussion today • What should the working group to guide the process look like (e.g. similar to the ALNAP Steering Committee)? • Is there anything missing from the consultation process? • Are there other options that need to be discussed, and/or are some of those presented not needed?