The Emergency Capacity Building (ECB) Project

1. A Guide to Measuring Change The Emergency Capacity Building (ECB) Project www.ecbproject.org

2. SUMMARY OF PRESENTATION The Emergency Capacity Building (ECB) Project Partnership in Innovation The need for a Guide to measure the ‘Contribution to Change’ Using a Livelihoods Approach Measuring Change for Households The Methodology Field Testing Importance to the Sector

3. Emergency Capacity Building (ECB) Project • Collaboration between 6 international NGOs 2008-2013 • Focused on 3 core themes: DRR, National staff capacity building, improving accountability to disaster affected communities & developing impact measurement best practice • 4 country and 1 regional consortia (Bangladesh, Bolivia, Indonesia, Niger and the Horn of Africa) with representatives in each country • Global Advisor Groups tackling shared challenges and programs for each core theme (DRR, Staff Capacity, Accountability & Impact Measurement) • Global Project team to support the structure– Director, Field Manager, Communications, M&E, Finance and Fundraising, Knowledge Management • Funding from Bill & Melinda Gates Foundation, DfID, ECHO, OFDA

4. Accountability & Impact Measurement (AIM) • Agreement on Key Elements of Accountability • Development Good Enough Guide to Impact Measurement & Accountability and supporting communications materials • Established an accountability standing team of deployable staff • Developing a new Guide to (just) Measuring Change • Piloted Joint Evaluations in four countries • Piloted Joint Needs Assessment (rapid first 72hours) in 3 countries (ACAPS partners) – training and data capture

5. Partnership in Innovation • The ECB Project is partnering with the University of East Anglia to research, pilot and publish a new methodology and supporting tools in a Guide to measure the ‘contribution to change’ brought about by humanitarian interventions in rapid onset emergencies.

6. The need for a Guide to measure the ‘contribution to change’ • Difficulties in measuring impact • Demand from donors and humanitarian NGOs/UN to demonstrate impact and improve the quality of emergency responses • Little guidance currently exists to assist agencies to measure impact quantitatively and qualitatively

7. Using a livelihoods approach • The approach is based on the sustainable livelihoods framework used by DFID. This framework is a tool aimed at improving the understanding of livelihoods, particularly the livelihoods of the poor. • The framework serves mainly as a way of organising data and analysis to view development interventions, so it is being adapted to be relevant to humanitarian interventions.

8. The Livelihood Tool

9. The Methodology • Designed to be robust enough to collect credible evidence of the ‘contribution to change’ but simple enough for field staff to use. • Whilst it was first envisaged that two methodologies would be tested, after Bihar it was decided to concentrate on the the Retrospective method

10. The Household Approach • Most humanitarian initiatives have impacts that are likely to be recognised by affected populations if they result in changes in daily activities and the livelihoods of households. • As livelihoods differ within communities, this project is studying the impacts at the household level to best determine how livelihoods have changed • The changes stemming from both humanitarian interventions and the coping strategies of affected communities will be analysed.

11. Retrospective Methodology • Where no baseline has been established, this methodology will enable the collection of data to best measure changes for households. • The field work will take place 12-15 months after the disaster has occurred. The data collection will include information that will enable the reconstruction of a baseline as well as information on how the situation has changed.

12. Field Testing • The first field test was conducted in late 2011 in Bihar, India. Following flooding earlier in the year, a team including researchers from the University of East Anglia, spent a week interviewing communities using the draft methodology and tools. • This field test led to a number of improvements in the methodology, of which a revised version was tested in Guatemala in March 2012 as a Retrospective study. • The third field test will be held in Sri Lanka in May/June 2012, and once the guide has been drafted, a final field test will be undertaken.

13. Who is the Guide for? • While Monitoring and Evaluation professionals are the primary target audience for this guide, there are many other staff who will benefit from this guide. • The reach of this Guide should not be restricted to those directly measuring impact in the field – headquarters communications or fundraising staff should familiarise themselves with the ways to measure impact to ensure a consistent approach across fundraising, programming and reporting.

14. Contents of the Guide • Guidance on how to sample data, train staff in data collection • Household and community surveys • Qualitative and quantitative studies • How to extrapolate the findings of data pertaining to individual programmes • Single-agency and multi-agency evaluation contexts

15. Importance for the sector • This Guide will provide practical and field tested tools for humanitarian NGOs, United Nations agencies and donors to measure the impact their programmes have on households. • The Guide will help agencies to ensure better quality and more needs-based humanitarian responses, through strengthened evidence on which to base decision making and reporting.

16. For further information contact info@ecbproject.org www.ecbproject.org/measuring-change