Post-Intervention Studies

1. Post-Intervention Studies Presentation delivered to the conference on Perspectives on Impact Evaluation, Cairo 1 April 2009 By Junaid Habib and Irko Zuurmond (irko.zuurmond@plan-international.org)

2. Context • Founded in 1937 as a child-sponsorship organisation • Geographical scope: • Programmes in 49 countries in Africa, Americas, Asia • Fund-raising through 17 National Organisations • Expanding programme scope: child health, education, livelihood, water and sanitation, ….. child protection, child participation • Changing programme approach: from a needs-based to a rights-based approach –increased emphasis on underlying processes of change • Long-term presence in the community/district • Budget: approx US$600 million

3. Increased pressure to assess programme effectiveness  Increased institutional commitment • Development and adoption of a Programme Effectiveness Framework  • A basket of initiatives to assess programme effectiveness using multiple methodologies and multiple sources of information • One of the new, proposed initiatives are post-intervention studies • Considered an essential component to strengthen future programme design, in particular sustainability

4. Sponsorship: phase-out process • Standard protocol in place for phase-out from communities/areas/districts • Phase-out process starts 18-24 months prior to phase out and includes justification for phase out. • Reasons include reaching of agreed development targets (e.g. immunisation levels, primary school enrolment, etc) • Once phased-out, there is no systematic process of going back to the community/area/district  Evaluation gap

5. Proposal for introduction ofPost-Intervention Studies • Aim: To assess Plan’s contribution to long-term changes and document lessons learnt. • Objectives: Each post-intervention study will have its context specific objectives, including the analysis and documentation of the lasting results (positive / negative), lessons learnt and factors of sustainability.

6. In practical terms • To what extent have programme outcomes been maintained after phase-out? • What are the contributing and impeding factors for sustaining (positive) programme outcomes?

7. Methodology • No fixed methodology • However, a quasi-experimental study design is proposed. To include a credible counterfactual analysis: • A carefully selected control (or comparison) group: E.g. neighbouring community or a physically distant community but with identical socio-economic indicators and comparable characteristics – essential • Pre-and-post intervention comparison of both groups - desirable

8. Counter-factual ?   Factual (with Plan programme)     10-15 yrs 3-5 yrs Phase-in Phase-out Post-intervention ?

9. Where are we? • Concept paper developed and circulated • Overall organisational buy-in into the concept of post-intervention studies • Participation of Country Offices is on voluntary basis  volunteers have been identified

10. Next steps • Assess ‘evaluability’  narrow down eligible countries • Selection of two pilot countries among those who have volunteered • In the first instance, the emphasis will be on developing a sound process and methodology/ methodologies, rather than having a representative sample of countries. • Depending on the feasibility of the two pilots, scale-up the approach across Plan.