What works in WASH? Evidence from systematic reviews

1. International Initiative for Impact Evaluation What works in WASH? Evidence from systematic reviews Hugh Waddington Geneva Evaluation Week 7 May 2015

2. 3ie is managing 3 linked work streams Impact evaluations • Funding available for 2 studies • RFQ (2-stage awards) • Review of proposals from June 7th • Presentation of baseline findings in Kathmandu (Nov-Dec 2015) • Draft report in December 2016 Mid-term review • WSSCC medium-term strategy • IFMR contracted • Draft report by January 2016 Systematic reviews • 2 questions • Call closes May 12 • Contracting in June • Draft reports in 2016

3. Call for SR proposals: deadline 12 May 2 questions developed with WSSCC and 3ie-WSSCC programme advisory group • What is the effectiveness of interventions aiming to promote sanitation and hygiene behaviour change in communities? • To what extent has the sanitation and hygiene sub-sector taken into account the life-cycle approach in the design, implementation, maintenance and use of programmes during the MDG period?

4. “Literature reviews are like sausages... I don’t eat sausages as I don’t know what goes into them”Dean Karlan

5. 3ie Evidence gap map gapmaps.3ieimpact.org

6. A sector-wide logic model

7. Key evidence gaps 138 Impact and 26 SRs of WASH evidence in low and middle income countries. Mostly measure diarrhoeal health outcomes (not very rigorously) Very few studies examining demand-side approaches (eg CLTS, san marketing) • Few estimate impacts for sanitation and hygiene programmes as part of scale up • No rigorous prospective studies of sanitation programmes in SSA • Few on governance

8. 3ie systematic reviews

9. Methodology • Extensive search of published and unpublished sources (updated in 2012-13) • PubMed, Embase, LILACs, Web of Science • JOLIS, IDEAS, British Library for Development Studies (BLDS), Cochrane Library, scholar.google • Personal communication with leading researchers • Hand-search, back-referencing and citation tracking • Inclusion criteria: • IEs measuring impact of intervention on diarrhoea morbidity using experimental (RCTs) and quasi-experimental methods • reported specific water, sanitation, and/or hygiene intervention(s); • were conducted in low- or middle-income countries; • use an infant or child as the unit of observation; and • estimate impact on diarrhoea morbidity, measured under non-outbreak conditions.

10. WASH systematic reviews search process 74,181 records identified through database search 1,024 records identified through other sources Identification 49,472 records after duplicates removed Screening 1,869 records screened 47,603 records irrelevant 225 full-text articles assessed 120 full-text articles excluded Eligibility ‘Counterfactual’ impact evaluation studies 137 studies included in quantitative synthesis of effects ‘Factual’ evaluation studies (egqualitiative, ethnographic etc) Included 44 studies included in qualitative synthesis

11. Diarrhoea impacts: evidence from 71 studies update ongoing)

12. Sustainability 1: less impact over longer periods

13. Sustainability 2: compliance falls over time • Ceramic filter provision in Cambodia; 3 years later only 31% households were still using the filters (Brown et al, 2007) • Pasteurisation in Kenya; 4 years later only 30% continued to pasteurise their water(Iijima et al, 2001) • Programme promoting POU water disinfectant in Guatemala 1 year later; repeated use among only 5% of households from original trials (Luby et al, 2008). • Water filters in Bolivia; compliance 67%; assessment made 4 months after trial ended (Clasen et al, 2006)

14. Sustainability 3: lack of WTP (Kremer et al. 2012) In Kenya, access to free chlorine increased uptake to over 60 percent, whereas coupons for even a 50 percent discount had a minimal effect

15. Sustainability of hygiene impacts Source: Waddington et al 2009 3ie Systematic Review

16. Scalability of hygiene promotion • Vietnam (Chase & Do, 2012): • Handwashing BCC (not soap) scale-up through 1) mass media & 2) inter-personal communication • No health or productivity effects • Peru (Galiani & Gertler): • Hygiene promotion through 1) mass media & 2) community level (health facilities, schools) • No impacts health or nutrition (compliance measured) • Bangladesh (Huda & Luby 2012): • SHEWA-B local community hygiene promoters • Low compliance rates & no impacts on diarrhoea or ARIs

17. Supply doesn’t create its own demand • Reduction in child disease rates not observed by carers or seen as substantial enough benefit to warrant costs (money/time) – what role for health education? • Adoption of innovations (social change) is a slow process (early adopters vs. laggards) – what role for ‘triggering’ (CLTS)? • Perceived benefits important (e.g. user satisfaction, time-use, safety) – role for appropriate technology/maintenance?

18. Thank you • Systematic review evidence library: http://www.3ieimpact.org/evidence/systematic-reviews/