Martina Bj örkman, IGIER, University of Bocconi, & CEPR

1. Power to the PeopleEvidence from a Randomized Field Experiment on Community-Based Monitoring in Uganda Martina Björkman, IGIER, University of Bocconi, & CEPR Jakob Svensson, IIES, Stockholm University, NHH, & CEPR

2. Background • Millions of children die from easily preventable causes • Weak incentives for service providers • Top-down approach to monitoring also lacks appropriate incentives • Recent focus on strengthening providers’ accountability to citizen-clients • Beneficiaries lack information • Inadequate participation by beneficiaries

3. Research Questions • Can an intervention that facilitates community-based monitoring lead to increased quantity of health care? • Increased quality of health care? • Did the intervention increase treatment communities’ ability to exercise accountability? • Did the intervention result in behavioral changes of staff?

4. Intervention • 50 rural dispensaries in Uganda • Drawn from 9 districts • Households w/in 5 km catchment area • 18 local NGOs • Provide communities with information on relative performance • Encourage beneficiaries to develop a plan that identified steps the provider and community should take to improve service performance and ways to get the community more actively involved in monitoring

5. Intervention Specifics • Pre-intervention survey data used to compile unique “report card” for each facility • Translated into community’s main language • Posters by local artist for non-literate • Information provided to community through participatory / interactive meetings • Community: suggestions summarized in action plan • Staff: review & analyze performance • Interface: contract outlining what needed to be done, how, and by whom

6. Timing • Intervention intended to “kick-start” community monitoring • Mid-term review after 6 months, but no other outside presence in communities • Not able to document all actions taken by communities

7. Data • Pre-intervention survey to collect data for report cards • Quantitative service delivery data from facilities’ own records • Households’ health outcomes, perceptions of health facility performance parameters • Whenever possible supported by patient records • Post-intervention survey 1 year after intervention • Child mortality (under 5) • Weight of all infants • Roughly 5000 randomly-sampled households in each survey round

8. Evidence of Increased Monitoring • More than 1/3 of Health Unit Management Committees in treatment communities reformed or added members; no change in control communities • 70% of treatment communities had some sort of monitoring tool (such as suggestion boxes, numbered waiting cards, duty rosters); only 16% in control communities • Performance of staff more often discussed at local council meetings in treatment communities • NGO reports suggest that discussions shifted from general to specific issues regarding community contract

9. Treatment Practices • At facilities in treatment communities significantly: • More likely to have equipment used during exam (19% increase) • Shorter wait times (10% decrease) • Less absenteeism (14%age points lower) • More on-time vaccinations • Larger share received information on dangers of self-treatment and family-planning • Also possibility of less drug-leakage

10. Utilization • At facilities in treatment communities significantly: • Higher utilization of general outpatient services (16%) • More deliveries at the facility (68%) • From household surveys: • Consistent increases in use of treatment facilities • Reduction in visits to traditional healers & the extent of self-treatment

11. Health Outcomes • Child mortality • 3.2% in treatment communities • 4.9% in control communities • 90% confidence interval for difference ranges from 0.3%-3.0% • Corresponds to roughly 540 averted deaths (per 55,000 households in treatment communities) • Infant weight • Compare distributions of weight-for-age (z score) • Difference in means is 0.17 z score • Reduction in average risk of mortality based on risk of death from infectious disease among underweight children estimated to be 8%

12. Institutional Issues • Did district or sub-district management react to intervention? • Check that treatment & control communities have comparable: • Monthly supply of drugs • Funding • Construction or infrastructure improvements • Visits from government or Parish staff • Employment (dismissals, transfers, hiring)

13. External Validity • Idiosyncratic process differed from community to community in experiment • In another context, process could play out entirely differently • Cultural factors key

14. Scaling Up • What actually caused the observed effects? • How to replicate the intervention? • Process dependent on NGO facilitators • No way to know which components of monitoring were influential

15. An Alternative Explanation • Possible (but unlikely) that intervention directly influenced providers’ behaviors • Outcomes not necessarily result of increased monitoring • Considered additional treatment of staff meetings only but decided against it • Financial reasons • Ethical reasons

16. Conclusion • Impressive effects, but intervention difficult to replicate • Important piece of causal chain undocumented ?