Ariel Fiszbein Chief Economist, Human Development World Bank

1. Governance and social service delivery: An agenda for research and actionBuenos AiresSeptember 29, 2009 Ariel Fiszbein Chief Economist, Human Development World Bank

2. The challenge of achieving human development results • More inputs ≠ better outcomes • Outcomes = f (inputs; technology, behaviors; context) • Complex systems: outcomes influenced by decisions of many actors

3. Outline • Conceptual framework • Reform strategies • Indicators and measurement tools • An agenda for research and action

4. 1. Conceptual framework

5. Governance of service delivery • Governance: rules under which actors interact • Can be seen as set of principal-agent relations defined by the: • Incentives facing agents • Accountability mechanisms available to principals

6. A set of principal-agent relations… Politicians & Policymakers Compact/Policies Voice/Politics Client Power Service Providers Citizens/Users

7. ..characterized by … (Dixit 2002) • Moral Hazard: Principal cannot attribute outcomes to (often unobserved) actions of agent –e.g. role of teacher effort in learning outcomes • Adverse Selection: Agents have private information on their ‘type’ –e.g. information on real costs at the health clinic level • Costly Verification: Outcomes are not easily observable –e.g. quality of medical care/teaching

8. Public policy as a function of provider & user/citizen behavior (Le Grand 1997, 2003) • Provider motivation: Knights or Knaves? • Knaves: self-interested, extrinsic motivation • Knights: altruistic, intrinsic motivation • Agency: Capacity for action and choice among citizens/service users • Pawns: Passive • Queens: Active

9. Queens Market Quasi-Market Agency Improve agency (information, client power) Improve provider incentives (Pay-for-performance, autonomy) “Mom & Dad” Pawns Motivation Knights Knaves

10. 2. Reform Strategies

11. Reform Strategies

12. I. Improving access to information • Are better informed citizens able to influence decisions by policy makers? • Can information empower users to: (1) make better decisions as consumers and (2) demand better service from providers? Providers Politicians & Policymakers Voice/Politics Client Power Citizens Users

13. Are better informed citizens able to influence decisions by policy makers? • Providing information is low cost intervention • When is information the binding constraint? • Is political system permeable? Examples • Newspaper campaign in Uganda • Recurso Peru

14. Newspaper Campaign in Uganda Politicians & Policymakers • In 1995 in Uganda, only 13 percent of non-wage recurrent spending on primary education reached primary schools. • From 1996-2001, amounts and dates of grant to local governments published in local and national newspapers. Voice/Politics Citizens

15. Schools in Uganda received more of what they were due, post-intervention: IMPACT ON LEAKAGE: A school close to a newspaper outlet experienced 25 percentage points less leakage compared to a school one standard-deviation (30 km) further away from a newspaper outlet. IMPACT ON LEARNING: A one standard deviation reduction (30 km) in distance led to a 0.09 standard deviation increase in average test scores. Source: Reinikka and Svensson (2001), Reinikka and Svensson (2003a)

16. Recurso – Peru • In 2005 Peru instituted a series of interventions to increase transparency and accountability in the social sectors. • However, despite an increase in stakeholder participation, quality of services remained stagnant. Is information a binding constraint? • Recurso produced instruments (including videos distributed by television stations and media) designed to educate stakeholders about benchmarks in education and nutrition. • Universal testing introduced for 2nd grade students, with results being provided back to parents starting in 2008 • Nutrition standards established and provided to communities. Politicians & Policymakers Voice/Politics Citizens

17. Can information empower citizens as users? • In decentralized settings where state has low capacity, community monitoring could be essential • Can community monitoring be promoted at low cost? • When there is choice, can information influence demand for services? Examples • Community monitoring of health services in Uganda • Community information campaign for education in India

18. Citizen report cards for health in Uganda • 2001 all public health services become free-of-charge. Dispensaries are controlled at the district level. • What’s the effect of providing information on service delivery status relative to other providers and gov’t standards to users? • Impact of Citizen Report Cards: • .17 z-scores increase in weight of infants and 33% reduction in infant mortality. • Utilization of services was 16% higher in treatment facilities than controls.

19. Impact of Community Information Campaigns:(Three Indian States: Karnataka , MP, UP) • India decentralized school management to local level and established ‘village education committees’ • Can information in the hands of communities help in making schools better and improve learning outcomes? Pandey, Priyanka et al. Community Participation in Public Schools: Impact of Information Campaigns in three Indian States. South Asia Human Development. Presentation. World Bank. 2007.

20. School Community Information Campaigns Campaigns to inform the community about its oversight roles in public schools & the services to which children are entitled. 8-9 meetings in a village over 2 months Providers Client Power Users Pandey, Priyanka et al. Community Participation in Public Schools: Impact of Information Campaigns in three Indian States. South Asia Human Development. Presentation. World Bank. 2007.

21. Impact of Information Campaign - On Educational Inputs & Student Learning: Pandey, Priyanka et al. Community Participation in Public Schools: Impact of Information Campaigns in three Indian States. South Asia Human Development. Presentation. World Bank. 2007.

22. Strategy IILinking pay to provider performance 1) Can paying health providers for results instead of inputs result in improved quality and quantity of health care? 2) Will linking teacher pay to student performance increase achievement? Politicians & Policymakers Compact/Policies Service Providers

23. Results-based financing (RBF) ≈ Pay-for-performance (P4P) Provision of payment for the attainment of well-defined results Transfer of money or material goods conditional on taking a measureable action or achieving a predetermined performance target (CGD, 2009) RBF takes many forms… Payers Payees DonorCentral governmentLocal governmentPrivate insurer Recipients of careHealth care providersFacilities / NGOsCentral governmentLocal governments $

24. Can the introduction of pay for performance for patient services impact the quality and performance of hospitals/health centers? • Impact on equity of services? • Will quality be affected by workers focusing on quantity of services provided? • Can facilities maintain an adequate amount of autonomy? • Issues of autonomy for facilities. Impact Evaluations:

25. RBF in Argentina’s Health Sector: Plan Nacer • PN (launched 2002) introduces RBF for maternal and child health care. Expanded to all provinces in 2006. • Performance agreements made between Nation and Provincial Ministries of Health; & between provinces and providers; • Capitation payments provide incentives for providers to expand coverage; • Accountability ensured through concurrent audits; • Social accountability through communication campaigns, beneficiary satisfaction surveys.

26. Impact Evaluation of Plan Nacer • How does the introduction of RBF impact health indicators for pregnant women and children from 0 to 6 years of age? • Does the program impact the coverage and quality of primary care for pregnant women and children from 0 to 6 years of age? • What is the impact on service delivery quality? And, how can the incentive structure be maximized to promote improved service delivery and health outcomes? Plan Nacer Questionnaire 2008

27. Rwanda Monetary incentives to health center conditioned on 14 maternal and child health outputs Payments allocated at discretion of management Advance from Rwanda Evaluation: Gertler, Basinga et al

28. Can merit pay for teachers improve student achievement? Concerns • Unintended consequences (e.g. “teaching to the test”, cheating) • How will it affect teacher motivation? • Attribution? • Political feasibility? • Examples • Teacher Incentives in Pernambuco, Brazil • Teacher Pay in Andra Pradesh, India

29. Very low levels of learning in India • ~60% of children aged 6-14 in India cannot read a simple paragraph, though 95% enrolled in school (PRATHAM, 2008) • Large inefficiencies in delivery of education • 25% teachers absent, less than half are teaching • Over 90% of non-capital spending goes to teacher salaries • Higher ‘levels’ of pay not associated with better performance • Strong unions, almost impossible to fire Teacher Pay in India – The Context Can linking pay to performance improve school quality? Identification of the causal impact of teacher performance pay is a central limitation

30. Teacher Pay Incentives in Andra Pradesh (Muralidharan and Sundararaman, 2008) • State introduces three policies: • Group and individual pay incentives based on averageimprovement in test scores of all students • Block grants to schools (extra teachers, more inputs) • Assessments conducted by an independent NGO Impact evaluation results (two rounds): • Incentives increase student performance by 0.22 SD • 1st year both incentives work well. 2nd year individual incentives work better • Additional inputs improve scores only by 0.08 SD • Teachers liked the program

31. Teacher Incentives: Pernambuco, Brazil • Background • School attendance increased. Quality still low. • Weak pay incentives (flat salary scale determined by tenure) • Low teacher motivation and high absenteeism • Poorly qualified teachers (do not pass student exams). • Pernambuco one of the lowest • performing states

32. Teacher Incentives - Pernambuco • Non-linear performance pay system (July 2008): • School goals for achievement in Math and Portuguese by grade • Staff reaching at least 50% of the goal get bonus (amount varies with salary & % of goal achieved) • Impact evaluation examining teachers responses: • Strength of incentives (measured by the targets set by an education quality index)? • Do teachers that come close to the bonus get encouraged or discouraged? • Sorting of teachers across schools?

33. 3. Indicators and Measurement Tools

34. A measurement framework

35. Measuring Governance Performance

36. Many of these indicators can be collected through Quantitative Service Delivery Surveys…. • The facility is used as the unit of analysis • Could be complemented with a household/users survey • QSDS collect quantitative information about • Physical infrastructure • Staff characteristics • Income and expenditures • Governance and management • Characteristics/Quality of service provision • Outcomes

37. Measuring quality of medical care – Medical Vignettes Getting at Clinical Quality Knowledge or “Competence” of Doctors Behavior – Actual Performance – of Doctors

38. What Doctors Know… and What They Do – Evidence from India Das and Hammer 2006

39. Measuring quality of teaching? Stallings Classroom Observation Instrument Trained enumerators take 1 minute “snapshot” of class every 6 minutes. Repeat on 2 successive days. Revisit every two months. Being utilized in the Pernambuco evaluation.

40. Beyond absenteeism: Effective supply of teaching Percent of time officially allocated to schooling; when a teacher is present; and spent in teaching and learning activities Sources: Egypt, Yemen and Lebanon from Lane and Millot (2002); Tunisia, Pernambuco, Morocco and Ghana from Abhadzi, Millot and Prouty (2006); Cambodia from Benveniste, Marshall and Caridad Araujo (2008); and Laos from Benveniste, Marshall and Santibanez (2007).

41. Abadzi, Millot and Prouty (2006)

42. 4. An Agenda for Research and Action

43. Where are the opportunities and what are the main challenges? • Measurement: Inter and intra-national benchmarking • Experimentation and evaluation • International platforms for information exchange on reform strategies • Entry points for reform: matching reform to politics

44. END

45. Absence rates among teachers and health workers Note: Surveys were all fielded in 2002 or 2003. Sources: Chaudhury et al (2006) except for PNG, World Bank (2004) and Zambia, Das et al (2005).

46. Disbursed public spending on school grants that actually reach schools Percent of school grants that actually reach schools

47. Disbursed public spending on school grants and funds allocated for health facilities that actually reached intended recipient/facility