Teacher Performance Pay: Experimental Evidence from India

1. Teacher Performance Pay: Experimental Evidence from India Karthik Muralidharan University of California, San Diego (with Venkatesh Sundararaman, World Bank) Workshop on Accountability in Education World Bank, 22 June 2009

2. Context • Very low levels of learning in India • ~60% of children aged 6-14 in India cannot read a simple paragraph, though 95% enrolled in school (PRATHAM, 2008) • Large inefficiencies in delivery of education • In India, 25% teachers are absent, less than half are teaching • Over 90% of non-capital spending goes to teacher salaries • Teachers are very well paid • (~ 4 * GDP/Capita); Pay = f (rank, experience) • No performance-based component • Correlations suggest that higher ‘levels’ of pay are not associated with better teacher performance • Strong unions, almost impossible to fire (only 1 in 3000 schools reported firing a teacher for repeated absence) • Performance pay for teachers is a frequently suggested way for improving school quality (being tried in many countries) – but limited evidence on effectiveness • Identification of the causal impact of teacher performance pay on learning outcomes is a central limitation

3. Location of Study • Indian State of Andhra Pradesh (AP) • 5th most populous state of India • Population of 80 Million • 23 Districts (2-4 Million each) • Close to All-India averages on many measures of human development

4. Questions/Contributions • Does teacher performance-pay improve test scores? • What, if any, are the negative consequences? • Should they be at the school or teacher level? • How does teacher behavior change? • How cost effective is the incentive program? • How will teachers respond to the idea?

5. Theoretical Considerations • Incentives and intrinsic motivation • Large literature in psychology on the possibility of crowding out of intrinsic motivation due to financial incentives • Might be especially relevant to teaching • But bigger problem here may have been the complete lack of differentiation between high and low performing teachers • Kremer, Muralidharan et al (2006) find that teachers with higher job satisfaction were more likely to be absent • Incentives and multi-tasking • Teaching basic as opposed to higher-order skills (Holmstrom, Milgrom 1991) • Test preparation instead of longer-term learning (Glewwe et al 2003) • Manipulating test-taking population (Jacob 2005) • Short-term boosting of caloric content (Figlio & Winicki, 2005) • ‘Cheating to the test’ (Jacob & Levitt, 2003)

6. Addressing Concerns About Incentives on Test Scores • Crowding out intrinsic motivation • Framing matters • “Teacher recognition” as opposed to “school accountability” • Teaching to the test/effort diversion • Less of a concern given extremely low levels of learning (Lazear, 2006) • Test-taking is an important skill in the Indian context • Test design can get progressively more sophisticated so that you cannot do well on the test without deeper knowledge/understanding • Threshold effects • Minimized by making bonus a function of average improvement of ALL students • Teachers neglecting/penalizing weaker children • Mitigate/avoid this by tying incentives to “changes” from the baseline performance and assigning low scores to drop outs • Cheating/Paper leaks etc. • Potentially a big problem – here the testing is done by an independent outside testing team with no connection to the school

7. Incentive Design • Teachers were given bonus payments over and above their regular salary on the basis of average improvement of test scores of all students in grade/school over base line • Subjects considered were math and language • Assessment papers were designed by an independent testing agency (EI) • All assessments were conducted by an independent NGO (APF) • Bonus formula • Rs. 500 bonus for every 1% point improvement in average scores • Calibrated to be around 3% of annual pay (and equal to input treatments) • Based on absolute improvement (contract/piece rate) as opposed to competing for a fixed prize (tournament) • Contracts dominate for risk-averse agents • Greater transparency of incentive system under contracts • Common shocks small relative to school/class level shocks • Same information costs • Easier to dynamically game tournaments that reward value addition • Both group and individual level incentives were studied • Free-riding/Peer monitoring/Gains to cooperation

8. Design Overview

9. Sampling

10. Summary of Experimental Design • Study conducted across a representative sample of 500 primary schools in 5 districts of AP • Conduct baseline tests in these schools (June/July 05) • Stratified random allocation of 100 schools to each treatment (2 schools in each mandal to each treatment) (August 05) • Monitor process variables over the course of the year via unannounced monthly tracking surveys (Sep 05 – Feb 06) • Conduct 2 rounds of endline tests to assess the impact of various interventions on learning outcomes (March/April 06) • Interview teachers after program but before outcomes are communicated to them (August 06) • Provide bonus payments and communicate continuation of program (Sept 06)

11. Specification

12. Impact of Incentives on Test Scores

13. Distribution of Gains by Treatment

14. Impact of Incentives by Mechanical/Conceptual

15. Performance on Non-Incentive Subjects

16. Group versus Individual Incentives

17. How did Teacher Behavior Change?

18. Comparison of Inputs and Incentives

19. Teachers Liked the Program • Teachers interviewed in August each year (before they know outcomes) • 75% of teachers say the program increased their motivation • 25% say their motivation was unchanged • 85% of teachers had a favorable opinion about the idea of bonus payments on the basis of improvement in student performance • 68% thought that the government should try and scale up this program in all schools • 75% were willing to accept a performance-pay system even under neutrality of the total wage bill • Teachers who show greater support for performance-pay (ex ante) are also likely to have performed better (ex post) • Implications for sorting into teaching profession

20. Summary of Results (Performance Pay) • Incentive schools perform significantly better (0.22 SD) • Improvements are across the board (all grades, districts, baseline scores) • Incentive school score distribution FOSD the control school score distribution • Limited evidence of heterogeneous treatment effects • Children in incentive schools perform better on mechanical and conceptual components of test, and also on non-incentive subjects • No difference between group and individual incentives in the first year – but in the second year, the individual incentives start outperforming the group incentives • Teacher absence does not change, but incentive school teachers report higher levels of teaching activity conditional on attendance • These differences in behavior are correlated with learning outcomes • Much more cost effective than inputs of the same value • Teachers really liked the program – ex ante preference lines up with ex post performance

21. Conclusions/Policy Implications • Performance pay for teachers is likely to be a highly cost-effective policy for improving learning outcomes • 2 years of data suggests unlikely to be a ‘novelty effect’ • Continued gains on both mechanical and conceptual components as well as non-incentive subjects suggests that distortions from multi-tasking are less of a concern in a context of very low levels of learning • Can be largely cost/budget neutral when implemented in the context of an across the board salary increase • The broader point is that of creating a meaningful career ladder for teachers so that their professional trajectories depend on performance (can experiment with including other measures so the weight on test-score gains is not 100%) • Implementation details are critical and any successful scale up will need to build systems and infrastructure to do this

22. Contract Teachers: Experimental Evidence from India (Highly Abbreviated) Karthik Muralidharan University of California, San Diego (with Venkatesh Sundararaman, World Bank) Workshop on Accountability in Education World Bank, 22 June 2009

23. Background • Large scale expansion of primary education in developing countries (MDG’s, EFA, SSA, etc) • Has led to significant increases in access and enrollment • But has also led to difficulties in maintaining and improving school quality • ~60% of 6-14 age cannot read at 2nd grade level though ~95% enrolled • Hiring and deploying enough teachers has been a big challenge • Fiscal difficulties (teacher salaries are ~90% of education spending) • Logistical challenges (the most qualified teachers are less willing to be deployed to underserved areas) • A common response has been to staff unfilled teaching positions with locally-hired contract teachers (not civil-service employees) • Main characteristics of contract teachers include: • Fixed-term renewable contracts – no/limited job security • Typically less qualified and much less likely to be formally trained • More likely to be from the local area (and hired by school committees) • Typically paid much lower salaries • Also more likely to be female (by product - not by design) • Different countries’ contract teacher systems have various combinations of the above, while in India all of these are typical

24. Regular vs. Contract Teachers (in sample)

25. Motivation/Contributions • The use of contract teachers is probably the most prominent policy innovation in primary education in the past 20 years in India and other developing countries • Several countries use or have used some form of contract teachers (Cambodia, Indonesia, Kenya, Mali, Nicaragua, Niger, Togo, etc) • Nearly 25% of teachers in large Indian states of UP, Bihar, MP (Popn. >300M) • This is the stock of contract teachers – the flow is even higher • The use of contract teachers is highly controversial • Supporters: Cost-effective, superior incentives (attendance) • Opponents: Untrained teachers won’t lead to learning, de-professionalizes teaching, reduces prestige of teaching and the motivation of all teachers • Limited evidence on effectiveness of contract teachers • We present experimental evidence on the impact of an additional contract teacher on student learning outcomes • First experimental study of an “as is” expansion of the current contract teacher policy in a large Indian state • Random assignment in a representative sample of schools

26. Summary of Results (Contract Teachers) • The provision of an extra contract teacher significantly improved average test scores in program schools in both years • After 2 years, students in program schools scored 0.16 SD higher on math and 0.10 SD higher on language across all grades • Consistent evidence of some heterogeneous treatment effects • Students in remote schools, smaller schools, and schools with poorer facilities benefit more from the extra contract teacher (these are correlated) • The youngest students (grade 1) benefit the most in both years of the program • But no heterogeneity based on baseline score (students at all levels benefit) • Contract teachers seem to have better incentives than regular teachers on multiple dimensions • Much lower rates of absence and higher rates of teaching activity • Also, contract teachers with lower rates of absence in a given year have a higher probability of having their contracts renewed the next year • We collect data on teachers in private schools in the same districts and find that private school teacher profiles are a lot closer to contract teachers than regular teachers on several dimensions • Suggests that expanded use of contract teachers would move education production closer to the efficient frontier

27. Salary Distribution by School and Teacher Type Percentage

28. Policy Implications (1 of 2) • Expanding contract teacher programs (at the margin) may be a cost effective way of improving learning outcomes in primary education in developing countries • The relevant policy comparison is not one regular versus one contract teacher, but one regular teacher versus several contract teachers • Several developing countries have focused on policies to get highly qualified teachers to move to backward areas • It may be more effective to hire several local contract teachers as opposed to an additional regular teacher – especially for primary schooling • Private schools in rural India mostly follow this model • Expanded use of contract teachers could get schools closer to the productive efficiency levels of private schools while preserving the public characteristics of schools

29. Policy Implications (2 of 2) • Three main concerns about expanding use of contract teachers • De-professionalizing education by promoting use of untrained teachers • Sustainable of such a two-tier system would be problematic • Political economy concerns – once you have enough contract teachers, they will all lobby to become civil-service teachers and the incentive and cost benefits are both lost • One possible solution is to create a career ladder for teachers whereby all new teachers are hired locally as contract teachers, and the best ones become eligible for bonuses/promotions on the basis of performance over a period of 6 – 10 years • Modular training over a period of 3 years over the summers • Continuous measurement of performance during this period • Adjunct versus tenure-track faculty model • Addresses most of the concerns by integrating the two types of teachers into a career track as opposed to creating a permanent cadre of non-professional teachers • Combines our results from both projects/papers