Presentation Overview

2. Presentation Overview • ACPP Description • Teacher Views: Research Questions • Teacher Surveys: Methods & Results • Teacher Interviews: Methods & Results • Bonus Amounts Awarded • Teacher Feedback • Conclusions

3. Why try merit pay? • Of all the inputs we can control, teacher quality matters most for student success. • “…a good teacher will get a gain of 1.5 grade level equivalents while a bad teacher will get .5 year for a single academic year.” • “…having five years of good teachers in a row (1.0 standard deviation above average, or at the 85th percentile) could overcome the average seventh-grade mathematics achievement gap between lower-income kids (those on the free or reduced-price lunch program) and those from higher-income families.” • Hanushek, E.A. & Rivkin, S.G. (2004). Similar findings by Sanders, W.L. & Rivers, J.C. (1996)

4. Compensation Reform as Policy Tool The Single Salary Schedule which rewards teachers for credential and experience does not align pay with the outcome we care about most – Student Achievement. Teacher compensation reform focused on pay-for-performance can have motivational (short term) and compositional effects (long range)

5. Achievement Challenge Pilot Project Description (ACPP) • ACPP - Little Rock, Arkansas • Merit pay program for all staff members in a school • Reward personnel solely for increases in student achievement • 3-Year Program • Created in 2004-05 at Meadowcliff Elementary • Expanded to include Wakefield Elementary, 2005-06 • Expanded to include 3 more schools (Romine, Mabelvale, & Geyer Springs), 2006-07

6. ACPP – Well Designed Plan • Straightforward – Based on growth, not complicated statistics • Non-Competitive for Teachers – Everyone could earn • Significant Awards - up to $10,000 for teachers • Focus on Growth of Students – address student placement • Everyone Participated – from secretaries to principals

7. Achievement GAINS! …BUT … An Unqualified Success??? • Evaluation Focus • Two years of modest positive student gains (Winters et al., 2008); (Barnett et al., 2007) • Should we think about other costs and benefits when evaluating a program? • What can we learn from implementation?

8. The Hypotheses We Tested: Proposed Benefits • Possible advantages of merit pay: • Leads to greater innovation in teaching • Leads to teachers working harder • Leads to greater salary satisfaction

9. The Hypotheses We Tested: Proposed Drawbacks • Possible disadvantages of merit pay: 1) Leads to a degraded school climate 2) Leads to an increase in counterproductive competition 3) Leads to the neglect of low-performing students

10. Teacher Views: Research Questions • Teacher Surveys (Compelling Control Groups) • Do teachers in schools with a long-term merit pay (IA) program have differing attitudes and perceptions than teachers in new merit pay schools (LA)? • Do the attitudes and perceptions of teachers in schools with a long-term merit pay program (IA) change over time? • Teacher Interview • Were teachers supportive of merit pay and the ACPP, and did their attitudes and behaviors change as a result of the merit pay program? • Did teachers feel the goals of the ACPP were fair and attainable, and how would they improve the program?

11. Teacher Surveys: Methods • Surveys were administered to teachers in Fall ’06 & Spring ’07 at all five participating schools • Survey participation was voluntary • Teachers responses were measured on a scale of 1 – 4, with a higher score representing stronger levels of agreement with that question • Questions were grouped into constructs that examined attitudes about oft-cited positives and negative associated with merit pay

12. Teacher SurveysComparison of IA vs. LA – Fall 2006 • RQ1: Do teachers in schools with a long-term merit pay (IA) program have differing attitudes and perceptions than teachers in new merit pay schools (LA)? * * * *

13. Teacher Surveys: Comparison of IA vs. IA Fall 2006 to Spring 2007 • RQ2: Do the attitudes and perceptions of teachers in schools with a long-term merit pay program (IA) change over time? • …ATTITUDES PERSISTED AFTER FIRST YEAR

14. Synthesis of Survey Findings

15. Teacher Interviews: Methods • Conducted in Fall 2007 after bonuses were disbursed • Questions explored the same areas as the surveys : • Teacher salary satisfaction was not covered in the interviews • 43 interviews were conducted (40 teachers, 3 principals) at the 5 ACPP schools • Responses were coded as positive, negative, or neutral • A positive response would advocate for performance pay Ex. Do teachers work harder at your school as a result of merit pay? Yes = positive response No = negative response No answer = neutral response • 21 coded as positive, 16 were negative, & 6 were neutral

16. Teacher Interviews: Results • RQ3: Were teachers supportive of merit pay and the ACPP, and did their attitudes and behaviors change as a result of the merit pay program? • RQ4: Did teachers feel the goals of the ACPP were fair and attainable, and how would they improve the program? Highlighted are different findings. *Not evaluated on the survey

17. Teacher Bonus Amounts: 2004-2007 * Other than $0

18. Ways to Improve the Program: Teacher Responses • “I think it should have been consistent across the board. The rules changed.” • “I don’t know if it was a lack of understanding, miscommunication, or it was literally changed after we were told that it would be this way, and then it wasn’t done that way. It was done a different way. That caused a lot of conflict.” • “I like the pretesting in the Fall when they [the students] got here. I like the post-testing at the end of the year because that told me how good of a teacher…or what I needed to work on.” • “I believe that no teacher should have gone without a check.” • “They [teachers] would have to be in on the planning/making. And you would have the exact same discussion with them…what do you think? What do we want? And they would have to have so much ownership of it.” • “Have one set of rules for the entire district. Everything has to be transparent. Our situation wasn’t transparent.”

19. Conclusions: Teacher Views of Merit Pay • Teachers initially supported the ACPP and were somewhat supportive of merit pay as a compensation reform over time. • Teachers did not associate merit pay with being more innovative, and responses varied in regards to working harder. • Teachers did not associate merit pay with counterproductive competition or viewing low-performing students as a burden, but did feel that the school environment became more negative after bonuses were disbursed. • Teachers reported higher salary satisfaction and felt more effective. • Teacher expectations play a significant role in the “success” of a merit pay program.

20. Take away message • Merit pay that ties bonuses directly to student achievement gains can lead to student improvement. • Merit pay does not necessarily lead to divisive competition and nor undermine collaboration. • Participant expectations are very important, so implementers must communicate clearly and openly to set and maintain expectations for participants.

21. Contact Information:Marc HolleyOffice for Education PolicyUniversity of Arkansashttp://www.uark.edu/ua/oepEmail: mjholley@uark.eduPhone: (479) 575-3773