Learning (about) policy: Lessons for policy practitioners

1. Learning (about) policy: Lessons for policy practitioners ArchivedInformation Heather Hill University of Michigan hhill@umich.edu

2. Good news from a researcher (for a change) • Policy can change practice, improve teacher knowledge, and even improve student performance • But only under certain conditions • Policy not as efficient as possible • But we may improve efficiency

3. Mathematics education policy: Two elementary school examples • California Mathematics Frameworks (1980s & 1990s)* • California Mathematics Professional Development Institutes, 2000-2002** *Sponsored by NSF REC# *Sponsored by NSF REC# 0207649

4. Case #1: Learning policy: When state education reform works David K. Cohen Heather C. Hill

5. The policy • 1985 and 1992 California Frameworks calling for more ambitious mathematics instruction • Policy instruments • Documents / frameworks • Curriculum materials – “replacement units” • Professional development • CLAS – policy-aligned assessment

6. The study • Survey of 595 K-5 teachers, 1993-94 • Survey asked about: • Classroom mathematics practices • Professional development experiences • Curriculum use (especially replacement units) • Experience with state assessment

7. The analytic questions • What were teachers doing? • What led to teachers “doing” policy? • What was the effect on student achievement?

8. Findings: Learning Policy • Teachers balanced their practice • Teachers who reported more policy-aligned practice: • Were using “replacement units” • Had attended professional development around those replacement units • Had used the state assessment as an opportunity to assess their own teaching and students’ learning

9. Findings: Learning Policy • Teachers reporting more conventional practice: • Had gone to “special topics” workshops • Were using conventional mathematics texts • Did not report using the assessment as an opportunity to revise their classroom practices • May or may not have read official policy documents

10. Effect on student achievement • Student achievement on CLAS higher in schools where teachers learned about and used replacement units • Student achievement on CLAS higher in schools where teachers reported learning from test

11. Case #2: Mathematics Professional Development Institutes (MPDIs)

12. The policy • MPDIs part of broader policy designed to improve content knowledge in reading and mathematics via professional development • Cost: $40 million • Multiple sites governed by University of California Office of the President

13. At elementary mathematics sites • Instructors: Mathematicians and mathematics educators • Focus was squarely on mathematics content • 40-120 hours of professional development during summer + follow-up

14. Hill-Ball study • At K-6 Elementary Number and Operations sites • Designed measures of teachers’ knowledge for teaching elementary mathematics • Pre/post assessment, summer 2001 data

15. MPDI teacher growth • For all institutes for which we have data, teachers gained roughly one-half standard deviation • Translates to 2-3 item increase on assessment • Considered substantial gain

16. Results from sample sites

17. Improving teacher knowledge • Length of institute predicts teacher gains • 120-hour institutes most effective, on average • But some 40-hour institutes very effective • Focus on mathematical analysis, proof, and communication leads to higher gains

18. Cross-case conclusions • Policy can change practice, improve teacher knowledge, and even improve student performance • Policy itself has little traction • Instruments of policy do have traction • But only if they are sites for learning • Effort is large, effect sizes small

19. Cross-case conclusions • Policy is not as “efficient” as possible • e.g., MPDI sites where teachers did not learn • But we can improve efficiency • Carefully design professional development itself • Grounded in classroom practice, such as curriculum materials, video/lesson study, study of the mathematics teachers use; long • Carefully design system of professional development • Assure coherence of system • Curriculum materials, in particular • Support continued, rigorous research