Adrienne Alton-Lee, PhD, Dip Tchg (Distinction) Iterative Best Evidence Synthesis Programme

1. The Use of Evidence to Improve Education and Serve the Public Good? University of Cambridge1 November 2012 Adrienne Alton-Lee, PhD, Dip Tchg (Distinction) Iterative Best Evidence Synthesis Programme Hei Kete Raukura Ministry of Education New Zealand www.educationcounts.govt.nz/goto/BES

2. Beyond Plato’s Myth of the Metals’ the role of teachers as one of maintaining inequalities in society by educating children differently gold, brass or iron education for gold, iron and brass children Sorting via education as a peaceful alternative to a military regime in maintaining differential status and access to material wealth in society

3. What works for valued outcomes for diverse (all) learners? • Why? • How? • What doesn’t work? First do no harm in education • What makes a bigger difference in areas of need?

4. Fit-for-purpose methodology • Influences on valued student outcomes • Trustworthy bodies of evidence • Rigorous eclecticism – iterative process • Comparative magnitude of impact • Effect sizes – impact & equity index • Critical role of theory • Case & vignette • Context matters

5. Inquiry & Knowledge Building Cycles

6. Early Best Evidence Synthesis iterations • The complexity of community & family influences e.g. • Poverty • Pre-school access to experiences • Undiagnosed hearing loss • Family interactions • Television • Effective school-home partnerships

7. Collaborative Knowledge Building & Use for Ongoing Improvement

8. Primary Teachers’ Union NZEI Te Riu Roa welcomes the opportunity to comment on this Best Evidence Synthesis. Drawing widely and systematically from national and international research on social sciences education, its authors have sought evidence of what works, for which students, and in what circumstances. The synthesis of findings contributes to our understanding of the relationship between pedagogy and outcomes and the importance of context. Teachers will welcome the many practical ways in which to strengthen practice – a particular feature of the BES Programme. National President, New Zealand Educational Institute, 2008.

9. Secondary Principals’ Association The Leadership BES already has significant traction in New Zealand secondary schools and is well regarded by the profession as being both aspirational and practical in content. We are proud to have been involved with it from the beginning and commend it to you as a well-researched, clear and detailed way forward for leaders at any level of the schooling system. We hope it gives principals in particular a focus for their work as well as being a useful tool to help us all find ways to improve student outcomes. President of the New Zealand Secondary Principals’ Association, 2011. 12,472

10. School Trustees’ Association (School Governance) • ...the final document reflects only a small proportion of the influence that this work has had for those of us who have been involved in its development. The process of developing the BES has triggered new learnings. It has built stronger links, within and across the sector, between academics and practitioners, and it has provided springboards for new initiatives in leadership...best of all, we found that ‘unpicking each section , as we reviewed it, raised questions in our own minds and those of our colleagues and helped us to think in a more disciplined way about what matters; our students, and how leadership contributes to enhancing their achievements... • President, Manager Training and Development & Manager, Service Delivery, New Zealand School Trustees Association

11. School Leadership & Student outcomes: Identifying what works and why: Best evidence synthesis

13. Backward mapping • Ensure administrative decisions are informed by knowledge about effective pedagogy • Engage in constructive problem talk • Engage in open-to-learning conversations • Analyse and solve complex problems • Select, develop and use smart tools • Build relational trust • Create a community that learns how to improve student success

14. Formative Quality Assurers’ warning Professors Ben Levin & Michael Fullan • We believe that any school leader, or system leader, or anyone with an interest in education improvement or leadership, will find this report stimulating and valuable. Certainly both of us did. • Having such a high quality review is an important accomplishment, but it is not enough…The challenge for all partners in New Zealand (and beyond) will be to make sure that the lessons and implications of this review leap off the pages and become part of the fabric of education in the country. • The BES report on leadership will be for nought unless there is a concerted plan to develop the core capacities of effective leadership in all New Zealand schools. This for us would not mean merely deriving a plan from the findings, but rather taking a concrete problem, such as raising the bar and closing the gap in literacy in New Zealand schools and incorporating the key leadership capacities into the implementation of the very problem to be addressed. It is always better to start with the concrete and incorporate what is needed to make specific improvements.

15. 14,145 Teacher Professional Learning & Development BES • Timperley, H., & Alton-Lee, A. (2008). Reframing teacher professional learning: An alternative policy approach to strengthening valued outcomes for diverse learners. Review of Research in Education 32, 328 – 369. • Much teacher professional development has no effect on valued student outcomes • Some professional development has negative effects • A comparative magnitude of impact analysis shows some outstanding professional development can lift achievement for all and accelerate progress of lowest 20% of achievers 3-6 times the rate of business-as-usual practice • http://www.ibe.unesco.org/en/services/online-materials/publications/educational-practices.html

18. A Perspective from the Auditor General • 3.21 Various stakeholders told us that the Ministry did not consistently base itsdecisions about funding and providing professional development initiatives onthe evidence it has. • 3.22 In our view, it would be helpful for the Ministry to review the professional development initiatives it funds against its BES evidence and any other relevantevidence on effective professional development. • Office of the Auditor General 2008

19. Quality teaching for diverse (all) learners • www.educationcounts.govt.nz/goto/BES • See ‘Overview of Findings’ Cross-curricula • ‘The use of evidence to improve education and serve the public good’ 19,276 copies requested in NZ 2500 schools To 140 countries from UNESCO

20. 10,221 copies requested in NZ Effective Pedagogy in Social Sciences

23. High Impact Pedagogies for Multiple Valued Outcomes • Highly effective pedagogies in areas of need • Approaches that simultaneously advance a range of valued outcomes – cognitive, well-being, social… • Developed through several iterations of collaborative research & development • Disciplined innovation by the profession as a resource for improvement

24. BES Exemplars • Response to feedback from the profession • Explain the complexity of teaching and the how • Use of student and teacher ‘voice’ • Not just teaching - professional learning and leadership supports • An inquiry and knowledge building approach to improvement • Implementation alerts • Access to trusted resources, smart tools, DVDs, websites • Access to Research Behind BES Think alouds

25. High Impact PedagogiesAccelerated improvement in areas of need for underserved or disadvantaged learners • BES Exemplar 1: Developing Communities of Mathematical Inquiry Dr Roberta Hunter • Students are scaffolded to engage with the teacher and peers in mathematical inquiry, reasoning & argumentation • 4-5 years of achievement acceleration in one year • ‘Don’t ‘dis her, man, when she’s taking a risk.”

26. BES Exemplar 1: Developing Communities of mathematical inquiry

27. Bes Exemplar 5 Learning Logs – Jennifer GlennThames high school • Teacher feedback • Then – students read and respond to these questions…. • What do you think I’ve said about your writing? • How do you feel about the outcomes/comment? • Set three goals for the next similar piece of writing? • Wendy: ‘I like using this blue book [learning log] because it makes you really think about what you need to improve • John: ‘This is the best thing that ever happened to my writing’ • Brian: ‘We cover things in class that are much more useful to me and everyone else’

28. BES exemplar 5 Learning Logs

29. Learning Logs in Physical Education Xanthe Sulzberger, Aquinas College • Over time, the emails became electronic “learning logs” that provided evidence of learning and resources for ongoing improvement, for me as well as the students. They provided me with rich information as to how much the students understood, and what I may have to re-address in class. I could see whether questions needed to be re-framed or concepts needed to be re-taught. This sometimes meant that I had to adapt my questioning or how I presented the question. The interaction became a two-way process, with students asking me for clarification that they may not have asked during class time’ • We also feel more involved as parents from reading the logs, and it has led to some great discussions around the dinner table with the family all being interested in the topic. This all contributed, I believe, to our son getting his first Excellence in a written assignment.” • Parent of student in Xanthe’s class • Xanthe Sulzberger • Physical Education Achievement Standard 2.2: The Badminton Smash Shot

30. Student feedback Xanthe’s Use of Learning Logs

31. Educational Policy, practice & Evidence use • Meta-analyses • ES 0.53 0.54 0.59 for cooperation • ES 0.36 0.23 for competitive or individualistic efforts Stanne et al (1999); Hattie (2009) • In comparison with schooling practices that are often supported by governments – such as tutoring, technology use and school restructuring – co-operative learning is relatively inexpensive and easily adopted. Yet, thirty years after much of the foundational research was completed, it remains at the edge of school policy. This does not have to remain the case: as governments come to support the larger concept of evidence-based reform, the strong evidence base for co-operative learning may lead to a greater focus on this set of approaches at the core of instructional practice. In the learning environments of the 21st century, co-operative learning should play a central role. (Slavin, 2010, p. 174). • Galton, M., & Hargreaves, L. (2009). (Eds). Group work: still a neglected art? Cambridge Journal of Education, 39(1) 1-6.

32. The New Frontier: The Use of Evidence in Policy • “Research evidence is perceived as “softly spoken” because, “empirical inquiry simply cannot make its voice heard amidst the clatter of other, political imperatives on policy making” (Pawson, 2002).

33. Research into how evidence is used in policyCoburn, Tourē & Yamashita (2009) • A strong tendency to discount evidence when it does not support the beliefs decision-makers hold • Orchestrated consensus to over-ride action on the available evidence • Extent of use of evidence depends on individuals with positional authority • Evidence brought to the resolution of disputes ‘rarely addressed assumptions about high quality instruction or about how children learn’ • In times of decision-making under pressure and/or contracting resources ‘decision-making trajectories became more interrupted, last minute and symbolic’ • ‘symbolic’ = the use of evidence to give legitimacy to a policy position rather than to specifically guide improvement efforts

34. Policy for the Use of Evidence in Educational Policy • Little research on linking research to policy in education • Passive dissemination of research is generally ineffective whilst multi-faceted interventions targeting multiple barriers more effective • Recommendations e.g. • Knowledge, awareness and skills capacity building in all parts of the research evidence production-to-use system • Policy decisions to develop evidence informed policy in education • Increasing capacity in research on research generation and use

35. System-wide Improvement in education: Ben Levin • Ben Levin is Canada Research Chair in Education Leadership and Policy at the University of Toronto. • As the Ontario Deputy Minister of Education he helped lead an effort that substantially lifted achievement, and reduced inequity across 5,000 primary and secondary schools in six years. • This reform enabled a shift from 55% to 70% of elementary students achieving high levels of literacy and numeracy while committed to a broad, not narrowed, curriculum. • High school graduation rates increased from 68% to 82%. ‘You won't get any closer to a succinct and cogent set of insights on how to achieve “whole system reform” than this booklet on “System-wide improvement in education”. No one integrates inside government reality and external research and evidence like Ben Levin.’ Michael Fullan, Professor & Special Adviser on Education to the Premier of Ontario.

36. An effective, system-wide change strategy requires the following elements: • 1. A small number of ambitious yet achievable and well-grounded goals, publicly stated • 2. A positive stance on improving all schools and success for all students • 3. An emphasis on capacity building and a focus on results • 4. Multi-level engagement with strong leadership and a ‘guiding coalition • 5. Continuous learning through innovation and effective use of research and data • 6. A focus on key strategies while also managing other interests and issues • 7. Effective use of resources • 8. A strong implementation effort to support the change process.

37. Use Evidence to Improve Education and Serve Public Good

38. Use Evidence to Improve Education and Serve Public Good