Innovation Around the World: Year 1 evaluation results from the Microsoft Innovative Schools Program

1. Innovation Around the World:Year 1 evaluation results from the Microsoft Innovative Schools Program Linda Shear, SRI International 18 March, 2009

2. Education in the 21st century “Success in the slowly changing worlds of past centuries came from being able to do well what you were taught to do. Success in the rapidly changing world of the future depends on being able to do well what you were not taught to do.” —Papert & Caperton, 1999

3. In the next hour, I will: • Introduce SRI International • Introduce the Innovative Schools Program • Describe our methods for this distributed, global evaluation • How can we take a coherent look at innovation around the world? • Present some results from the first year of the evaluation

4. Introducing the Innovative Schools Program Program goals: Help schools through “holistic school reform” with a “locally tailored roadmap,” in order to… Improve teaching and learning, in order to… Give students the skills they need to be successful contributors in tomorrow’s workforce and communities 12 pilot schools representing 12 countries The schools are united by the goal of improving education for the 21st century… … but they vary widely in their specific designs, approaches, and local contexts.

5. Where are the schools? • Sao Paulo, Brazil • Ontario, Canada • Santiago, Chile • Oulu, Finland • Amiens, France • Munich, Germany • Sheung Shui, New Territories, Hong Kong • County Meath, Ireland • Hermosillo, Sonora, Mexico • Doha, Qatar • Nacka, Sweden • Huyton, Knowsley, UK This map was created by students at Ecole Chateaudun, the Innovative School in Amiens, France.

6. School characteristics

7. Brazil: Instituto Escola Lumiar • Founded in 2003 as a private school; 70 students • Staff of part-time tutors and masters instead of a traditional teaching staff • Tutors focus on students’ personal and academic growth • Masters focus on helping students develop required competencies • Student work is project-based and driven by student interests • Instituto Escola Lumiar is implementing a “Digital Mosaic” learning management system to map student learning by competencies

8. Hong Kong: Fung Kai Innovative School • Primary school in Sheung Shui • Goals: prepare students for the 21st century and help them be more active learners • In partnership with a publisher, teacher teams are designing new digitized curriculum in English and Maths • A portable computer for every student, with lessons available online

9. Ireland: Dunshaughlin Community College • Public secondary school with a history of use of technology, located in the Dublin commuter belt • Developing a framework for teacher professional development that will support teachers to: • Use digital tools in the classroom, even within a very traditional required curriculum • Support development of 21st century learning and values

10. Qatar: Al Bayan Independent School for Girls • Independent school following the American charter school model, but serving a Qatari population • Al Bayan aims to: • Empower Arab women • Raise the level of professionalism for teachers • Collaborate with parents and other partners • Integrate technology with educational practices

11. Finland: Ritaharju School • This new school is still in the planning stages; expected to open for the 2010-2011 school year • Plans include: • Laptops for all students • Project-based and student-centered learning • Learning in both formal and informal environments • Small learning communities within the school • Close collaboration with the community • New approaches are currently beingpiloted in schools throughout the townof Oulu

12. Country context makes a difference • Each school’s reform is shaped by a specific local context of education • Educational systems range from traditional to progressive, with varying degrees of school autonomy

13. Supports provided by the Innovative Schools Program • The schools: • Attend twice-yearly in-person meetings, at which they access outside expertise and discuss reform both within and across country teams • Participate in “Virtual Universities” to discuss their process and challenges with one another and to hear the ideas of educational experts such as Michael Fullan • Receive counseling from an assigned mentor • Are supported by a Microsoft Academic Program Manager, dedicated to half-time work with the Innovative Schools Program • Are guided by tools for reform, such as the “6i Process” and “Innovation Framework” which provide them with frameworks for making decisions about school reform

15. About the evaluation • A coordinated global evaluation design, implemented locally by an international network of evaluation partners • This design combines in-depth localized research with cross-cutting themes, issues, and trends • Evaluation questions: • Do schools participating in the Innovative Schools Program increase their provision of 21st century learning opportunities for their students? • Does the provision of 21st century learning opportunities occur without sacrifice to student performance on more traditional measures of achievement? • What role does technology play in changing classroom practice?

16. The global evaluation team

17. Evaluation methods • Data collection and reporting within each country: • Interviews, classroom observations, and focus groups at each school • Thematic case studies of 6 selected schools • Collection of locally-relevant achievement data, with appropriate local comparisons • Analysis of samples of learning activities and student work collected from teachers at each school • Global synthesis of country-level reports to tell a coherent story across countries

18. Evaluation methods • Data collection and reporting within each country: • Interviews, classroom observations, and focus groups at each school • Thematic case studies of 6 selected schools • Collection of locally-relevant achievement data, with appropriate local comparisons • Analysis of samples of learning activities and student work collected from teachers at each school • Global synthesis of country-level reports to tell a coherent story across countries

19. What do we mean by “21st-century skills”?Skills that will prepare students for a world with . . . • A global employment market • Less stable employment relationships and the need to prepare for multiple careers over one’s lifetime • Accelerating shift in job market toward high-skill, high-service, and high-creativity employment • Ever more capable ICT and vastly increased access to information--and to misinformation • Global challenges such as economic instability, climate change, and limitations on resources such as fresh water and petroleum

20. Why look at learning activities and student work? • Things like “21st-century learning opportunities” are easy to talk about but hard to implement • … and just as hard to measure • Looking at instructional artifacts (teacher assignments, student work) helps us see what’s really happening in classrooms • With this method, progress toward instructional change can be measured across educational contexts, subject areas, and grades

21. How it works • Evaluators collect samples of the assignments teachers give and the work that students do in response • Evaluators recruit and train experienced teachers to act as independent coders within each country • Teachers of the same subjects/age levels as the teachers from whom we collected learning activities, but from different schools • This workshop can be a valuable professional development opportunity for the coders

22. Coding follows a common framework • This project developed rubrics and definitions that make “21st century learning opportunities” concrete and measurable • Coders analyze learning activities in 5 “dimensions,” looking for evidence that they encouraged students to develop skills in these areas: • Knowledge construction • Collaboration • Problem-solving and innovation • Global tools and perspectives (including use of IT) • Self-regulation

25. Analysis of student work • The same coders analyzed student work done in response to the learning activities, using rubrics to look for evidence of: • Knowledge construction • Problem-solving and innovation • Global tools and perspectives (including use of IT) • Skilled communication • The results (plus some complex statistical methods) allow us to paint a picture of the “21st-century-ness” of teaching and learning, and how that changes over time

26. Findings from Year 1 of the evaluation • Organizational learning and change processes in Innovative Schools • School strategies for creating improved learning environments • Measures of teaching, learning and assessment in Innovative Schools

27. Implementation process and strategies • Reform can follow a number of paths: • Whole-school • Staged by grade, by subject, by teacher • Top-down or bottom-up • For whole-school reforms, clarity of vision is essential • Some schools that used an early adopter strategy report that many teachers who were hesitant gained confidence in reforming their practices by seeing new ideas work well for their colleagues In Ireland, the reform is designed to let teachers opt in when they’re ready and make changes at their own pace.

28. Teacher learning communities • Many Innovative Schools are working to develop collaborative professional environments • Peer-to-peer coaching, mentoring, use of master teachers, cross-disciplinary collaboration • Teachers in many Innovative Schools report positive changes • What makes it hard: • Creating time for collaboration • Rearranging work schedules, providing release time, deliberate overstaffing • Creating processes to make sure the time is well-used • Building comfort, trust, and a supportive culture “The cooperative and collaborative environment implies a shared responsibility… it breaks our schemes and represents a crisis for some teachers.”

29. Extended learning communities • Student involvement in the school learning community • Asking students to review planned coursework or make recommendations to teachers • In the UK, students give input on the iterative design of new learning spaces • Parent and community engagement • In Sweden and other countries, schools are using a web portal to communicate with parents. • An international community of schools • Innovative Schools conferences provide a regular forum • Some schools have arranged visits to each other to see reform in action, or conducted joint learning projects

30. Other aspects of schoolwide culture • Professional development opportunities for teachers • Nearly all the schools provide training on basic technology skills • Only some focus deeply on how to integrate technology with teaching and learning—teachers feel they need more help with this • Developing a culture of self-evaluation • Regular reflection on what’s working and what can be improved is early in development at most schools Canada’s Literacy@School program trains teachers on literacy teaching techniques, technology use, and how they work together. Fung Kai Innovative School regularly surveys teachers, parents, and students.

31. Infrastructural changes (a few among many) • New and more flexible learning spaces • Adding technology – hardware, software, and supports • Promoting network reliability At one school, teachers avoid taking the risk of using technology because they fear it will stop working mid-lesson.

32. School strategies for creating better learning environments • Schools are hoping to create innovative learning environments characterized by: • Project-based learning • Group work and collaboration • School work that is relevant to students’ lives • Student autonomy and self-regulation • New forms of student assessment • Use of technology as a tool for learning

33. Project-based learning • An extended activity in which students seek the answer to a question or create a complex product • Can allow students to pursue their own interests and questions, make decisions, and be active learners • At the Innovative Schools, projects usually involve students creating a product, such as a documentary or presentation • Some schools are being creative at finding opportunities to embed extended projects into a rigorously-structured curriculum Students created a documentary on living conditions in Sao Paulo, or researched how to save electricity at home and then implemented their solutions.

34. Group work and collaboration • Group work is common at the Innovative Schools • Collaborative activities vary in their sophistication • The most common form of group work is students working together for a brief period on a particular activity • A more sophisticated form involves students produces complex, interdependent group products that take more than a single class period to develop • Technology can enable collaboration between students • Assessing group work is often a major challenge, especially in countries that require an individual grade for every assignment

35. Increasing relevance of schoolwork to students’ lives • Teachers in Innovative Schools used a number of strategies to increase the relevance of schoolwork to students’ lives: • Incorporating students’ interest and cultural background • Teaching knowledge and skills students can apply to everyday life • Anchoring learning in students’ prior knowledge or experiences • Involving outside partners such as museums to make learning more realistic • Incorporating technology, similar to students’ everyday lives • When asked how their schoolwork would serve them later in life, students particularly noted the importance of technology for their future careers

36. Student autonomy and self-regulation • Planning and monitoring one’s own activities is an important skill for 21st century workplaces • At four pilot schools, student revision of their own work appears to be a regular part of classroom practice • Technology can facilitate the feedback and revision process and motivate student attentiveness to quality “I think our students need to explore and construct knowledge... In this learning process, students not only learn the subject knowledge but also the methods of learning how to learn.”

37. New forms of student assessment • Traditional assessments pose a challenge to schools trying to move toward new ways of teaching and learning • New ways to assess students: • A “Digital Mosaic” which will record individual students’ competencies and track growth over time (Brazil) • Rubrics to assess students’ development of cross-curricular “PLTs” (Personal, Learning, and Thinking Skills, UK) • Efforts to develop new assessments are generally in their early stages “To a vast extent, these [grading] regulations demand reproduced and comparable knowledge. This is why I simply can’t give a student credit for creativity, because this is not part of the grading regulations for exams.”

38. Use of technology as a tool for learning • Technology is a big focus area at Innovative Schools • Progress in increasing use of technology has varied • At one school, a teacher said: “The computer was only used for recreation and research a year ago. Now students produce more with technology. The Smartboard is used everyday.” • In Hong Kong, students in some grade levels have portable computers. Some other schools are planning to implement 1-to-1 computing. • Use of technology has not been transformative at all schools • Teachers tend, at first, to use technology as a replacement for existing tools (e.g., PowerPoint instead of the chalkboard) rather than as a means of changing pedagogy • Teachers felt they needed more support to integrate technology into teaching practice

39. Measures of teaching, learning and assessment • To evaluate schools’ progress in changing learning environments, we observed classes and analyzed samples of learning activities and student work, and interviewed teachers and students • These data are from the first year of the program, and schools are starting from very different places • Therefore, we expect results on these measures to vary as well

40. Class observations • Evaluators visited 4 to 8 classes in each school, • Three humanities classes • Three science classes • Two other classes that the school felt best represented their innovation efforts • In total, evaluators visited 65 teachers in 73 classes visited (some classes were visited more than once) • Evaluators used an observation protocol to allow comparison across observations

41. Innovative teaching practices • In class observations, evaluators looked for evidence of 10 research-based innovative teaching and learning practices: • Students gave feedback to peer or received feedback from peer/teacher • Students had opportunities to connect learning to the real world • Students revisedtheir own workbased on feedback or self-assessment • Students work on extended/in-depth projects • Students were prompted to assess their own learning • Students had choicesabout toolsor resources for learning • Students had choices about topicsof learning • Students had opportunities to develop cross-cultural understanding • Students engaged in performance assessments or portfolio assessments • Students were exposed to issues related to global interdependency

42. Innovative teaching practices

43. Innovative teaching practices • Two practices were seen in a majority of classes: • Students received feedback on their work • Students had opportunities to relate what they were learning to the real world • Seen much less frequently: • Students revised their work based on feedback • Students work on extended projects • Students make choices about content/tools There is room for teachers to experiment with a wider variety of innovative types of instruction.

44. Student use of technology

45. Use of technology • In about half the observed classes, students used technology to support their work • In classes where students used technology, more of the innovative teaching and learning practices were also present • This relationship was particularly strong in classes where students were using technology for higher-level purposes such as analyzing data or designing a multimedia product Student use of technology is associated with learning environments that are more innovative, particularly when students are using technology in ways that support higher-level learning.

46. Learning activities and student work

48. A typical learning assignment at a Year 1 Innovative School: • Required some knowledge construction, but that was not the main focus; most of the activity or assignment could be completed through reproduction of information that students had read or heard. • Allowed students to work in groups but did not require it; students produced individual products rather than products that are interdependent. • Required problem solving but did not give students choices about how to address the activity or require them to implement their solution in the real world. • Required only one aspect of global tools and perspectives (use of technology, multiple academic disciplines, or information or perspectives from multiple cultures or countries). • Required only one aspect of self-regulation (extended duration with multiple parts, assessment criteria given to students in advance, or receipt of feedback in time to revise their work).

50. Learning activities • The learning activity codes were similar for humanities and science assignments • There was large variance between the countries; some appear to be further along in encouraging students to develop 21st century skills than others • Learning activities that involved technology were more like to encourage students to develop other 21st century skills, especially collaboration and self-regulation • Similar to findings from classroom observations