Launch of PISA 2006

1. OECD Programme for International Student Assessment (PISA) Launch of PISA 2006 London, 4 December 2007 Barbara Ischinger Director Directorate for Education, OECD

2. PISA A three-yearly global assessment that… … examines the performance of 15-year-olds in key subject areas as well as a wider range of educational outcomes • Including students attitudes to learning, their beliefs about themselves, and their learning strategies … collects contextual data from students schools, parents and systems to identify policy levers Coverage • Representative samples of between 3,500 and 50,000 15-year-old students drawn in each country • Most federal countries also draw regional samples • PISA covers roughly 90% of the world economy .

3. PISA countries in 2003 2000 2001 2006 2009 1998 Coverage of world economy 83% 77% 81% 85% 86% 87%

4. How PISA works • A strong international network of expertise among the participating countries… • From establishing the assessment frameworks… • The PISA assessments include tasks from more than 40 countries … developing the instruments… • Cross-national and cross-cultural validity … to analysing and interpreting the results • National, regional and international analyses and reports • In-depths country peer reviews … supported by a consortium of the leading research institutions… • ACER, CITO, ETS, NIER, WESTAT … co-ordinated through the OECD in collaboration with other international organisations .

5. Science in PISA 2006PISA defines scientific literacy in terms of an individual’s: • Scientific knowledge and use of that knowledge to… … identify scientific issues, … explain scientific phenomena, and … draw evidence-based conclusions about science-related issues • Understanding of the characteristic features of science as a form of human knowledge and enquiry • Awareness of how science and technology shape our material, intellectual and cultural environments • Willingness to engage with science-related issues .

6. High science performance Average performanceof 15-year-olds in science – extrapolate and apply Qualityin educational outcomes … 18 countries perform below this line Low science performance

7. Mean science scores – OECD countries OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Table 2.1c

8. Comparison of performance on the different scales in science (UK) Scientific competencies Scientific knowledge OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Figure 2.13

9. Gender differences in science performance (UK) OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Tables 2.1c, 2.2c, 2.3c, 2.4c, 2.7, 2.8, 2.9, 2.10

10. PISA Proficiency Levels in Science UK OECD OECD Science Level 6 Student can consistently identify, explain and apply scientific knowledge and knowledge about science in a variety of complex life situations 3% 1% Level 6 11% 8% Level 5 22% 20% Level 4 Science Level 1Student have such a limited scientific knowledge that it can only be applied to a few, familiar situations 26% 27% Level 3 22% 24% Level 2 12% 14% Level 1 Below Level 1Unable to use scientific skills in ways required by easiest PISA tasks. BelowLevel 1 5% 5% OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Table 2.1a

11. Top and bottom performers Large prop. of poor perf. Large proportion of top performers 20 OECD (2007), Learning for tomorrow’s world: First results from PISA 2006, Table 2.1a

12. Investments and outcomes • Since 2000, expenditure per primary and secondary student increased across OECD countries by 39% (in real terms) … … while PISA outcomes generally remained flat… … but there are notable exceptions…

13. Poland raised its reading performance by 28 PISA points, equivalent to ¾ of a school year - What happened? Between PISA 2000 and 2003 Poland delayed the separation of students into different school types beyond the age of 15 years In 2003, performance variation among schools had fallen from 51% to 16% of the variation of student performance But did this lead to genuine improvements of school performance? Between 2000 and 2003 showed the second-largest increase in reading (17 points) and a further 11 point increase since 2003 Most of that increase resulted from smaller proportions at the bottom level (23% in 2000, and three-quarters in vocational tracks, 17%in 2003) Did this harm the better performers? 20 OECD (2007), Learning for tomorrow’s world: First results from PISA 2006, Table 6.1a

14. Mean reading scores – OECD countries OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Table 6.1c

15. Mean mathematics scores – OECD countries OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Table 6.2c

16. High science performance Average performanceof 15-year-olds in science – extrapolate and apply Attitudestowards science … 18 countries perform below this line Low science performance

17. General value of science OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Figure 3.2

18. Personal value of science OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Figure 3.4

19. Student engagement with science Students show strong support for scientific enquiry 93% said that science was important for understanding the natural world 92% said that advances in science and technology usually improved people’s living conditions 75% said that science helped them to understand things around them 57% said that science was very relevant to them personally Students expressed confidence in be able to do scientific tasks, but more so for some tasks than others 20 76% said they could explain why earthquakes occurred more frequently in some areas than in others 64% said they could predict how changes to an environment would affect the survival of certain species 51% said they could discuss how new evidence could lead to a change in understanding about the possibility of life on Mars OECD (2007), Learning for tomorrow’s world: First results from PISA 2006, Table 4.1a

20. High science performance Average performanceof 15-year-olds in science – extrapolate and apply High average performance Large socio-economic disparities High average performance High social equity Equityin educational opportunities Strong socio-economic impact on student performance Socially equitable distribution of learning opportunities Low average performance Large socio-economic disparities Low average performance High social equity Low science performance

21. High science performance Durchschnittliche Schülerleistungen im Bereich Mathematik High average performance Large socio-economic disparities High average performance High social equity Strong socio-economic impact on student performance Socially equitable distribution of learning opportunities Low average performance Large socio-economic disparities Low average performance High social equity Low science performance

22. Student performance PISA Index of socio-economic background Disadvantage Advantage School performance and socio-economic background Finland Student performance and students’ socio-economic background within schools School performance and schools’ socio-economic background Student performance and students’ socio-economic background Schools proportional to size

23. Student performance and migration Native students Second-generation students OECD average = 500 First-generation students PISA 2006: Science Competencies for Tomorrow’s World, Figure 4.2a.

24. Is it all innate ability?Variation in student performance Performance variationacross schools 20 OECD (2007), Learning for tomorrow’s world: First results from PISA 2006, Table 4.1a

25. Is it all innate ability?Variation in student performance PISA 2006: Science Competencies for Tomorrow’s World, Figure 4.1a.

26. Is it all innate ability?Variation in student performance Variation of performance within schools Variation of performance between schools PISA 2006: Science Competencies for Tomorrow’s World, Figure 4.1a.

27. Money matters - but other things do too Some school and system factors

28. School autonomy and standards-based examination on science performanceSchool autonomy in selecting teachers for hire PISA score in science

29. Impact of selected student and school factors on school performance (after accounting for all other factors in the model) School principal’s positive evaluation of quality of educational materials(gross only) Schools with more competing schools(gross only) Schools with greater autonomy (resources)(gross and net) School activities to promote science learning(gross and net) One additional hour of self-study or homework (gross and net) One additional hour of science learning at school (gross and net) School results posted publicly (gross and net) Academically selective schools (gross and net) but no system-wide effect Schools practicing ability grouping (gross and net) One additional hour of out-of-school lessons (gross and net) 20 Each additional 10% of public funding(gross only) School principal’s perception that lack of qualified teachers hinders instruction(gross only) Effect after accounting for the socio-economic background of students, schools and countries Measured effect OECD (2007), PISA 2006 – Science Competencies from Tomorrow’s World, Table 6.1a