PISA Scientific Literacy: What’s it all about? - Declan Kennedy

1. PISA Scientific Literacy: What’s it all about?- Declan Kennedy Second National PISA Symposium 5 April 2005, Dublin.

2. In this talk….. 1. Look at the background developments in science education that gave rise to scientific literacy as defined in the PISA project. 2. Outline how scientific literacy is examined by the OECD in the PISA project 3. Report on the performance in science of 15-year-old students in Ireland in PISA 2003.

3. The term “literate” is no longer used solely in the context of having the ability to read and write. The term now carries the general meaning of being able to engage effectively with different aspects of modern life. • In general terms, being “scientifically literate” means having the knowledge and skills that are needed by everyone in society, not just those who will be specialists in science or make a career in science. • The emphasis in scientific literacy is not on mastering the details of a body of knowledge but on “having and being able to use a general understanding of the main or key ideas in making informed decisions and participating in society” (Harlen, 2001).

4. PISA Definition of Scientific Literacy Scientific literacy is defined in PISA as the capacity to use scientific knowledge, to identify questions and to draw evidence-based conclusions in order to understand and help make decisions about the natural world and the changes made to it through human activity. (OECD 2003, p. 133)

5. Where does the OECD definition of scientific literacy come from? The concept of scientific literacy has now been embedded in the vocabulary of science education throughout the world as a result of the STS movement.

6. Over the last twenty five years or so, a wide range of science teaching resources using everyday applications and contexts have been developed. • Teaching approaches which use contexts and applications as the starting point for developing the understanding of scientific ideas are commonly referred to as STS (Science-Technology-Society) approaches or "context-based” approaches

7. STS materials and courses aim to develop pupils’ scientific literacy through one or more of the following • developing an understanding of what is meant by science and technology, and how they relate to each other. • developing an understanding of the way scientists work;

8. developing an understanding of the ways in which science and technology affect society, including environmental issues, ethical issues and the economic and industrial aspects of technology; • developing an understanding of the nature of science • promoting the discussion of personal opinions and values.

9. Arguments for inclusion of STS topics in science courses 1. “Citizen science”: people need to know something about science which will help them to think and act appropriately on scientific matters which may affect their lives and the lives of other members of the local, national and global community;

10. 2. “Relevant science”: science which emphasises applications rather than abstractions is more likely to foster interest in pupils; 3. “Added value”: approaches to science teaching which include decision-making and problem-solving are likely to enhance students’ more general skills in these areas.

12. STS Movement driven by • Feeling of technological inferiority after Sputnik. • Science teachers dissatisfied with science syllabi.

13. “The problems of science and technology in a world of human communities add a great deal of colour, human interest and student involvement to science lessons. Once teachers see this in the eyes of their students, the battle for change is won. Teachers know the worth of the STS approach when they know how well it teaches science to their students” - Joan Solomon (p. 10)

14. “Project synthesis” analysis (1981) “….a new challenge for science education emerges. The question is this: Can we shift our goals, programs and practices from the current over-whelming emphasis on academic preparation for science careers for a few students to an emphasis on preparing all students to grapple successfully with science and technology in their own, everyday lives, as well as to participate knowledgeably in the important science-related decisions our country will have to make in the future?” - Harms and Yager (p. 119)

15. National Science Teachers Association USA Policy Statement (1982) “The goal of science education during the 1980s is to develop scientifically literate individuals who understand how science, technology, and society influence one another and who are able to use their knowledge in their everyday decision-making…. This individual both appreciates the value of science and technology in society and understands their limitations”

16. STS materials produced by Association for Science Education UK, e.g. SATIS programme (1988) (“Science and Technology in Society”)

17. ICASE (International Council of Associations for Science Education) philosophy on STS and publications with UNESCO. • NSTA policy statement (1991) “STS is the teaching and learning of science/technology in the context of human experience”. • “Science for all” – Fensham (1985) and Black (1993) • Salters Science – UYSEG (University of York Science Education Group)

19. International Dimension • USA (R Yager), • Canada (G Aikenhead) • UK (Solomon, Waddington, A Hunt, ASE, et al) • Australia (P Fensham, G Giddings, etc.) • Japan (N Nagasu) • Developing Countries (Holbrook et al) • Netherland (H Eijkelhof) • Ireland (S O Donnabhain et al., NCCA syllabus committees in Physics, Chemistry and Biology)

20. In Ireland we are well ahead at Leaving Certificate level in the movement to embrace STS in our teaching as STS topics are specifically written into the new Leaving Certificate syllabi in physics chemistry and biology in order to show young people the links between the science they study in school and their everyday lives. The position at Junior Certificate level is unclear pending the publication of Teacher Guidelines.

22. Why STS approaches are being adopted 1. A concern by those involved in science education about the seeming irrelevance for their students of much of the material being used in science lessons. 2. A widely held concern in many countries over the low levels of uptake of science subjects, particularly the physical sciences, at upper secondary level. 3. A concern over science courses provided for “non-science specialists”. (Bennett 2003)

23. At the lower secondary school level, STS materials have two main aims: 1. To show young people that science can help them understand and explain things which are going on around them in their everyday lives. 2. To foster an interest in science among young people in the hope that they will want to continue with their study of science beyond the compulsory period.

24. There is strong evidence in the literature of the motivating effects of STS approaches to teaching. (Bybee, 1985; Fensham, 1988) • Many teachers find that beginning a topic or a lesson with some form of story which leads into the science to be covered helps engage the interest of the students, e.g. the story of Archimedes when introducing the concept of density.

25. This document makes ten recommendations for science education in the 21st century. Recommendations 5 and 6 are of particular relevance to the area of STS.

26. Recommendation 5: Work should be undertaken to explore how aspects of technology and the applications of science currently omitted could be incorporated within a science curriculum designed to enhance “scientific literacy”. • Recommendation 6: The science curriculum should provide young people with an understanding of some key ideas-about-science, that is ideas about the ways in which reliable knowledge of the natural world has been, and is being, obtained.

28. Scientific approach to enquiry “In order to understand the major explanatory stories of science, and to use this understanding in interpreting everyday decisions and media reports, young people also require an understanding of the scientific approach to enquiry. Only then can they appreciate both the power and the limitations of different kinds of scientific knowledge claims………” (p. 19)

29. “Practical work in a school laboratory can provide contexts for learning some of these ideas-about-science. But, in our view, it cannot provide all that is required. For instance, case studies of some historical and contemporary issues involving science will also be necessary so that pupils can improve their appreciation and understanding of the complex relationships between evidence and explanation, and the complexities of applying scientific knowledge in real-world situations” (p 20)

30. “Pupils should also become familiar with stories about the development of important ideas in science which illustrate the following general ideas: • That scientific explanations ‘go beyond’ the available data and do not simply ‘emerge’ from it but involve creative insights (e.g. Lavoisier and Priestley’s efforts to understand combustion)”.

31. Assessment of Scientific Literacy in the PISA Project • “scientifically literate” means having the knowledge and skills that are needed by everyone in society. • “having and being able to use a general understanding of the main or key ideas in making informed decisions and participating in society”. (Harlen 2001)

32. To transform this definition into an assessment of scientific literacy, three broad dimensions of scientific literacy have been identified for the PISA study (a) Scientific knowledge or concepts, i.e.the scientific knowledge and conceptual understanding that are assessed by application to specific subject matter.

33. (b) Scientific processes or skills: the mental processes that are involved in addressing a question or issue, e.g. identifying evidence or explaining conclusions. (c) Situations or Context in which the knowledge and processes are assessed and which take the form of science-based issues e.g. the personal context of health and nutrition, the global context of climate.

34. (a) Scientific knowledge or concepts 1. Structure and properties of matter (thermal and electrical conductivity) 2. Atmospheric change (radiation, transmission, pressure) 3. Chemical and physical changes (state of matter, rates of reaction, decomposition) 4. Energy transformations (energy conservation, energy degradation, photosynthesis) 5. Forces and movement (balanced/unbalanced forces, velocity, acceleration, momentum) 6. Form and function (cell, skeleton, adaptation)

35. 7. Human biology (health, hygiene, nutrition) 8. Physiological change (hormones, electrolysis, neutrons) 9. Bio-diversity (species, gene-pool, evolution) 10 Genetic control (dominance, inheritance) 11. Ecosystems (food chains, sustainability) 12. The earth and its place in the universe (solar system, diurnal and seasonal changes) 13. Geological change (continental drift, weathering)

36. (b) Scientific processes for assessment in PISA Process 1: Describing, explaining and predicting scientific phenomena. Process 2: Understanding scientific investigation. Process 3: Interpreting scientific evidence and conclusions

37. (c) Situations or context 1. Science in life and health Health, disease and nutrition Maintenance of and sustainable use of species Interdependence of physical/biological systems 2. Science in Earth and environment. Pollution Production and loss of soil Weather and climate 3. Science in technology Biotechnology Use of materials and waste disposal Use of energy Transportation

38. PISAWarsaw Meeting 2004

39. Two Main Decisons 1. In PISA 2006, when science will be the major domain examined, 65% of the items in PISA 2006 should assess “knowledge of science” and 35% should assess “knowledge about science”.

40. Enhanced Definition of Scientific Literacy Scientific literacy refers to an individual’s: • ·Scientific knowledge and use of that knowledge to identify questions, to acquire new knowledge, to explain scientific phenomena, and to draw evidence-based conclusions about science-related issues; • ·Understanding of the characteristic features of science as a form of human knowledge and enquiry; • ·Awareness of how science and technology shape our material, intellectual and cultural environments; and • ·Willingness to engage in science-related issues, and with the ideas of science, as a reflective citizen.

41. Examples of PISA assessment materials 1. Semmelweis’ diary 2. Ozone 3. Stop that germ.

42. How did students in Ireland perform in Pisa 2003?

43. 1. Overall Performance • In terms of overall performance on the science scale, Ireland achieved a mean score of 505.4 in science. • Ireland’s ranking in science is 16th of 40 countries. • Although just 5 points higher than the OECD average of 499.6, the difference is statistically significant.

45. 2. Performance of low and high achievers • The score for Ireland at the 10th percentile was 383.9, which is 22.3 points higher than the OECD average of 361.6. This score ranks Ireland 11th out of 40 countries at this marker. • The score for Ireland at the 90th percentile on the science scale is 624.5, lower than the OECD average of 636, giving a rank of 20th out of 40 countries. • This suggests that low achievers in Ireland are performing comparatively well in science, but high achievers are performing comparatively poorly in science. • There is an obvious need to investigate ways to extend the knowledge and skills of the highest performers in schools in Ireland.

46. 3. Students not studying science • As in PISA 2000, students in PISA 2003 who reported that they did not study science as a subject for the Junior Certificate Examination (5.2% of males, 14.6% of females, 9.9% of all students) achieved a mean score that was lower by 69.7 points than that of students taking science as a subject. • Clearly, the fact that almost 10% of students in the sample had not taken science as a subject for the Junior Certificate is cause for concern and contributed to the lowering of the average score for Ireland. • Students not taking science also performed less well in mathematics and reading than those who did take the subject.

47. 4. Comparison of the performance of students not taking science as a Junior Certificate subject and those taking Ordinary Level • In terms of performance, there is very little difference between students taking Ordinary Level Junior Certificate Science (443.3 points) and those not taking science for the Junior Certificate (451 points). • The difference of only 8.5 points in favour of those not taking science (around one tenth of a standard deviation) is not statistically significant (Table 4.24, Cosgrove et al. 2005 p. 118).

48. 5. Performance of students in disadvantaged schools. • Students in schools categorised as disadvantaged performed less well (478.6 points) than those in schools not categorised as disadvantaged (515.2 points), i.e. a difference of 36.6 points (Table 4.29, p. 122 Cosgrove et al. 2005). • Clearly, there are additional challenges for science teachers teaching in disadvantaged schools.

49. 6. Gender differences in science. • For science, there was no clear pattern of gender differences across countries, with an OECD average of just 5.8 points favouring female students. • (This contrasts with reading in which females outperformed males and in mathematics where males outperformed females). • In Ireland, the mean score for females was 504.4 points and that for males was 506.4 points (p. 98 Cosgrove et al., 2005). • This finding differs from the finding relating to science in the Junior Certificate Examination. Females taking the examination in 2003 scored about half a grade point higher than males. This difference is worthy of further investigation.

50. Looking to the Future – PISA 2006 • Firstly, it will offer a second opportunity to examine changes in achievements in mathematics, reading and science, allowing for more confident interpretations to be drawn, as methodologies become more refined. • Secondly, science will become the major focus of the assessment, allowing in-depth examination of students’ achievements across a wider range of content areas than has been possible in either 2000 or 2003. It will also provide an opportunity to describe student achievements along proficiency levels similar to those which already exist for mathematics and reading. It will also allow for comparisons of student performance along a number of distinct science themes.