Results to math teachers. Why? Which results? How to provide the results?

1. Results to math teachers.Why? Which results? How to provide the results? Lena Lindenskov, In collaboration with Peter Weng and Morten Misfeldt Department of Education Aarhus University (DPU, Campus in Copenhagen) PISA 2012 NPM Meeting, Singapore1

2. Context (1/4) • DK population: 5.5 million • Comprehensive school system • No level streaming • Tailored teaching according to the student’s different proficiency levels

3. The national objectives for mathematics In Danish: FællesMål 2009 Aim is to develop mathematical competencies and acquire knowledge and skills to act appropriately in mathematics related situations, concerning everyday life, societal life and natural conditions. Mathematics classes are organized in order to let the pupils - independently and through dialogue and collaboration with others - experience that working with mathematics demands and supports creative practice, and that mathematics contains tools for problem solving, argumentation and communication.

4. Continued Mathematics classes shall help the pupils to experience and recognize the role of mathematics in cultural and societal contexts, and shall help the students to evaluate the use of mathematics in order to take responsibility and to act influentially in democratic communities.

5. Central knowledge and skills areas are organised as: • Mathematical competences • Mathematical topics • Mathematics in use (see next slide) • Mathematical working methods

6. Goals for ‘Mathematics in use’ • that students acquire knowledge and skills in order to be able to • mathematize problems in daily life, society, and nature, and interpret mathematical models’ descriptions of reality • use mathematical tools, concepts and competences to solve mathematical problems in relation to daily life, society, and nature • use mathematics as a tool to describe or predict a development or event • acknowledge possibilities and limits of mathematics when describing reality (FællesMål, 2009, p. 10)

7. The Danish PISA-team wish to inform mathematics teachers on PISA results • Why? • Which results? • How to provide the results?

8. Whyinformteachers? • Mathematics teachers (in DK) are mostly sceptical, saying: • “PISA does not supports classroom practice” • “Although some correlations within-country and between-countries seem relevant, then what should I do?” • Very much double-code data is provided, but not used • So: Why not use PISA-results for formative assessment!

9. Which results? Twokinds of results: • 1. Troublesome, interestingcorrelationswith-in country and betweencountries • 2. In-depthresults on students’ performance on single items and units • Richdescriptionswhichteachercanrelate to ownpractice

10. 1. Troublesome, interestingcorrelationresults • Self-relatedconstructs: • Extremelyhighinterest and enjoyment in Mathematics, internationallycompared • Extremelylowmathematicsanxiety, internationallycompared • Performance: means of 514, 514, 513, 503 and belowaverage differences between25th and 75th percentiles. • Immigrant students perform relativelyweak in mathematicsliteracy • Girls perform relativelyweak in mathematicsliteracy

11. 1.Media dimension in testing (PISA ERA) • From ERA we see that DK girls performs very low in the electronic test. • Despite that Danishyouth are among the most equipped with digital tools and, with the highest frequency of computer use in schools • Question: Can we isolate the media dimension in testing mathematics literacy in the PISA 2012? • Initial hypothesis is that Danish youth (girls) are not used to perform in a computer environment • Knowledge could have ict-didactical implications

12. 1.ERA Computer use in primarylanguageeducation

13. 1.PISA scores ERA

14. 1.How to provide the troublesomecorrelationresults? • We suggest that PISA internationallykeepreporting on • The factors ussuallyreported • Paper-basedand computer-basedmathematicsliteracy performances separately, so thattheycanbecompared for each country

15. 2.Which results – part two • In-depthresults on students’ performance on single items and units • Becauseteacherscanrelatesuchrichresults to ownpractice

16. In depth-analyses of students’ papers from PISA 2013 on fifteen released test units 100 – 400 student papers on • 4 units in Space and Shape, S & S • 2 units in Change and Relationship, C& R • 5 units in Uncertainty, U • 4 units in Quantity, Q

17. The fifteen test units S & S • 143 Cubes, • 555 NumberCubes, • 547 Staircase, • 266 Carpenter. U • 079 Robberies, • 467 ColouredCandies, • 468 Science test, • 505 Litter, • 702 Support for the president. C & R • 150 Growing Up, • 704 Best Car. Q • 513 Test Scores, • 510 Choices, • 520 Skateboard, • 806 Step Pattern.

18. Report (74 pages): • Information: general, unit and item • Item results from Denmark and someothercountries: • right, partly right, wrong, • seconddigit, • average, genderaverage • Item code information • From eachcode: • Student answers from coding guide • Danish authentic 2003 answers • Suggestions for formal assessment and teaching

19. Simple-coded items • For items coded as right or not-right, we found interesting information for teachers in categorizing the wrong answers. For instance, we find three kinds of wrong answers to Number cubes. One kind is repeating, another is mirroring, and the third one is calculation errors. • In aVygotskyan approach,students who give one kind of wrong answers need a different type of teacher help than students who give another kind of wrong answers.

20. Complexcoded items • In the case of Robberies some students involve everyday knowledge in right, partly right and wrong answers • More everyday knowledge and less mathematical knowledge is used in the not-right answers • The right and partly right answers are longer than the not-right answers • All nine second digit codesare represented in the Danish student answers

21. The Robberies item motivates students • The diversity of the answers – being correct, partly correct or non-correct - shows the complexity of the item, and it seems that Robberies motivates students to engage in interpreting a diagram and in reasoning. Here are – translated by us from Danish into English – some examples:

22. Examples of Robberies student texts • Some development has taken place. We see more robberies, but not in any strong sense. It has grown with approx. 8 robberies (found at the graph), and that is not very much. The journalist has exaggerated, but when you look at the graph it looks bad, but the ‘titles’ [Danish: benævnelser] are close to each other, that is why a growth of eight robberies looks very big.

23. Examples of Robberies student texts • Such a small growth may be random, and next year you may have a markedly decline in robberies. So I think the interpretation is unreasonable. • I don’t think 9 robberies is a very big growth. • What do you mean? It is reasonable, but how can I show it? • Reasonable. I suppose so, but you cannot precisely see how many burglaries were in 1998. It would have been better with a line diagram. • It would have been easier, if you had shown it on a circle diagram instead. (our translation)

24. Small scalestudy with 21 students in a grade 9 class, 2011

25. The 21 students showedlowanxiety and highself-concept • I oftenworrythat it willbedifficult for me in mathematicsclasses • PISA 2003 p. 139: 34% • The grade 9 class, 2011: 19 % • I learnmathematicsmathematicsquickly. • PISA 2003 p. 134: 70% • The grade 9 class, 2011: 81%

26. Examplesof the 21 students’ Robberiestexts • Yes, there is an increase, so it is a fine interpretation, but it makes her unreasonable to say it is a huge increase • No, because it is not a huge increase, but you know journalists can say anything. • No, it looks huge at the illustration, you see the relative heigt of the two coloms, but looking at the numbers only an increase of about 9.

27. The 21 students’ reflections on Robberies as item • Somesaidit is not mathematics as • It lacksnumbers or othermathematical elements • It lacksthat I or the journalist activelyinvolve in calculations or othermathematicalactivities • Somesaid it is a veryinterestingmathematicstask as • You have to think • Thereare more thanone solution • Youcandiscuss real worldmathematics

28. The 21 students’ reaction to the use of autenticanswers in the clssroom • Good idea • Inspiring – wesee new perspectives • Wewish more activities, lessuse of books, lesslooking at the teacher’swritingthanwe do now in ur mathematicsclassroom • I’manxiousthatseeingwronganswers I maycopythem • Welikehowourtacherteachesusnow, and hedoes not useauthenticanswers from Denmark or abroad

29. Reflectionsof 24 teachers at grade 10 – 12, 2011 to the use of authenticanswers • The teacherssawauthenticanswers as potential learningmaterials: • As starting points for students which I as teacher find difficult to coach • As a tool for raising performance • As a stimulation for discussionsamong students

30. Conclusion • It seems that authentic student answers can be effective and motivating resources for formal assessment and for learning • It seems that it will be motivating for Danish students and teachers to get access to (studies of) authentic student answers from a range of countries.

31. How to provide the results? • So, we suggest collaborative studies on this kind of study to be done in other countries as well in PISA 2012. We suggest a project with a common design for documentation and analyses • Using double digit coding • Looking for student strategies in all items formats

32. References • Lindenskov, Weng (2010). 15 matematikopgaver i PISA. www.au.dk

33. Thankyou! Questions! Suggestions!