IF PISA IS THE ANSWER, WHAT IS THE QUESTION?

1. IF PISA IS THE ANSWER,WHAT IS THE QUESTION? Elizabeth Oldham Trinity College, Dublin Third National PISA Symposium 18 April 2008

2. Questions • What cross-national study of mathematics education (“CSME”) are we addressing to-day? • Describe the Irish performance 2000-2006 • Which CSME, despite not containing a curriculum analysis, has provoked most debate about mathematics curricular issues in Ireland? • Discuss the role of curriculum analyses in providing a context for considering results • Which CSME is grounded in “Realistic Mathematics Education”? • Explain the main features of RME

3. From which CMSE can we learn most in order to address problems in Irish mathematics education? • With relation to “Project Maths”, discuss how problems in Irish mathematics education might be addressed “Problems in mathematics education” can be read in two ways….

4. What cross-national study of mathematics education are we addressing to-day? PISA

5. Description of the Irish performance 2000-2006 • The PISA framework for “mathematical literacy” • Four broad content areas (“overarching ideas”) • Space and shape • Change and relationships • Quantity • Uncertainty …. among other features

6. Overview • 2000 • Minor assessment domain • The first two areas tested • 2003 • Major assessment domain • All four areas tested • Content-area subscales and “proficiency levels” developed • 2006 • Minor assessment domain • Subset of the 2003 items used

7. Results • Descriptive data on achievement • Ranking of countries by mean score • Country means and standard deviations • Scores at key percentiles • Percentage of students at “proficiency levels” (2003 and 2006) • Subscale scores for content areas (2003) • Further analyses • Multilevel models • Achievement differences between schools • Associations with background variables … etc., etc., etc.

8. Summary of results 2000-2006 • (a) Ranks, means and SDs for Ireland • (b) Further details on distribution • (a) Ranks, means and SDs (see on) • To be viewed with caution! • Not on the same tests… • … and despite the wonders of IRT scaling • Trends technically available from 2003 to 2006 • … even then tentative because testing context had changed (items differently distributed)

9. Ranks, means and SDs for Ireland

10. (b) Further details on the distribution • Scores at key percentiles (see handout) • Not a great “tail” (i.e. comparatively few low scorers)… • … but not a great “head” either • “Proficiency levels” (see handout) • Empirically determined, but can be associated with skills • Again, Irish scores “bunch” • Summary • “Could do better” performance by students • … at least the average and the higher achievers • Homogeneous schools

11. [Even] with caution… … remarkably similar story! … this time lacking detail of subscales

12. Country comparisons with Ireland in 2006 • Countries scoring significantly higher include • Finland • Netherlands • New Zealand • Countries scoring at about the same level include • The UK (… and note Northern Ireland: a much larger SD, reflecting a selective system)! • France • Germany • Poland • Countries scoring significantly lower include • USA

13. Which CSME, despite not containing a curriculum analysis, has provoked most debate about mathematics curricular issues in Ireland? PISA!

14. The role of curriculum analyses in providing context • Threefold curriculum • Intended • Specified by state or other body • Implemented • Taught by teachers in classrooms • Attained • Learnt by students They differ!

15. Typical approach in cross-national studies • Work as follows • Design comprehensive list or grid • Collect data on intended curricula from many countries • For reasonably common areas, collect data on implemented (teacher questionnaires) and attained (student tests) curricula • Interpret attained results in the context of findings • “Curriculum as a variable” • Note • “League table” presentations of results that ignore this are perhaps at best problematic….

16. PISA • Study of “mathematical literacy” for life and work etc. • Therefore, claims to be independent of curriculum • … but who says what is relevant for life and work? • …… not philosophically neutral! • Bridging the gap • Countries – notably Ireland! – have found the need to put in the missing variable by relating the tests to their curricula • “Test-curriculum rating project” • … and for us, there are marked discrepancies

17. Which CSME is grounded in “Realistic Mathematics Education”? PISA!

18. Principles / tenets of RME • According to Cobb (see de Lange, 1996) • Starting points of instructional sequences (ISs) should be experientially real to the students … • … and should be justifiable in terms of the potential end points of the learning sequence • ISs should involve activities in which students create and elaborate symbolic models of their informal mathematical activity • … and also (de Lange, 1996) • Instruction should be interactive • The real phenomena in which mathematical concepts and structures manifest themselves should lead to intertwining of learning strands

19. Relevant concepts • Cognitive fidelity • “Does it make sense?” • Mathematical fidelity • “Is the mathematical representation correct?” • Transition from informal via preformal to formal • See Dekker (2007) – Proceedings of MEI2 • Hence • Use of engaging contexts… • … which lead to appropriate mathematics • … and maybe are found empirically (some such contexts work)

20. Examples • Video material • Note in particular Math in Context with an explicit RME philosophy (see for example www.showmecenter.missouri.edu/showme/mic.shtml) • Video clips from MiC and similar projects’ lessons are available at www.mmmproject.org • “National scripts” for lessons: if such exist (and they are controversial) … these have not been ours! • PISA questions • See handout

21. Mathematisation • The process • Starting with a problem situated in reality • Organising it according to mathematical concepts • Gradually trimming away the reality through processes which promote the mathematical features of the situation and transform the real problem into a mathematical problem that faithfully represents the situation • Solving the mathematical problem • Making sense of the mathematical solution in terms of the real situation Thanks to S. Close

22. Components of mathematisation • Horizontal mathematisation • Real-world problems mapped into world of mathematics • Vertical mathematisation • Development within the world of mathematics • Freudenthal’s comment • “How often haven’t I been disappointed by mathematicians interested in education who narrowed mathematising to its vertical component, as well as by educationalists turning to mathematics instruction who restricted it to the horizontal one…” (Freudenthal, 1991, p. 41)

23. Possible curricular philosophies RME: horizontal and vertical components “Modern mathematics” etc. (as intro-duced in Ireland): (over?-)emphasis on vertical component

24. Potential problems for RME • (E.g.) If the horizontal component is too large compared to the vertical… • Some evidence of difficulty, e.g. in the Netherlands • Pelgrum, personal communication, April 2006

25. From which study can we learnmost in order to address problemsin mathematics education? PISA?

26. Project Maths • Background • From around 2000, growing worries about performance in (senior) school mathematics • Performance in Leaving Certificate Ordinary level, especially with regard to non-routine work • Third level students’ understanding of and ability to apply school mathematics • Moderate PISA results (for age 15+) • NCCA Discussion Paper, Autumn 2005 • National consultation, 2005-2006 • Summary of feedback, summer 2006 • Mandate for (e.g.) some change towards greater emphasis on applicability Also Conway & Sloane

27. Development • Broad outlines established by NCCA • Usual representative structures and processes • Work by Course Committees (for both Junior and Leaving Certificate) and – as a newer feature – the relevant Board of Studies [hence, partners in education]… • … engaging with issues as raised above, but notvia wholesale acceptance of RME or any other philosophy • Proposals adopted by NCCA Council and then agreed by the Minister • New implementation model (see on) • (Therefore?) perhaps rather less engagement to date with constituencies (at least IMTA) than before… • … but greater future role anticipated

28. Emphases (addressing the problems) • See flier, gone to schools this week! • … “greater coherence and progression”, “more relevant”, “greater understanding and application”, “logical reasoning and problem solving”, “appreciation of the value…”, “more … study higher level mathematics” , “alignment [of learning and assessment]” • Bill Lynch, quoted in Sunday Times (30/3/08) • “… change the emphasis in learning to allow students to put maths in context and develop problem-solving skills, rather than learning abstract formulas by rote” • Hence • Higher-order skills rather than rote learning… not“dumbing down”!

29. Syllabus / curriculum / course… • Structure: five “strands”, building from the strands in the Primary Curriculum • Statistics and probability • Geometry and trigonometry • Number • Algebra • Functions • Content: some changes (naturally!) • Teaching / learning: crucial changes (for many…) • Assessment: ongoing change and eventual overall review (also crucially, to obviate inappropriate learning…!!)

30. Implementation • Start in 2008 • 24 pilot schools • Undertake two strands (1 and 2) at JC and LC level • Resources available and teachers supported • Alternative state examination questions • Go nationwide after 2 years (with adjustments reflecting experience) • Repeat as required for other strands • Fully implemented over a period of years • Assessment then reviewed, when review can be more fundamental

31. Historical note • All recent (intended) curricular revisions have shared aims with regard to persistent problems • Emphasising understanding and/or application… • … though context emphasis is perhaps new • Increasing uptake of Higher courses… • … recall there was a lower Intermediate Certificate course with very substantial uptake – whereas other subjects had “Junior Higher” as the norm • Differences this time • Aim to implement / attain the intended curriculum by • Supporting teachers (to a greater extent than heretofore) • More appropriate assessment (to be developed!) • Small-scale start with use of feedback [etc.]

32. Best of luck to all concerned!