Caroline Long, CEA, University of Pretoria, Caroline.long@up.ac.za

1. Assessment in mathematics education: The interface between the multiplicative conceptual field and the Rasch measurement model Caroline Long,CEA, University of Pretoria, Caroline.long@up.ac.za

2. Presentation outline • Challenges for mathematics education • Theoretical tools • Theory of conceptual fields (Vergnaud, 1983) • Threshold concepts (Meyer & Land, 2005) • Zone of proximal development (Vygotsky, 1962, 1978) • Research study • Rasch measurement theory • Reflections on the outcomes • Educational implications • Application of the Rasch model in mathematics education settings • Further developments

3. Challenges for mathematics teaching • The transition from working with whole number to rational numbers (and real numbers), is noted as a critical mathematical development in the transition years, Grades 6 to 10 (Usiskin, 2005), which in large part determines the mathematical future for learners. • Fractions, ratio and related topics like percent, are taught procedurally without regard for their complex interconnections and their applications in everyday situations (Kieren, 1976; Parker & Leinhardt, 1995; Lamon, 2007). • It is the assimilation and accommodation of the component subconstructs comprising rational number that are required for synthesis in the predicative form (Kieren, 1976; Vergnaud, 1983; Lamon, 2007). • The danger is that “tricks” are introduced to avoid real engagement with the concept (Davis, 2010).

4. Theory of conceptual fields (Vergnaud, 1983 - ) • Mathematics is developed through encountering concepts embedded in problem contexts • No one-to-one mapping from mathematical concept to problem situation. In fact all such mappings are many-to-many • Concepts are never learned singly but in relation to other concepts • The multiplicative conceptual field include • at base multiplication and division, but also • the problem situations which are modelled by multiplication and division and • the various representations that may be required. • The purpose of mathematics education is to transform extant implicit and local conceptions of individual learners encountered in single problem situations into generalised mathematics concepts and theorems that may be applied to a class of related problems – from operational to predicative form

5. Threshold concepts (Meyer & Land, 2005) • An historical perspective • increasingly abstract systems, radical conceptual shifts (Dantzig, 2007) • parallel conceptual reorganisation in the classroom (Sfard, 1991) • resonates with the notion of “epistemological obstacles” (Brousseau, 1983) • A threshold concept is a critical concept which provides the gateway (to higher mathematics), and which by corollary inhibits mathematical progress where learners have not gained mastery. • These conceptual gateways may be • transformative – “occasioning a significant shift in perception of a subject” • “troublesome” knowledge - defines “critical moments of irreversible conceptual transformation” • irreversible – “unlikely to be forgotten”, “unlearned only through considerable effort”, and • integrative – “exposing the previously hidden interrelatedness of concepts” • The danger of substitution by naïve concepts which confine progress

6. Zone of proximal development (Vygotsky,1962) • Theory of conceptual fields builds on the work of Piaget and Vygotsky • “(t)he discrepancy between a child’s actual mental age and the level he reaches in solving problems with assistance indicates the zone of his proximal development” (Vygotsky,1962, p. 102). • Current interpretation - a conceptual and cognitive space comprising the distance or extent from current proficiency to a higher level, over which learning with assistance from teachers or peers may occur . • There may be hierarchies of concepts that build on one another and that some concepts may be within the learner’s current “zone of proximal development”. More abstract concepts may require prior mastery or at least partial mastery. For example an understanding of ratio may precede an understanding of time distance and speed.

7. Assessment instruments • The contextual features discussed lead us to assert that educational experiences in mathematics are ordered somewhat hierarchically and aligned with learners current proficiency levels. • How do we identify for individual learners, or groups of learners a suitable current conceptual space, an educational locale (or zone of proximal development) which is neither too tedious because the associated problem solutions are obvious, nor too difficult for the learner and hence frustrating or demotivating? • Can assessment instruments be designed that make explicit a conceptual and cognitive pathway that provides insights for the teacher, in the preparation of learning sequences and subsequent engagement with individual learners? And in this process, can instruments help identify threshold concepts?

8. Research design • Instrument construction, 36 items (18 MC and 12 polytomous items) • TIMSS 2003 released items, adapted slightly in some cases • Matrix design (12 common items and a further 24 distributed across 4 groups) • Learners at two schools, across three grades,7, 8 and 9, 16 classes, and 330 learners • Reasonably well-functioning schools, each covering a range of South African demographics • Follow-up interviews on 4items of graded difficulty with 6 learner groups (21 learners in all, located at specified proficiency levels as determined from a Rasch analysis).

9. Person-Item Map Persons and items on the same scale, according to probabilistic estimates.Selected ratio, rate and proportion items are described on the right. Levels* aligned with logits are demarcated by bands from (- ,-2) [-2, -1), [-1,0), [0, 1), [1,2), [3, + )

10. A note on terminology • “Levels” • overused, a place holder, • familiar (Griffin, 2006; Pisa 2009; TIMSS) • denotes a defined position on a unidimensional line • “Locale” • an area, a domain, or range on the 1D line • a difficulty locale (for a subset of problem situations) • a proficiency locale (for a subset of learners currently exhibiting a similar proficiency) • Criterion zone (Wilson, 2005)

11. Item analysis by level and analytic category

12. Distractor analysis

13. Item analysisSelected items (Item 1, Item 20, Item 5, Item 10, Item 30) at graded levels of difficulty, levels of proficiency: means of 4 quartile groups (LQ, MLQ, MHQ, HQ).The key mathematical concepts for success on the item noted below.Multiple choice distractor plots: percentage of quartile group selecting each false choice

14. Summary information MCF concepts embedded in the problem situations at each level.Associated errors by levelMean percentages correct for items by level and by quartile

15. Interviews: Item level by learner proficiency

16. Interview dataInterviews with the middle high proficiency group, on Item 5.Written responses on the left, explanations of thinking about the problem on the right.

17. InterviewtranscriptItem 5, middle high quartile group transcriptDiscussion occurred after individual explanations of reasoning about the problem.

18. Composite summaryComposite information from summaries of item analyses at levels of difficulty and summaries of interview data at levels of proficiency.On the horizontal axis the proficiency levels are arranged from high to low.On the vertical axis, the key concepts identified are listed from lesser difficulty to greater difficulty.Common errors are noted in brackets,

19. Reflections: theory of conceptual fields and Rasch measurement • The insights gleaned from such a study have application in their local context, for example for the population targeted in the study, Grades 7, 8 and 9 in South African schools. • While some learners have yet to achieve fluency with multiplication and division, others need extension at the higher end of the spectrum of multiplicative structures, for example, rate and double proportion problems • The selection of proximate items not yet correctly mastered by learners is easily executed given the outcomes of the Rasch analysis, in conjunction with the theoretical analysis of the multiplicative conceptual field. • The method advanced in this study follows that advocated Wilson (2005) for the most part, though further cycles of refinement may be implemented for any context.

20. The interactive relationship of theory and measurement • This development and validation may elicit joint orderings of sets of items within a conceptual field, and respondents, the distances between which have a meaningful interpretation and permit the specification of a ZPD. • We may infer from these relationships, positing educationally pertinent interventions. • We claim that, given theoretical analysis, we may infer a degree of hierarchical ordering that, with further cycles replicated in these grades, may prove to be a helpful instrument in the planning and support of teaching. • Provided that the instrument has been carefully constructed to include potential threshold concepts, these thresholds may be hypothesised and perhaps identified.

21. Concluding remarks • The theory of conceptual fields raises some questions about current articulations of the intended curriculum • From relationships of item difficulty and learner ability, educationally pertinent interventions can be targeted at groups of similar learners, as a means of reaching all learners efficiently – notion of ZPD • The identification of threshold concepts - somewhat more complicated • From an historical perspective, the topics which caused particular consternation, for example the critical concepts that led to the construction of new number systems, the zero, the integer, the rational number etc., may be the starting point. • The theoretical insights (see for example, Dantzig, 2007) into the topics which appear particularly troublesome and which require attention • Critical for threshold concepts to be hypothesised and then included in assessment research

22. Further developments: A model of assessment (Bennett & Gitomer, 2009)

23. Accessible and powerful assessment • Accountability • A process of teacher involvement at the various phases of the item construction • Accountability becomes intrinsic to the teacher • Professional development component • Deep domain specific knowledge • Expert input on identified problem areas • Formative assessment • A classroom component where teachers develop, experiment and reflect on the suitability of current assessment in relation to the knowledge domain • Professional learning communities (PLCs)

24. Critical questions for future research • How may the theoretical notions of conceptual fields (threshold concepts and zones of proximal development) inform educational assessment and measurement? • And reciprocally, how may measurement theory highlight and extend insights into conceptual fields, threshold concepts and zones of proximal development as they manifest in the progressivedevelopment of learning?

25. Thank you for your attention Caroline.long@up.ac.za