Assessment in harmony with our understanding of learning

1. Assessment in harmony with our understanding of learning Mary James University of Cambridge, UK BERA President

2. Key questions Assessments need to be congruent with our views of learning if they are to be valid Questions for teachers: • To what extent do the assessments required of me, available to me, or that I create, reflect the kind of learning that I aim to promote in students? • If assessments lack congruence with learning, how can I try to bring them into closer harmony?

3. Three generations of assessment practice Three generations of ideas about links between assessment and learning and their implications. Headings adapted from Chris Watkins’ (2003) descriptions of different views of learning: • Learning is being taught • Learning is individual sense-making • Learning is building knowledge as part of doing things with others.

4. First generation: assessing learning of what is taught • individual • performance under test conditions. • ability to recall facts and demonstrate skills. • Tests or tasks are specially constructed - separate from learning. • Preparation involves practice • Time-limited • No access to materials/resources • Questions organised in a hierarchy of difficulty • Responses assessed as correct or incorrect • Comparison with other learners or against a standard • Areas for improvement inferred from incorrect responses

5. Underpinnings: Two different sets of assumptions underpin these assessment practices ‘Folk’ views of learning • The brain or mind is a kind of vessel into which information is poured. • Learning is assumed to have occurred when knowledge is ‘retained’. • Learner is passive • Learning depends largely on innate ability (IQ). Behaviourist views of learning • Learning as conditioned response to external stimuli • Rewards and punishments ‘condition’ desired responses. • Repetition of the stimulus makes the response habitual – value of repeated practice. • Eventually the response is so automatic that it can be produced at speed. • Complex skill broken down into components, taught separately, then re-assembled. • Economical to teach generic skills separately then applied in many situations. • Learn basic facts and skills first

6. Second generation: assessing learning as individual sense-making • Individual • Problem-solving and understanding. • Application of conceptual frameworks to find solutions to problems, and demonstrate cognitive skills. • Specially constructed • Tasks may be extended e.g. essays, open-ended assignments, projects, coursework. • Often time-limited - problem solving with ‘less search’. • Some access to materials - less a test of memory than understanding. • Assessed according to specified criteria - rarely one correct answer. • Normal trajectory of progress can be described. • Improvement comes from closing the gap between novice and expert. • Areas for improvement inferred from misunderstandings

7. Underpinnings: Cognitive constructivist views of learning • Learning is determined by what goes on in people’s heads • Building mental models of how the world works helps people interpret new information • Active process of meaning-making. • Emphasis on concepts • Higher (or deeper) level processes such as comprehension, application, analysis, synthesis and evaluation • Application of concepts valued for its capacity to reveal (mis)understanding • Novices need help to acquire the ways of thinking possessed by experts - the ways they organize knowledge and the way they control their mental processes through metacognition - thinking about thinking.

8. Third generation: assessing learning as building knowledge as part of doing things with others. Not much evidence that this exists in schools Underpinnings: The socio-cultural or ‘situated’ view of learning • Learning involves both thought and action in context • Thinking is conducted through actions that alter the situation and the situation changes the thinking - the two interact • Learning is a mediated activity - tools and artefacts have a crucial role e.g. books, equipment, language and sign systems • Learning is a social and collaborative activity - people develop their thinking together • Learning is distributed within the social group e.g. language skill is not solely an indication of individual intelligence but the intelligence of the community that developed it • The collective knowledge of the group is internalised by the individual. As an individual creates new knowledge, then he or she will externalise it in communicating it to others who will put it to use and then internalise it (an expansive learning cycle).

9. Implications for teaching • Teachers need to create environments in which people can be stimulated to think and act in authentic tasks • Access to tools is important • Activities that a learner can complete with assistance so that the ‘more expert other’ can ‘scaffold’ learning. • Tasks need to be collaborative • Students need to be involved in the generation of problems as well as solutions. • Teachers and students become a learning community and jointly solve problems • Most valued outcome is engaged participation in ways that others find beneficial • Problematic to judge an individual as having acquired knowledge abstracted from action settings

10. Implications for assessment Pointers for third generation assessment: • Assessment alongside learning, not as an ‘after learning’ event • Done by the community - role for self-assessment, peer- assessment and teacher assessment. • Assessment of group learning as well as the learning of the individual • ‘In vivo’ studies of complex, situated problem-solving i.e. participation in authentic (real-world) activities or projects • Focus on how well people exercise ‘agency’ in their use of the resources – proper justification for course-work assignments • Achievement captured and reported through narrative accounts and audio- and visual media • Portfolio has the potential to capture ‘assessment as inquiry’. • Holistic and qualitative judgement - not atomised and quantified as in measurement approaches.

11. Example 1: The ‘exhibition’ in US Coalition of Essential Schools (Ted Sizer, Grant Wiggins) • The exhibition brings together a number of dimensions of learning and meaningful assessment.   • It asks ‘students-as-workers’ to work across disciplines in a respectful way by creating ‘real’ learning activities. • Tasks are not necessarily devised by teachers; students can devise them for themselves, providing they understand the principles that underlie their construction. • It asks students to apply accumulated knowledge to new situations. • It insists on effective communication in a number of forms: oral, written and graphic. • It requires students to be reflective, persistent and well organised. • It creates a focus for learning by describing the destination for their journey, although precise learning objectives are not tightly pre-specified.

12. An example of a final performance across the disciplines “Discuss behavior patterns as reflected in the insect world, in animals, in human beings, and in literature. Be sure to include references to your course work over the term in Inquiry and Expression, Literature and the Arts, Social Studies, and Science. This may include Macbeth, the drug prevention and communication workshop, Stephen Crane's poetry, ‘A Modest Proposal’ and other essays you have studied, Mark Twain's fiction, and behaviors you have observed in our School-within-a-School. You may also add references to what you have read about in the news recently.” (Melinda Nickle, Arkansas)

13. Procedure • Day one of the exam: You will be given four periods in which to brainstorm, make an outline, write a rough draft, and write a final copy in standard composition form. You will be graded not only on how well you assimilate the material but also how well you reflect the ‘student as worker’ metaphor and how responsibly you act during the testing period. • Day two of the exam: You will assemble in villages of three, evaluate anonymous papers according to a set of criteria, and come to a consensus about a grade. Each paper will be evaluated by at least two groups and two instructors. Again, a part of your overall semester grade will have to do with how responsibly you act and how well you demonstrate the ‘student as worker’ metaphor.

14. Example 2: UK Yr2 project using Mantle of the Expert (Dorothy Heathcote) “Children and teachers work together to create an imaginary community within which they function as if they were experts e.g. mountain rescuers or archeologists. As the work progresses many possibilities begin to emerge which the learning community uses to define and deepen the imaginary world and explore the lives of the people that inhabit it. The community engages in a series of collaborative tasks, often motivated by a client’s demands, with teamwork, communication and problem solving central to the process. There is a group responsibility for the project as it progresses and the children act and make decisions with responsibility and authority, tackling authentic issues that seem purposeful and urgent to them.” (Jenny Lewis, Norfolk)

15. Range of assessment practices used • Ongoing dialogue • Blog diary (http://theseacompany.blogspot.com/ ) • Ongoing collective portfolio • Individual learning diaries • Daily sessions on meta-learning (Sniffles the Hopeless Hamster) • Self/peer assessment tools • Home/school contact books • Learning surgeries • Connnections sessions • Questionnaires (parents and children) • Evidence of children’s work • PSHE assessments e.g. Blob Tree; Feelings Wall • Mind Maps • Peer Teaching

16. Responses: Children • ‘All our little ideas have made one big, fantastic idea.’ • ‘It’s like real life – there’s big trouble! We can learn about solving problems.’ •  ‘It’s exciting because anything can happen.’ • ‘It’s fascinating. You get to learn about things much more. You’re learning things while you’re having big adventures.’ •  ‘You can pretend to do things you can’t do in real life.’ • ‘I think it’s brilliant because there are so many dilemmas and adventures that never end. I learnt about myself that I keep trying until I get it perfect.’ • ‘I like it because it seems so real. I’m learning about other people and how to work as a team.’

17. Responses: parents • ‘He will talk about what he has learned without prompting which isn’t normal.’ • ‘She’s definitely motivated by this type of learning – there’s obviously a great deal of enthusiasm for this project, not to mention the excitement factor. She is always pleased to tell us she will be doing company work today. It’s almost role reversal and she’s become the adult!’ • ‘There are lots of exciting things to do – we’ve heard all about who is doing what in the company, about diving – looking at the real gear - and about the history of the Titanic. We’ve taped TV documentaries and looked up things in books to feed the interest developed.’ • ‘It seems to give school work a purpose.’

18. Pros, cons and solutions • Pro: attractive to employers who claim they are interested in recruits who can demonstrate their capability to work in teams to find creative solutions to complex problems. • Con: raises questions about how to ensure the trustworthiness of such assessments when large numbers of students are involved and when those who are interested in the outcomes of such learning cannot participate in the activities that generate them. • Solution? The apprenticeship model with its concept of the guild as the guardian and arbiter of developing standards. • Another Con: within vocational education, systems of internal and external assessors and verifiers have attempted this although large-scale systems become bureaucratic, unwieldy and reductive. Clearly, more work needs to be done

19. Is synthesis possible? The big question is whether these theories of learning and their assessment counterparts are mutually exclusive, or whether, for practical purposes, they can be combined. The teacher in Example 2 thought some combination is possible and probably necessary.

20. How does this square with current developments at national level in England? Recommendations of the NCR Expert Panel, of which I was a member: a political compromise – or fudge? N.B. The EP’s remit was to advise on ATs and issues of progression but not on the whole of the assessment system.

21. The Form of Programmes of Study and Attainment Targets • We emphasised the importance of establishing a very clear relationship between ‘that which is to be learned’ and all assessment. • We did not believe that Attainment Targets in the present level descriptor form should be retained. Instead, we suggested that the Programme of Study is stated as a discursive statement of purposes, anticipated progression and interconnection within the knowledge to be acquired. Attainment Targets should then be statements of learning outcomes related to essential knowledge (broadly conceived as facts, concepts, principles and ‘fundamental operations’).

22. Assessment, Reporting and Pupil Progression 1 • We were concerned by the ways in which England’s current assessment system encourages a process of differentiating learners through the award of ‘levels’. • A distinctive feature of some of the high-performing systems is a focus on fewer things in greater depth in primary education, with particular attention on ensuring all pupils have an adequate understanding of key elements prior to moving to the next body of content.

23. Assessment, Reporting and Pupil Progression 2 • We recommended an approach to pupil progression that emphasises ‘high expectations for all’. • The focus should be on ensuring all pupils are ‘ready to progress’ at the end of each key stage, having mastered the knowledge identified in relevant schemes of work and/or Programmes of Study. • This approach conveys necessary teacher commitment to both aspiration and inclusion, and implies the specific set of fundamental achievements that all pupils should attain.

24. What have Ministers decided to date? • To abolish NC levels • To retain some form of KS2 testing in Ma and En (Bew Report) • To ‘reform’ 16+ qualification by: • Replacing GCSEs with EBacc Certificates (EBCs) in academic subjects • Getting rid of modules, coursework, controlled assessments, resits etc • Introducing 3 hour exams and making them harder, with no access to resource material • Replacing A*-G grades with Grades 1-6

25. Is all lost? • Not necessarily • It is reasonable to expect public demand for some kind of external summative assessment in basic skills at the end of KS2 and in the subjects of a broader curriculum at the end of KS4 • But, if Government is genuine in its encouragement of more teacher and school autonomy, there is space for curriculum and assessment innovation that might itself have a beneficial washback effect on achievement - and the nature of terminal assessments in the long term. • But this will depend on teachers having the confidence to seize the initiative. They are unlikely to be able to do this alone. They need partnerships. • Remember that the average term of an Education SoS is less than two years, and a Government less than 5 years.

26. Conclusion Overarching principles are ‘fitness for purpose’ and ‘validity’ - assess what you value rather than value what you assess. Assessment washback is powerful, especially in high stakes settings, so it is imperative to achieve a better alignment between assessment, teaching and learning. There may be possibilities for synthesis whereby a more complete theory can emerge. The possibility for a more complete and inclusive theory of learning to guide practice of teaching and assessment seems to be a goal worth pursuing.

27. References • James, M. (2006) Assessment, teaching and theories of learning. In J. Gardner, Assessment and Learning, London, Sage. • James, M. (2008) Assessment and learning. In S. Swaffield, Unlocking Assessment, London, David Fulton. • James. M. with J. Lewis (2012) Assessment in Harmony with our understanding of learning: problems and possibilities. In J. Gardner, Assessment and Learning, Second Edition. London, Sage.