A conversation with Ruth Sutton Feb. 17 th 2010

1. Curriculum for Excellence and BTC 5 : possible implications for the Scottish Qualifications Authority A conversation with Ruth Sutton Feb. 17th 2010

2. CfE, BTC 5, teachers and schools • National Assessment arrangements have now begun to roll out, with very rapid change expected over the next few months • For some ‘CfE deniers’ this is coming as a shock • Record-keeping and reporting are still unknown quantities: BTC 5 refers constantly to ‘regular’ reporting to parents, but provides no indication of frequency, without which the implications are impossible to determine • Secondary teachers seem more anxious and confused than their primary counterparts. Is this the case, and if so, why?

3. Some questions I’ve raised recently with Scottish teachers about assessment • How can we achieve an acceptable balance of validity (accuracy) and reliability (consistency and fairness) in national assessment and avoid the ‘collateral damage’ to teaching and learning? • How can we substitute ‘shared professional judgement’ for the external test ?

4. The assessment balancing act: do the elements pull in different directions? Validity Reliability Best fit Manageability Cost and ‘credibility’

5. Assessment: 2 Key Purposes Assessment of learning Checks learning to date Audience beyond the classroom Periodic Uses numbers, scores and grades Criterion/standards referenced No need to involve the learner Assessment for learning Suggests next learning Audience is teachers and learners Continual – conversation and marking Specific feedback, using words Self-referenced, ‘ipsative’ Must involve the learner – the person most able to improve learning

6. ‘Shared professional judgment’ What is happening with Curriculum for Excellence? • Standards remain relatively ‘loose’ • Those involved in making judgments need to agree on the precise meaning and implications of the ‘standards and expectations’ • Judgments will be shared and cross-checked, to reduce ‘rater variables’ (ie. the individual teacher’s judgment) to an acceptable level, and improve reliability, given the inevitable margin or error

7. The benefits of ‘moderation’ in the current Scottish circumstances • CfEemphasises skills rather than content • The evidence of skills is necessarily contextual, and more easily found across a range of classroom activities than in separate assessment ‘events’ • Teachers will share their understanding of the CfE framework in order to design their teaching activities • Sharing their understandings and planning is a more efficient way to work than each teacher or subject team working alone • Teachers will be more confident and clear, gathering only as little evidence as they need, not as much as they can manage • Parents will receive more accurate reports about their children’s progress, based on shared considered evidence and judgment

8. Costs and benefits of moderation,for teachers? Costs: • Time to discuss expectations • Possible change in planning habits • Initial confusion resulting from a poor understanding of the purposes and principles of assessment Benefits: • Greater clarity about CfE ‘experiences and outcomes’ • Getting teaching and assessment ideas from other teachers • Improved manageability through gathering less evidence • Greater confidence when reporting to parents • Better understanding of the purposes and principles of assessment

9. Professional sharing of ‘standards’ The teachers’ discussion will probably have four stages: • What do these ‘standards’/ ‘experiences and outcomes’ really mean, and what do they look like in practice? • What evidence would properly support a teacher’s judgment of a student’s achievement of the ‘standard’? • What ‘success criteria’ will we need to identify progress within the level? • What teaching activities will be necessary to enable the student to learn, practice and demonstrate achievement of the ‘standards’?

10. Backwards Design • Traditionally teachers have planned ‘forwards’ from a definition of learning objectives, into teaching activities and finally into assessment activities and evidence of performance • Backwards Design – the process • ask big questions about what the pupils will learn, and prioritising the most important elements • Consider what evidence would/will there be that this desired learning has happened? • ensure that this evidence will be a valid reflection of the expected learning, and sufficient (‘proportionate’) for reporting purposes • design teaching and learning activities which will generate this evidence of learning, starting with the necessary check for prior learning and misconceptions • The benefit of Backwards Design is that it enables both teachers and pupils to identify the learning goals and focus on them in both teaching and assessment.

11. Experience of ‘moderation’ This is a personal view, based on experience of helping teachers implement standards-based curriculum and assessment over the past twenty years or so Teachers find ‘moderation’ - • Challenging, as it exposes their professional learning and judgment to the scrutiny of their peers • Very helpful, as it involves detailed practical discussion of learning expectations and the teaching tasks necessary to encourage learning and achievement • Necessary, to avoid individual teachers gathering too much, poor quality evidence to compensate for their lack of confidence

12. Some suggestions.. • Don’t spend too long initially on semantic deconstruction of the words in the ‘standard: when someone opens the Roget’s Thesaurus it’s probably time to move on • To help lower the temperature, use neutral exemplars to clarify and illuminate the discussion, before reaching for work done by your own students • Encourage teachers to accept that professional differences of view are probable and acceptable. They are also opportunities for professional learning if we keep an open mind.

13. Learning from others • Wales has begun a process of structured moderation at KS2 and KS3, to improve the quality of teacher judgment and to encourage sharing of best teaching practices. • Queensland, Australia has a long history of assessment using teacher moderation. Check out www.education.qld.gov.au/qcar/social-mod.html • New Zealand’s new curriculum will also be supported by a moderation process, leading to ‘Overall Teacher Judgment’.

14. Queensland’s Conference Model of Moderation • “Using the conference model for moderation, teachers discuss and deliberate in making their judgments about the quality of all of the evidence presented as student work. • Teachers make judgments on several criteria to reach an 'on-balance' holistic judgment. This is not a procedural approach but one that is based on the teachers' professional knowledge in shared and collaborative decision making. • Teachers mark (some or all) student responses individually, and then select assessment samples representative of their application for A to E standards. They meet with other teachers to discuss their judgments by sharing their samples. Teachers reach a consensus on the interpretation and application of the standards.”

15. The Role of the Facilitator “In the Conference Model of social moderation the role of the facilitator may include: • Establishing the moderation environment • Identifying the curriculum intent • Leading professional dialogue • Facilitating conversations that support evidence-based teacher judgment • Clarifying moderation protocols It is not expected that the facilitator act as an expert, but rather assist teachers reach consensus through a shared understanding of the curriculum intent and the grade awarded”

16. Queensland Social Moderation Protocols • “Commit to the purpose of the moderation process • Adopt a sense of responsibility in and for the group • Respect and listen to others openly • Accept where others are at • Cooperate in good faith • Aim for consensus in decision making • Address problems respectfully by seeking clarification and understanding, focusing on the student work and not the teacher who presents it • Treat others as you would like to be treated • Critique not criticise”

17. Finding the assessment balance: the ‘Muddle in the Middle’ In the primary years, • the norms of assessment remain largely ‘formative’ • AiFL has been accepted and adopted, on the whole • The structures and cultures of most primary schools encourage collaboration In the secondary years, • the dominant assessment culture has been external and summative • AiFL is ‘patchy’, at best: ‘tools’ are used without necessary understanding of ‘principles’ • Structures are fragmented vertically, discouraging collaboration in skills assessment (Num., Lit., and HWB) across ‘subjects’ S1 (and possibly P7) to S3 sit uncomfortably between these two

18. SQA’s position • My impression is that SQA, like any other qualifications authority, has been over many years focused on ‘subjects’, reliability and a particular definition of ‘fairness’ • CfE and BTC 5, and the principles of AiFL that ostensibly underpin both, challenge many of the ‘old’ paradigms assumptions. • Secondary schools and teachers are looking for clarity and leadership: without this some more basic instincts and attitudes may prevail

19. Some basic instincts and attitudes • Assessment is primarily about measurement • ‘Grading and sorting’ is the first purpose of secondary assessment • Involving the students (a la AiFL) doesn’t make sense in these circumstances • External measures are required to prevent teachers and students ‘cheating the system’ • In a content-led curriculum, planning and teaching is largely about ‘coverage’: how then to plan and deliver a skills-based curriculum in which the context is determined by the teacher? • A ‘level’ is a line, not a large space within which students might learn non-sequentially

20. Questions • What are the issues and concerns facing SQA at this stage in the roll-out of CfE and BTC5? • What is SQA’s role as an essential provider of teachers’ professional learning over the next months/years? • What are the key messages around assessment that SQA can help to disseminate, and how is this best achieved?

21. Thanks for the invitation to talk with you • All my views are personal and can therefore be completely disregarded • Additional ‘outsider’ views might provide necessary ‘triangulation’ • Sutton.ruth@gmail.com