…..Finding a middle way

1. …..Finding a middle way Assessment Quality Assuring Assessment : A Local Authority view

2. Fife Context • Second/third biggest LA (55,000 pupils) • Lean Management • Largest schools in Scotland to the smallest • Focus on Attainment and Quality Improvement • Increase in 5-14 for the last 7 years • Increase in SQA in the last 2 years • Personalised e-assessment (4 years); PIPS (10 years) • A focus on Tracking and Monitoring (E1)

3. National Context – what is in place • National Debate, AIFL, 4 Capacities • National Collection of 5-14 results discontinued • Curriculum Framework • Skills for learning, life and work • Assessment Framework • Outcomes and Experiences • Commencement of NAR within GLOW • BTC 5 and BTC 3 condensed

4. National Context – what is not yet in place • Success Criteria/standards/milestones of progress • Clarity on progression pathways • National approach to moderation that will lead to consistent, valid and reliable judgements • Clear Reporting Processes • Confidence that our OECD results will improve • Clarity on the qualifications performance criteria (Senior Phase) • Clarity on the way success will be measured at a pupil/school/LA level particularly in the senior phase

5. Strategy Energy, Enthusiasm, Enablement, Enrichment • Purpose - agreeing the philosophy • Principles - agreeing the outcomes • Pragmatism - agreeing the actions/timescales

6. Enablement • Head Teachers leading strategic groups at all levels • Power Point presentations with notes • Reflective questions • An assessment plan outlining actions and timescales and outcomes • Suggested activities with resources for use with school staff and across the cluster

7. Purpose • Accelerating the learning of individual pupils • Assuring learners and Accountability to interested third parties • Access to further learning opportunities. • Accrediting and Awarding learning against a national and, if possible, international benchmark.

8. Assessment for Learning Learners learn best when ... • they understand clearly what they are trying to learn, and what is expected of them • they are given feedback about the quality of their work and what they can do to make it better • they are given advice about how to go about making improvements • they are fully involved in deciding what needs to be done next, and who can give them help if they need it

9. Assessment as learning • Share learning intentions, success criteria, Learning Goals • Learning through dialogue and self and peer assessment, effective questioning • Metacognition- Learners become aware of how they learn. • Learners participate more in the process of learning. 

10. Assessment of learning Judgments about pupils’ learning and progress need to be dependable. This means that: • they are VALID (based on sound criteria) • they are RELIABLE (accuracy of assessment and practice)  • and they are COMPARABLE (they stand up when compared to judgments in other departments or schools).      

11. Principles • Assessment should be valid • Assessment should be reliable and consistent • Assessment information should be explicit, accessible and transparent • Assessment should be inclusive and equitable • Assessment should be an integral part of curriculum design relating directly to the O’s and E’s • Assessment should be manageable • Assessment should promote improvements in learning by providing timely feedback • CPD in Assessment should be planned for

12. Polarised views • “Teacher assessment is unfair because it is unreliable and biased” • “Exams are simply snapshots and are unrepresentative of the work that has really been done”

13. Trusting teachers’ judgementHarlen 2005 “The findings of the review by no means constitute a ringing endorsement of teachers’ assessment; there was evidence of low reliability and bias in teachers’ judgements”

14. Assessment Hypothesis The best teachers will make the best judgments about pupils’ achievements and what next steps they need to do to take to improve further.

15. Use of assessment information WE DO WANT: • Validity, Reliability • Predictability • Public and professional confidence in our assessment system • Clarity for learners and their parents in the way assessment is reported • Clear impact on pupils progress

16. Use of assessment Information WE DO NOT WANT: • League tables • A blame culture • Increased teacher workload • Constant high stakes testing which limits creativity in the curriculum

17. Personalised E-assessment • Does NOT replace teachers’ judgement • Does NOT replace all the other assessment information • Does Not lead to a blame culture • Does NOT reduce creativity in the classroom • Does significantly reduce teacher work load • Improves reliability and validity assessment of judgements • Provides diagnostic information on pupils progress, class, year, school, LA

18. Assessment: which matters most? • Economy/Society   • LA    • School     • Teacher    • Parents       • Pupil       

19. Diagnostic – multi level • Macro diagnostic – what you do with schools and cohorts (E1) • Micro diagnostic – what you do with pupil information (E1)

20. Macro diagnostic More deprived Less deprived

21. Q: does this relationship persist across years?

22. Macro diagnostic

23. Macro diagnostic

24. More deprived Less deprived

25. Micro diagnostic

26. Micro diagnostic

27. Goodhart’s Law When a measure becomes a target, it ceases to be a good measure.

28. Teacher reliability • How should reliability be assessed • By looking at the internal consistency of judgements? • By looking at the link to external assessments? • By comparing over time? • By comparing one teacher with others? • Facets model within Rasch measurement

29. Assessment: Predictive validity Personalised E-assessment Summative test (Std Grade/ Highers) Teachers/pupils Assessment (with moderation) Later success – job, degree, salary etc

30. Finding the Middle Ground • AIFL • Communicating clearly with parents, Reporting • Defining the standards/success criteria (child development) • Personalised e-assessment • Meaningful Moderation (reliable, valid) • Senior Phase- assessment systems and qualifications systems are contiguous • Sensitive but serious accountability - analysing the LA/school/teacher effect

31. Pragmatism: We need to • Actively promote AIFL strategies • Know what success looks like • Have effective policy on the use of assessment information • Reduce teacher workload but improve the reliability and validity of the assessment system • Communicate clearly with parents • Measure what we value and celebrate progress • Ensure that pace and challenge are not dissipated • Adapt personalised e-assessments to CFE outcomes • Be clear on what we mean by moderation • Ensure that assessment systems and qualification systems cohere

32. Pragmatism : KISS principle challenge A new language (BTC5) Application Higher order thinking skills HOW WELL HOW MUCH LEARNERS’ RATE OF PROGRESS breadth

33. Pragmatism – Action Plan • Draft reporting template will be disseminated for consultation (June) • Communication strategy for parents is developed (June) • Timetable for the phasing out of 5-14 assessments (June) • A Fife Success Criteria (the standards) will be developed for Numeracy and Literacy • The Implementation plan for moderation and embedding AIFL strategies, • E- assssment – All our schools for May/June 2011 in Reading, and Numeracy is established. P1,3,5,7,S2/3 • A draft Curriculum and Assessment/ Qualification paper supporting the Senior Phase and the revised SQA framework to be developed (October)

34. Appeal: Finding a middle way Support Leadership Accountability for Children’s development