National assessment: how to make it better

1. National assessment: how to make it better Dylan Wiliam King’s College London

2. What assessments do we need? ...the model that says ‘learn while you are at school the skills that you will apply during your lifetime’ is no longer tenable. These skills will be obsolete by the time you get into the workplace and need them, except for one skill – the skill of being able to learn. It is the skill of being able, not to give the right answer to questions about what you were taught in school, but to make the right response to situations that are outside the scope of what you were taught in school. We need to produce people who know how to act when they are faced with situations for which they were not specifically prepared. (Papert, 1998)

3. There is only one 21st century skill The test of successful education is not the amount of knowledge that a pupil takes away from school, but his appetite to know and his capacity to learn. If the school sends out children with the desire for knowledge and some idea how to acquire it, it will have done its work. Too many leave school with the appetite killed and the mind loaded with undigested lumps of information. The good school-master (sic) is known by the number of valuable subjects which he declines to teach. Sir Richard Livingstone, President of Corpus Christi College, Oxford, 1941

4. Who are assessments for? • Employers/FE/HE • Parents • Students and teachers

5. What do employers need? • Examination results are used: • To gain information about students’ • skills • competences • attitudes • ‘trainability’ (criterion-referenced uses) • To sort and rank applicants (norm-referenced uses)

6. Why are employers unhappy? • Things are getting better: • average IQ has increased (Flynn effect) • school achievement has increased • But: • needs of work have increased more • link between IQ and exam results is weakening(can’t use exam results as proxies for ‘intelligence’) • teaching to the test has narrowed the curriculum(can’t generalise to things that weren’t tested)

7. Scores Time Lake Wobegon • All the women are strong, all the men are good-looking, and all the children are above average X

8. Improvements are limited • Scores on national curriculum tests in mathematics and English for 11-year olds are increasing • Scores on other, comparable, tests have remained constant • So, improvement is limited to those things that are actually tested

9. Improvements are transient • Proportion of 11 year olds achieving level 4 in mathematics has increased steadily over the last five years • According to Ofsted, 25% of those who achieved level 4 at the end of year 6 fail to achieve the same level at the end of year 7. • So, achievement has improved at age 11, but not at age 12.

10. Goodhart’s law • Every performance indicator loses its usefulness when used as an object of policy • Railtrack • National Health Service • Test and exam results • The clearer you are about what you want, the more likely you are to get it, but the less likely it is to mean anything

11. What do parents need? • Before choosing a school: • “How well will my child do if they go to this school?” • Failing that • “How well did children similar to my child do when they went to this school?” (value-added approach) • Failing that • “How well do the children who go to this school do?” (raw results approach) • Once a child is at school: • “How is my child doing?”

12. What do students & teachers need? • Students need to know: • where they are in their learning • where they are going • how to get there • Teachers need to know • where students are in their learning • what to do about it • When assessment supports all these, it is formative

13. Formative assessment • Frequent feedback is not necessarily formative • Feedback that causes improvement is not necessarily formative • Assessment is formative only if the information fed back to the learner is used by the learner in making improvements • To be formative, assessment must include a recipe for future action

14. Formative and summative • Fine-scaled data that supports formative uses can be aggregated to serve a summative function • Aggregate summative data cannot be dis-aggregated to identify learning needs • Assessment for formative purposes should be the foundation of all assessment in schools

15. The problem • Our current assessment system provides poor answers to the wrong questions, and directs attention on the assessment of learning, rather than assessment for learning • Three main problems • Reliability (too many wrong levels/grades) • Validity (misleading levels/grades) • Focus (summative rather than formative)

16. National Assessment • Should provide parents with meaningful, accurate data on school performance • Should reflect all the desired outcomes of schooling • Should yield information which is comparable across schools • Should not compromise the school’s use of assessment data for formative purposes

17. What can be done? • Retain tests and examinations • Supplement these with other formal (externally marked) assessments: • Extended course-work tasks • Group tasks • Oral and practical assessments • Students assigned randomly to different assessments • Results provide an ‘envelope’ of levels or grades that the school is allowed to award to students (up to the school to propose how).

18. Would this work? • It would be better if only value-added scores were published • And even better if no scores were published • But, even with ‘league tables’ • levels and grades would be based on more than one or two short tests/exams (increased reliability) • the only way to increase levels or grades would be to improve the achievement of all the students on all the tasks, tests & exams (increased validity)