The present and future of assessment systems

1. The present and future of assessment systems

2. About the Group City & Guilds City & Guilds NPTC The Institute of Leadership & Management (ILM) City & Guilds Centre for Skills Development (CSD)

3. Our vital statistics

4. What is an International Vocational Qualification? Competence-based awards that combine underpinning knowledge with practical assessment designed for use internationally with generic occupational standards. Holders of an IVQ can practise and work effectively at the level achieved.

5. Europass Certificate • Europass Certificate Supplement • City & Guilds is the first UK awarding body to launch the Europass Certificate Supplement • Provides additional information to a vocational certificate by describing skills City & Guilds graduates have obtained • Easier for learners to find employment or training beyond borders of their own country • Helps employers choose the best candidate to work for their organisation

6. Seven things that might make an assessment system work

7. What is an assessment system? • A collection of exams, tests and qualifications. • During schooling (~11 or 14) • accountability testing: whole cohort or sampling • End of schooling (~16+ and 18+) • academic qualifications/university entrance exams • Vocational qualifications • Alternative to academic path in schools; workplace qualifications, licence to practice

8. Complications … • Not always a national system • Regional involvement (eg Poland, Switzerland, Spain) • Globalisation is becoming a feature • International systems (PISA, TIMSS) influence countries (Norway, Germany) • International, private-sector providers becoming more prominent (IB; Microsoft) • Doesn’t have to be examinations • Teacher assessment (TA) can have major role • Many countries have mixture of exams and TA (Netherlands, Queensland – Aus) • Lots of different systems; lots of change within systems • What makes a system successful?

9. Seven things that mightmake an assessment system work • Have regard to history, traditions and future. • Involve people in decision making. • Specify what the system is for. • Try to make a fair and balanced system. • Get the best technical experts to design it. • Acknowledge that it can never be perfect. • Support it.

10. Have regard to history, traditions and future needs • Policy learning is better than policy borrowing • UK government Numeracy Strategy 1998 influenced by mathematics teaching in Taiwan • But English kids are different to Far Eastern kids • What works in one culture unlikely to work in another, unless very careful attention to context

11. Involve people in decision making • Simple idea • Stakeholders (teachers, parents, employers, children and young people) have a point of view and insights • If they can express their views, they are more likely to accept the system

12. Specify what the system is for • Purposes can get ‘loaded onto’ an assessment system over time • Don’t overload your assessment system • National curriculum assessment (England) • 1988 started with four purposes • 2007 it had 22 purposes • Purposes can work against each other • eg • Provide formative feedback to pupils (teachers highlight pupils’ weaknesses) • Evaluate teaching quality and publish league tables (teachers hide pupils’ weaknesses)

13. A fair and balanced system • English schools’ exam and test results are published in league tables • Encourages the idea of competition, winners and losers • Also gives teachers a natural reason to oppose the national testing system • Teachers’ supporters (unions, academics) campaigned against the tests for many years • Alienating major group of stakeholders makes it more difficult for the system to succeed

14. Technical design ofassessment systems • Assessments are sophisticated technologies • Reasonable to demand that technical experts set them up • Expect transparency, but also sophisticated measurement science • Einstein: ‘as simple as possible, but no simpler’

15. It can never be perfect … • Human beings are complex. • Assessment provides a usable summary of a human’s abilities. • Error is inherent in assessment. For example: • Strict and lenient markers • Strict or lenient awarding panel • Selection of questions in a test • Candidate has a good day or a bad day • Good assessment systems can reduce the amount of error, but not eliminate it entirely. • Users have to have realistic expectations of what assessment systems can do.

16. Support the system • A beautiful English summer sport • An ugly English summer sport • Many commentators criticise assessment systems • politicians, journalists, academics, teachers, parents, union leaders, former education advisors • All parts of the system • national tests, school-leaving exams, vocational qualifications • From variety of perspectives (eg different political viewpoints) • It is important to report problems with qualifications systems • But we also need balance • What is reasonable for a qualifications system? • Error is inherent, practical problems will crop up. • Are things better elsewhere in the world? • Did ‘the good old days’ ever exist? • A qualification is a currency – like money. • It is only valuable if people trust it. • Commentators need to give the benefit of the doubt to young people and education as a public good.

17. A vision of the future

19. Randy Bennett's Re-invention • Bennett is a leading researcher from Educational Testing Services, USA • Has done a lot to define electronic assessment (e-assessment) • Re-invention is a ‘blue skies thinking’ paper • Freely available on the internet • Quite short, non-technical • Written in the late 1990s • Not necessarily correct, but inspirational and visionary • Invites readers to formulate their own responses

20. The situation ‘now’ • Large-scale testing unchanged for many years • Serve institutional purposes • Make little use of technology • Based on dated psychological models • Impact of wider economy • Intensified competition • National and global competitors • Need for: innovation, productivity, customer service • Populations are becoming more diverse – expect products and services to reflect this.

21. Three generations of e-assessment (1)

22. Three generations of e-assessment (2)

23. Three generations of e-assessment (3)

24. Some aspects of ‘generation ‘R’’ (1) Computerized assessments … will be embedded in the school curriculum and occur frequently. Sometimes an assessment will be made known in advance; at other times – with informed consent – it will simply be embedded seamlessly in the … session and be indistinguishable from the instructional components of that session.

25. Some aspects of ‘generation ‘R’’ (2) In the current model, … large-scale tests are too often divorced from schooling in both content and delivery. … By virtue of moving assessment into the curriculum, the locus of the debate over performance differences must logically shift from the accuracy of assessment to the adequacy of instruction.

26. Some aspects of ‘generation ‘R’’ (3) … Assessment runs the risk of falling out of synch on multiple counts, … Change, unfortunately, does not always come easily or as quickly as it should to well-established institutions. But large-scale educational assessment must change in the most fundamental ways, for nothing short of reinvention will prepare it to meet the dramatically different demands it will soon face.