Gathering Evidence in External Student Assessment José Noijons Cito, Institute for Educational Measurement, the Netherl

1. Gathering Evidence in External Student Assessment José Noijons Cito, Institute for Educational Measurement, the Netherlands

2. Cito Activities • Measurement of skills, competences, achievement and aptitude • Monitoring student development • Development and production of assessment instruments, examinations, tests & items and educational surveys • Training & Consultancy • Research & Development

3. Cito clients (1) Dutch Ministry of Education, Culture & Science • National assessments (primary education) • School leaving examinations (secondary education) • State examinations for Dutch as a second language • PISA (National) • Relating exams to the CEFR Dutch Schools - primary, secondary, vocational, higher education • Monitoring systems • Proficiency tests, Placement tests • Portfolios

4. Cito clients (2) International: Authorities and Organisations • Consultancy and training (e.g. Governments) • Pisa International (OECD) • Language Indicator (EU) Groups and individuals • Courses and trainings at Cito or in company • Certification Private companies • Assessment and testing instruments • Examination logistics

5. International Experience • In Central European countries such as: Czech Republic, Croatia, Hungary, Macedonia, Montenegro, Rumania, Slovenia, Poland • Development of new Matura • Development of National Assessment • Funded by National Governments, the Dutch Government, EU, World Bank

6. World Bank Context • Modernization of National Education Systems • Recent Series of Publications on National Assessment of Educational Achievement • Designed to address many of the issues involved in making learning outcomes a more central part of the educational agenda

7. Two Types of External Assessment National Assessment • designed to describe the achievements of students in a curriculum area, providing an estimation of the achievement level in the education system as a whole National Examinations (cf. Matura) • designed to certify student achievement, in selecting students for further study, and in standardizing what is taught and learned ins schools

8. Differences between the two types

9. Issues in External Assessment (1) • Design • Relations with stakeholders • Determine needs and fears of stakeholders • Look for support • Implementation • Description of content, exemplar tests • Involvement of professionals • Validation of tests, field trialling

10. Issues in External Assessment (2) • Analysis • Expertise statisticians • Quality of data (data cleaning) • Analysis of data • Reporting • Timely • Relevant to interest groups • Technical manual for repetition

11. Where things may go wrong! • Political context • Quality of measurement • Range of difficulties in items • Relation to Curriculum • Equivalence of assessment • Lack of motivation among students • Standards and cut scores • Reporting to stakeholders • Lack of appropriate follow-up

12. References • Greaney, Vincent, Thomas Kellaghan, (2009). Assessing National Achievement Levels in Education. World Bank, Washington • Kellaghan, Thomas, Vincent Greaney, T. Scott Murrey (2009). Using The Results Of A National Assessment Of Educational Achievement. World Bank, Washington