1 / 22

Mapping the Dutch Foreign Language State Examinations onto the Common European Framework of Reference Report of a Cito r

Mapping the Dutch Foreign Language State Examinations onto the Common European Framework of Reference Report of a Cito research project commissioned by the Dutch Ministry of Education, Culture and Science José Noijons & Henk Kuijper. Project Goals.

vivi
Download Presentation

Mapping the Dutch Foreign Language State Examinations onto the Common European Framework of Reference Report of a Cito r

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Mapping the Dutch Foreign Language State Examinations onto the Common European Framework of Reference Report of a Cito research project commissioned by the Dutch Ministry of Education, Culture and Science José Noijons & Henk Kuijper

  2. Project Goals • To establish links between the existing examinations in French, German and English and the CEFR, following the steps as outlined in the Manual published by the Council of Europe. • To study the possibilities of developing more comprehensive CEFR-related examinations in the foreign languages.

  3. Four Project Phases Phase 1: Familiarisation Phase 2: Specification Phase 3: Standardisation Phase 4: Empirical validation

  4. Phase 2: Specification Specification of texts • Related to the reading scales for communicative language activities (CEFR chapter 4) • Related to text type, source, topic, domain (Dutch Grid) • Related to the scales for communicative language competence (CEFR chapter 5 & Dutch Grid)

  5. Phase 2: Specification Specification of items • Question types • Task dimensions 1(recognising, inferences, evaluation) • Task dimensions 2(explicit vs. implicit) • Task dimensions 3(content of operations)

  6. Specification texts: reading scales communicative activities German

  7. Specification texts: text type, source, topic, domain • Great variation in text source and communicative themes (topics) • Increase of proportion of expository and argumentative texts from lower to higher educational levels • Domain: mainly personal

  8. Specification texts: communicative language competences • Level of abstraction • Grammatical complexity • Vocabulary • Text length Dutch grid

  9. Specification texts: communicative language competences, level of abstraction

  10. Specification texts: communicative language competences, grammar

  11. Specification items From lower to higher educational level: • more inferences and evaluation, less recognising • Relatively more implicit information • Greater variation of operations

  12. Specification: general conclusions • Increase of linguistic and cognitive complexity of texts • Increase of variation of operations demanded in the items

  13. Claims through Specification and Standardisation If a claim of a link to the CEFR is based on specificationonly, we do not know what score a candidate needs to claim that the candidate’s ability is at the CEFR-level the test claims to be at. Claims can be further substantiated through standardisation of judgements: item-difficulties are judged in relation to CEFR levels.

  14. Standardisation Standard-setting Basket procedure: judges are to assign items (texts + tasks) to CEFR-levels. What minimum CEFR level does one need to master this item?

  15. Results for Exams in English Required minimum CEFR level for Exams in English

  16. Results for Exams in French Required minimum CEFR level for Exams in French

  17. Results for Exams in German Required minimum CEFR level for Exams in German

  18. Scores & Standards: English Distribution of scores & cut score vwo-exam English VWO English Score distribution Score distribution

  19. Scores & Standards: French Distribution of scores & cut score vwo-exam French VWO French VWO French Score distribution

  20. Scores & Standards: German Distribution of scores & cut score vwo-exam German VWO English VWO German Score distribution Score distribution

  21. Conclusions standard-setting • In the Netherlands a candidate can “pass” an exam without attaining the relevant CEFR level for that exam. • Only scores at, or higher than the CEFR cut score indicate that the candidate is at or above the CEFR level the exam has been estimated to be at. • Empirical validation may help to show that a CEFR level (a score) attained on one exam is equivalent to a CEFR level on another exam.

  22. General Conclusions • A test that is linked or validated through specification only, cannot provide sufficient information on how candidates need to perform on the test to claim they have reached relevant attainment targets. • It is necessary through standard setting to compute minimum scores that are needed for candidates to claim they have reached relevant attainment targets. • External validation is needed to verify claims of links to the CEFR.

More Related