1 / 8

10. Validation

RELANG Relating language examinations to the common European reference levels of language proficiency: promoting quality assurance in education and facilitating mobility. 10. Validation

hashim
Download Presentation

10. Validation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. RELANGRelating language examinations to the common European reference levels of language proficiency: promoting quality assurance in education and facilitating mobility 10. Validation European Centre for Modern Languages and European Commission cooperation on INNOVATIVE METHODOLOGIES AND ASSESSMENT IN LANGUAGE LEARNING

  2. LinkingProcedures in the Manual • Familiarisation with the CEFR • Linking on the basis of specification of examination content • Standardization and Benchmarking • Standard setting • Validation: checking that exam results relate to CEFR levels as intended

  3. What is validity? Validity refers to the degree to which evidence and theory support the interpretations of test scores entailed by proposed uses of tests. In simpler terms: does the test measure what it intends to measure?

  4. Aspects of Validity • Content Validity • Operational validity: pilots and pretests • Psychometric aspects • Procedural validity of standardization • Internal validity of standard setting • External validation

  5. Content Validity • Does the test accurately reflect the syllabus on which it is based AND reflect the descriptors in the CEFR? • Does the content specification reflect all areas to be assessed in suitable proportions?

  6. Quality Criteria for Items (1) An item must be: • relevant • at intended level • specific • objective • acceptable • transparent • efficient • in correct language • in a clear lay-out

  7. Quality Criteria for Items (2) Relevance • Is the item addressing the intended knowledge or skill at the intended CEFR level? • Does the item not test other knowledge and abilities than the intended ones (e.g. reading skill, general intelligence, knowledge of grammar)? How to realize: • Refer to specific CEFR descriptors for each skill • Use test matrix (or test grid) • Relate questions with purpose of the test • Make items that are recognizable for the student

  8. Validation of Standard Setting • Has the procedure of standard setting had the effects as intended: was the training effective, did the judges feel free to follow their own insights? These are questions of procedural validity. • Are the judgments of the judges to be trusted? This relates tothe internal validity of the standard setting. • Are the results of the standard setting – allocating students to a CEFR level on the basis of their test score – trustworthy? The basic answer to this question comes from independent evidence which corroborates the results of a particular standard setting: empirical validity.

More Related