1 / 15

INTRODUCTION

RELANG Relating language examinations to the common European reference levels of language proficiency: promoting quality assurance in education and facilitating mobility. INTRODUCTION

season
Download Presentation

INTRODUCTION

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. RELANGRelating language examinations to the common European reference levels of language proficiency: promoting quality assurance in education and facilitating mobility INTRODUCTION European Centre for Modern Languages and European Commission cooperation on INNOVATIVE METHODOLOGIES AND ASSESSMENT IN LANGUAGE LEARNING

  2. The CEFR Model of Language Use Elements in action-oriented approach: • Actions performed by persons - individuals and social agents • A range of competences, both general and in particular communicative language competences • Various contexts under various conditions and constraints to engage in language activities • Language processes to produce and/or receive texts in relation to themes in specific domains

  3. The user/learner’sCompetences

  4. Language useand the user/learner

  5. Situational and Interactional Authenticity • Situational authenticity refers to the accuracy with which tasks and items represent language activities from real life. • Interactional authenticity refers to the naturalness of the interaction between test taker and task and the mental processes which accompany it.

  6. How toEnhanceAuthenticy • To make an item or task more situationally authentic, the key features of the real-life task must be identified and replicated as far as possible. • Greater interactional authenticity can be achieved in the following ways: • using situations and tasks which are likely to be familiar and relevant to the intended test taker at the given level • making clear the purpose for carrying out a particular task, together with the intended audience, by providing appropriate contextualisation • making clear the criterion for success in completing the task.

  7. Aligning Tests to the CEFR • Starts with adapting the CEFR to the context of the test. • Aligning should not be an attempt to apply the CEFR to any context in the same rigid or mechanical way. • The test developer must be able to justify the way they relate or ‘translate’ the CEFR to their context.

  8. What is Validity? • A test is valid if it measures what we intend it to measure. • This definition has been extended in recent years to encompass the way tests are used, i.e. validity relates to: ‘the degree to which evidence and theory support the interpretations of test scores entailed by proposed uses of tests’ (AERA, APA, NCME 1999).

  9. Validityand the CEFR • Validity comes down to demonstrating that what we claim is true: that a learner reported to be at B1 actually is at B1 according to the evidence we can provide. • If the focus is more on use, then validity evidence will relate to language actually being used for various communicative purposes. • If the focus is more on competence, then validity evidence will relate to cognitive skills, strategies and language knowledge.

  10. ValidityRelatedto Test Development The stages of test production are described as sequential, and each step must be satisfactorily completed if the final inference is to be valid. Chain of reasoning in a validity argument (adapted from Kane, Crooks and Cohen 1999; Bachman 2005)

  11. What is Reliability? • Reliability in testing means consistency: a test with reliable scores produces the same or similar results on repeated use. This means that a test would always rank-order a group of test takers in nearly the same way. • Note that high reliability does not necessarily imply that a test is good or interpretations of the results are valid. A bad test can produce highly reliable scores.

  12. Reliability in Practice • Be aware of the likely sources of error, and do what is possible to minimise it. • Using statistics to estimate the reliability of a test’s scores is an important post-hoc step. • There can be no reliability target for the scores of all tests, because reliability estimates are dependent on how much test taker scores vary. • Reliability estimates can depend on the item or task type and the way it is marked.

  13. Fairness • An objective for all test and examination providers is to make their test as fair as possible. • Various bodies have produced Codes of Practice or Codes of Fairness to assist test providers in the practical aspects of ensuring tests are fair. • Test providers can try to minimise bias when designing tests. A list of topics to avoid in test items can be given to item writers.

  14. Ethical Concerns • High-stakes language tests can have negative consequences for individuals, e.g. in the context of migration, where decisions made about a person on the basis of a test score can have serious and far-reaching consequences. • Test providers must ensure that the relevant principles are widely disseminated and understood within their organisations. This will help to ensure that the organisation complies with ethical guidelines.

  15. The Stages of Test Development andUse

More Related