1 / 17

Linking of the DESI Writing Test to the CEF

DESI English. Linking of the DESI Writing Test to the CEF. Claudia Harsch University of Augsburg Department of Didactics of English - Prof. Dr. Konrad Schröder claudia.harsch@phil.uni-augsburg.de. Overview.

tobias
Download Presentation

Linking of the DESI Writing Test to the CEF

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. DESI English Linking of the DESI Writing Test to the CEF Claudia HarschUniversity of AugsburgDepartment of Didactics of English - Prof. Dr. Konrad Schröder claudia.harsch@phil.uni-augsburg.de EALTA Conference Krakow

  2. Overview ? What is to be linked to the CEF levels – the writing tasks or the proficiency scale? cf. EALTA discussion half a year ago: no general agreement yet, depending on tasks DESI: Open tasks which allow broad range of performance levels, not targeting a specific CEF level but reflecting German Curricula Linking of DESI proficiency levels to CEF levels

  3. Overview ? Why link the DESI levels to CEF levels: allows comparison of results across Europe empirical basis to decide about level of proficiency of 9th graders in Germany could be used for evaluation of German Educational Standards

  4. Overview Linking procedure: Work in Progress • Specification of the DESI writing tasks • Standardisation:Data collection, proficiency scaleComparison DESI levels – CEF levelsTo be adressed: Expert Ratings • Empirical Validation: To be adressed

  5. 4 writing two tasks in 40 minutes grade 9 of German school system handwritten assessment of general writing proficiency Specification of writing tasks: ALTE Grid Usage of ALTE grid, e.g.:

  6. Standardisation: Data Collection • 219 schools, two classes per school • All four tasks in one class, each student worked on two tasks • 20.000 performances • Double rating of each performance by trained raters using DESI rating scales constructed for each task: 40.000 ratings

  7. Standardisation: Proficiency Scale • Scaling of ratings, accounting for rater harshness and task difficulty (Prof. Lehmann / A. Neumann at Humboldt University at Berlin) • Scaling resulted in one global “writing” factor • One proficiency scale with five levels

  8. Standardisation: Proficiency Scale

  9. Standardisation: Proficiency Scale

  10. Standardisation: Comparison with CEF scales

  11. Standardisation: Preliminary Claim

  12. Standardisation ? How can the DESI levels be linked to the CEF levels in a reliable and valid way? Workshop recommendation: Use expert judgements to place given performances on the CEF levels. We could then use our existing data and, based on expert judgments, link the DESI levels to the CEF levels.

  13. Standardisation ? How can performances be placed on the CEF levels in a valid way, since the CEF levels are not reflecting performance characteristics but rather general descriptions of proficiency? Workshop recommendation: Use writing grid in the Manual, p.82

  14. Standardisation – to be adressed 1. Sample size: four levels to be linked four tasks involved ? Are400 performances enough? 100 performances per level 100 performances per task ? Two tasks of one student? 2. Experts to place the performances on the CEF levels

  15. Outlook Empirical Validation to be adressed in cooperation with the IQB

  16. Thank you for your attention. claudia.harsch@phil.uni-augsburg.de

  17. Selected Bibliography ALTE CEFR Grid for the Analysis of Writing Tasks: http://www.coe.int/T/E/Cultural_Co-operation/education/Languages/Language_Policy/Manual/, visited 13.09.2005. Carl Bereiter: “Development in Writing“. In: Lee W. Gregg & Erwin R. Steinberg (eds): Cognitive Processes in Writing. Hillsdale, NJ: Erlbaum 1980, 73-93. Council of Europe: A Common European Framework of Reference for Language Learning and Teaching. Strasbourg: 1996. Claudia Harsch: Der Gemeinsame europäische Referenzrahmen: Leistung und Grenzen. Die Bedeutung des Referenzrahmens im Kontext der Beurteilung von Sprachvermögen am Beispiel des semikreativen Schreibens im DESI-Projekt. Inaugural-Dissertation (forthcoming). Claudia Harsch, Rainer H. Lehmann, Astrid Neumann & Konrad Schröder: „Übergreifende Konzeptualisierung sprachlicher Kompetenzen: Schreibfähigkeit.“ In: Beck, Bärbel & Eckhard Klieme (Hrsg.): Sprachliche Kompetenzen: Konzepte und Messung. DESI-Studie. Weinheim: Beltz (in press). 50-72. KMK-Bildungsstandards, visited 13.09.2005: http://www.kmk.org/schul/Bildungsstandards/Rahmenvereinbarung_MSA_BS_04-12-2003.pdf http://www.kmk.org/schul/Bildungsstandards/Deutsch_MSA_BS_04-12-03.pdf http://www.kmk.org/schul/Bildungsstandards/1.Fremdsprache_MSA_BS_04-12-2003.pdf, Manual for Relating Examinations to the CEF: http://www.coe.int/T/E/Cultural_Co-operation/education/Languages/Language_Policy/Manual, visited 13.09.2005

More Related