Online CEF-Based Assessment of Oral Proficiency for Intercultural Professional Communication

1. Online CEF-Based Assessment of Oral Proficiency for Intercultural Professional Communication Lutgart Dusar <lutgart.dusar@lscoop.com> Prof.dr Lut Baten <lut.baten@ilt.kuleuven.be>dr. Jan Van Maele <Jan.Van.Maele@groupt.be> www.cefcult.eu

2. 1. CEFcult, the sequel • With CEFcult we want to contribute to • ongoing development & validation of CEFR (scaling language proficiency) and INCA (intercultural competence assessment) frameworks • a new e-learning culture bringing together self-learning, the worlds of work and education • mainstreaming e-Portfolio-based assessment Www.Cefcult.Eu

3. 2. From CEFR (‘93-96) to WebCEF (‘06-09) • WebCEF established a social learning community for self, peer and expert assessment in which both productive and interactive performance in a foreign language is video recorded and online displayed to be assessedbyCEFR-scales. • http://www.webcef.eu/ www.cefcult.eu

4. … to CEFcult (2009 – 2011) + Intercultural Communicative Competence Assessment: • Allowing a set of communication tasks (scenarios) through which foreign language learners can assess their speaking skills for intercultural professional purposes • Map the road to a tested and viable model for online collaborative assessment among HEi, enterprises and various partnership brokers. www.cefcult.eu

5. Scenario Database Scenario e-Portfolio (Europass: Language, Mobility) Add scenario Follow scenario Export to: Select scenario Create sample CEFcultUser Video sample Add profile Social media(LinkedIn) Self assess Invitation to: Feed-back Peer assess Groups(Alumni,..) Expert assess www.cefcult.eu

6. 3. Assessing ‘Intercultural Competence’ • The model: INCA (EU 2001-04) criterion based assessment model, linking Byram’s framework to learning needs of (engineering) industry • Scaling: to what extent INCA criteria including value systems, are developed and put into action: Full – Intermediate – Basic • Respect for otherness • Ambiguity tolerance • Knowledge discovery • Empathy • Behavioral flexibility • Communicative Awareness • Warning “INCA tasks and grid are tentative“ www.cefcult.eu

7. - INCA based online scenarios • Simulations designed in a way to guarantee fairness, reliability and accuracy • Work-sample approach: a selected real-world task transformed into assessment task • Challenge: lack of definition in INCA descriptors www.cefcult.eu

8. - Different values, different ratings ‘I am honest’: • Considered a quality in low context cultures: ‘we say what we mean, we mean what we say’ • Considered ‘blunt’ in high context cultures where a meaning is stated in an indirect way What matters: cultural (self) awareness: • Observe • Describe • Interpret • Suspend your judgment, look for and verify other perspectives before E valuating (S.Ting Toomey) www.cefcult.eu

9. - Pilot Leuven ‘Screening interview’1st experiences(October, November, December 2010) Positives Critical Summative work-sample testing mainly indicates a degree of overall communication competency, much less in detail Performances may meet INCA criteria in two or more categories, making it difficult to select the one best description Peer reviews (student community) do not provide specific feedback to test takers about the strengths and weaknesses of their performance. • Descriptors are written generically, can be used with many tasks • Corresponding rating criteria determined through collaboration of language assessment specialists and (lay) stakeholders from the workplace • Expert raters from the world of work tend to apply scales consistently, resulting in more reliable measurement. www.cefcult.eu

10. - Task specific descriptor writing & review cycle Task specific Peer assessment Task descriptor acceptable, understood? Yes No Internal review Flag(What’s the problem?) Commentexplanationrecommendation

11. 4. A Tested Model www.cefcult.eu

12. 5. A Viable Model Outcome: e-Portfolio based assessment • The emergence of new global actors will make our world more colorful and more polyvalent. There is a need for foreign languages learning, inter-personal & inter-cultural monitoring and development tools in a safe, respectful environment. • Existing ICC-assessment tools mostly address criterions and references for individual competence development; not enough the organizational needs and seldom those of development partnerships (N-S). • E-Portfolio tools must go mainstream, embedded in the internal learning and monitoring processes of companies, organizations and higher education institutions. www.cefcult.eu

13. www.mymahara.com References • John De Jong & Glyn Jones , Getting the levels right: deriving item writer guidelines from CEFR descriptors (Pearson, Item Writer Guidance-extract ppt) • Peter Lenz and Raphael Berthele, Assessment in Plurilingual and Intercultural Education. Paper Policy Forum ‘The right of learners to quality and equity in education – The role of linguistic and intercultural competences’, Geneva, Switzerland, 2-4 November 2010 • Byram • … www.cefcult.eu

14. References • Peter Lenz and Raphael Berthele, Assessment in Plurilingual and Intercultural Education. Paper Policy Forum ‘The right of learners to quality and equity in education – The role of linguistic and intercultural competences’, Geneva, Switzerland, 2-4 November 2010 • Elisabeth Prechtl and Anne Davidson Lund, Assessing and developing intercultural competence. Perspectives from the INCA Project in Handbook of Intercultural Communication, Section 5, ch.22, p.467-490(2007)eBook ISBN: 978-3-11-019858-4 • John De Jong & Glyn Jones , Getting the levels right: deriving item writer guidelines from CEFR descriptors (Pearson, Item Writer Guidance-extract ) • www.cefcult.eu - www.incaproject.org - http://www.webcef.eu/ www.cefcult.eu