570 likes | 1.1k Views
Bridging the Gap between Theory and Practice?. J Charles Alderson Lancaster University. What Gap?. Between What and What? Whose View? From what Perspective? Should the Gap be Filled? By Whom? When and How?. Conference claim.
E N D
Bridging the Gap between Theory and Practice? J Charles Alderson Lancaster University
What Gap? • Between What and What? • Whose View? From what Perspective? • Should the Gap be Filled? • By Whom? • When and How?
Conference claim • In the last 25 - 30 years, research in the field of Language Testing has grown exponentially. • A perennial concern, however, is that the day-to-day practice of language testing has not changed as quickly as our understanding of the theory of language testing. • Language testing practitioners complain that research reports are hard to understand and often address matters that are irrelevant or at least far removed from their daily concerns. • Researchers and practitioners seldom co-operate. We sense, however, that this climate is changing and we think that EALTA can contribute to and perhaps accelerate the pace of change.
The dichotomies • Long-standing – as old as the hills • But true? • Beware dichotomies • Is action research a solution? Exploratory practice? • Can theory ever be relevant?
Is the first term always better than the second? Or the second better than the first? • Is reality better than what we aspire to? • Are the two in conflict? • Why opposition? Why not synergy?
Is practice necessarily “behind” research? • Can theory not learn from practice? • Is research necessarily hard to understand? • Must everything be applicable to all realities tomorrow? • Perhaps theory can help us understand practice, rather than be directly applicable
Is rejection of theory and research mere Philistinism? A refusal to try to understand, a rejection of “jargon” without going beneath the surface? • A result of practitioners’ unwillingness to come to grips with unfamiliar concepts? • Are theorists and researchers remote from everyday practice?
Challenge • Is there such a gap in language testing? • Between theory and practice? • Research and the concerns of practitioners? • Test researchers and test developers? • Testers and Teachers? • Official claims and actual practice? • The Ideal and the Real?
Does the Gap exist? • And if so, is it a problem?
Research articles: irrelevant? • The effect of test-taker gender, audience and topic on task performance in tape-mediated assessment of speaking • Examining the predictive validity of a screening test for court interpreters • Do visual chunks and planning impact performance on the graph description task in the SPEAK exam?
Research articles: irrelevant? • To change or not to change: investigating the value of MCQ answer changing for Gulf Arab students • Relating examinations to the Common European Framework: a Manual • Limitations of the Common European Framework for developing comparable examinations and tests
Research articles: irrelevant? • The development of a suite of computer-based diagnostic tests based on the Common European Framework • The Common European Framework and the European Language Portfolio: involving learners and their judgements in the assessment process • Assessing the language of young learners
Research articles: irrelevant? • Progress and problems in reforming public language examinations in Europe: cameos from the Baltic States, Greece, Hungary, Poland, Slovenia, France and Germany • Language testing in the military: problems, politics and progress • Dark alleys and blind bends: testing the language of learning
Research articles: irrelevant? • Stakeholders' conflicting aims undermine the washback function of a high-stakes test • Reading to learn and reading to integrate: new tasks for reading comprehension tests? • Linguistic sources of item bias for second generation immigrants in Dutch tests • Generalizability of writing scores: an application of structural equation modeling
Research articles: irrelevant? • Did they take the same test? Examinee language proficiency and the structure of language tests • Self-assessment, preparation and response time on a computerized oral proficiency test • Peer assessment of language proficiency • Building and supporting a case for test use
Research articles: irrelevant? • The impact of wearing a face mask in a high-stakes oral examination: An exploratory post-SARS study in Hong Kong • Examining rater effects in TestDaF writing and speaking performance assessments: A Many-Facet Rasch analysis • Individual feedback to enhance rater training: Does it work?
Research articles: irrelevant? • Re-thinking second language admission requirements: Problems with language residency criteria and the need for language assessment and support • Resolving score differences in the rating of writing samples: Does discussion improve the accuracy of scores?
Testers are remote from practitioners?The example of ENLTA • 1 year experience in secondary (12 year olds) • 17 years experience teaching adults • 10 years experience teacher-training (primary,secondary, adult)
Testers are remote from practitioners?The example of ENLTA • Teaching languages for 25 years: • 9 years training school • 5 years secondary school • 11 years primary school
Testers are remote from practitioners?The example of ENLTA • 3 years teaching in evening classes/study circles • 6 months teaching in secondary school • 2 months teaching in upper secondary school • Two summer courses for adult students • 15 years teaching in municipal adult education • 20 years in-service training for teachers • 15 years teaching language didactics and assessment at university level
Testers are remote from practitioners?The example of ENLTA • 2 years teaching language courses to adults and children • 2 years teaching part-time continuing courses for people who did not graduate from a primary, secondary or a vocational school; • 1 year teaching secondary school pupils; • One-to-one language teaching (mostly at home), usually helping school children to improve their language skills • 6 years teaching university students language classes, and language testing,
Testers are remote from practitioners?The example of ENLTA • Teacher in international language school • In-company business English teacher • Kindergarten/primary school teacher in private school • Temporary EFL summer school teacher • Temporary pre-sessionals • Teacher Trainer British Council Teaching Operation • Director of Studies, University Language Teaching centre, Language courses for teachers • Undergraduates Teacher Training College • Private teaching: young learners 7-14; secondary school learners 16-19; adults
Testers are remote from practitioners?The example of ENLTA • 3 years teacher of English at primary level, • 10 years EFL teacher, technical secondary school, • 3 years EFL teacher, grammar secondary school, • 9 years EFL teacher, private language school • More than 15 years private one-to-one English teaching
ENLTA was remote from practitioners? Pre conference workshops • Krakow, 2006 • Assessing speaking at B1 level • Designing and reporting research • Voss, 2005 • Using the European Language Portfolio • Classical test analysis
ENLTA was remote from practitioners? • Activity Two: Survey of training in testing • Activity Four: Survey of attitudes to testing
Only high-stakes proficiency testing? Irrelevant to practitioners? • DIALANG – no stakes, freely available, diagnostic • New school-leaving examinations • Training teachers in modern approaches to language testing
Only high-stakes proficiency testing? Irrelevant to practitioners? • Increasing interest in diagnosing students’ strengths and weaknesses • Researching what language develops as learners progress through the CEFR levels • Washback of tests on teachers and learners • Teachers’ practices in assessing learners in class
Importance of classroom testing • Increased interest in researching classroom testing and understanding the issues and the needs. • Important to understand better both what actually happens in classsroom assessment and what should or could happen, how it might be improved. • EALTA’s mission is to contribute to this.
Gaps? Codes of practice as the interface with reality • Many Codes of Practice already (ILTA, AERA/APA, JALT, ALTE, etc etc) • Who needs another one? • Who monitors compliance, or even activity? • What use a code of practice that is not implemented?
Gaps? Codes of practice and Reality • Alderson, J.C. and G. Buck (1993). ‘Standards in testing: a study of the practice of UK examination boards in EFL/ ESL testing’. Language Testing 10(1), 1-26
LINKAGE TO THE COMMON EUROPEAN FRAMEWORK • What evidence is there of the quality of the process followed to link tests and examinations to the Common European Framework? • Have the procedures recommended in the Manual and the Reference Supplement been applied appropriately? • Is there a publicly available report on the linking process?
Gaps? Claims about links with the CEFR and reality • Importance of CEFR in testing, training, publishing and curricula • Many claims of links to CEFR • Who monitors quality? • Council of Europe? • ALTE? • Self-monitoring? • EALTA
Gaps? Claims about links with the CEFR and reality • Role of EALTA as Europe’s prime, independent body of language testing practitioners? • How? Where? When? Who? How much? What? • Why? Why Not?
Bridging the Gap between Theory and Practice? • Necessary? • Classroom formative and summative testing and assessment • Codes of practice • CEFR