International Conference on Task-Based Language Teaching Leuven 2005

1. TASK-BASED ASSESSMENT: INTEGRATING RESEARCH AND PROFESSIONAL DEVELOPMENTGeoff Brindley, Stephen Moore, Helen Slatyer& Steven Ross Macquarie University, Sydney, Australia International Conference on Task-Based Language Teaching Leuven 2005

2. Background:TheCertificates in Spoken and Written English(CSWE) • The Adult Migrant English Program (AMEP) • Competency-based curriculum and assessment framework • Explicitly described learning outcomes at four proficiency levels

3. CSWE assessment: Validity and reliability issues • Construct validity (cf listening/ reading descriptors: can invisible operations be described? • Comparability of teacher-developed tasks • Consistency of rater behaviour • Generalizability of skills across task types

4. Research findings • Speaking -Effects of interlocutor behaviour on test scores -Effects of task type on difficulty (cf structured vs - unstructured) • Listening -Effects of task characteristics on scores (cf speed, item format) • Writing -Problems with interpretability of performance criteria -Rater halo effect • Rater inconsistency

5. Research findings (contd) TASK DIFFICULTY: TEACHER PERCEPTIONS OF FACTORS AFFECTING DIFFICULTY • Lack of fit between task and text • Task type design and cognitive load • Ambiguity in question wording

6. Research findings (contd) LEARNER PERCEPTIONS OF FACTORS AFFECTING TASK DIFFICULTY • Match between text and question • Ambiguity in question wording • Lexical knowledge

7. Creation of an assessment resource for teachers Aims • To develop a bank of trialled assessment tasks to be made available on-line • To create a program of professional development in assessment task design

8. Creation of an assessment resource (contd) Task development process • Tasks sourced from teachers or new tasks developed by NWG • Tasks vetted, modified and piloted by NWG • Tasks moderated by panel of teachers and academics • Tasks trialled nationally

9. Addressing professional development needs • Professional development program derived from task development workshops • Devolvement of responsibility to teachers and educational institutions • Workshops provide a forum for exchange • “It has been extremely useful and rewarding to be involved in the national think-tank” • “(one of the most valuable aspects is) the opportunity to meet people round the country”

10. Future developments Task bank • Improve the quality of tasks • Increase the number of tasks • Extend coverage of LOs to acccommodate curriculum changes

11. Future developments (contd) Professional development program • Online PD kit: resources for local PD -Choice of 3 workshops 1. Task design issues 2. Task quality evaluation 3. Development of new tasks

12. Future developments (contd) Professional development program • PD kit includes Powerpoint presentations, assessment materials and readings • Develop workshop formats according to needs • Develop interactive self-access package

13. Future developments (contd) Research • Refine think aloud protocol analysis • Investigate effect of specific task/item types on perceptions of difficulty

14. Conclusions Four key principles underpin project: • Partnership between educational institutions, teachers and university research centre • Devolvement of responsibility to institutions & teachers through workshop program • Sustainability of the project through a commitment from stakeholders in government and educational institutions • Ongoing program of collaborativeresearch