TASK-BASED ASSESSMENT IN LANGUAGE LEARNING PROGRAMS: PIECES OF THE PUZZLE

1. TASK-BASED ASSESSMENT IN LANGUAGE LEARNING PROGRAMS: PIECES OF THE PUZZLE Geoff Brindley Macquarie University, Sydney, Australia

2. Structure of the presentation 1. Introduction 2. Task-based assessment: Definitions and features 3. Implementing TBA: Differing stakeholder perspectives -Theoretical perspectives -Educational versus managerial perspectives 4. The need for assessment literacy 5. Conclusion

3. Defining task-based assessment “… the process of evaluating, in relation to a set of explicitly stated criteria, the quality of the communicative performances elicited from learners as part of goal-directed, meaning-focused language use requiring the integration of skills and knowledge” (Brindley, 1994) • UK -Teacher assessment to be used for statutory reporting at 11 and 14 in Wales • US

4. Positive features of TBA • Teachers' and learners' attention is able to be more focused on language as a tool for real world communication rather than on language knowledge as an end in itself, thus potentially improving student motivation. • Assessment can be integrated into the learning process through the use of attainment targets which are directly linked to course content and objectives.

5. Positive features of TBA (contd) • Learners can obtain useful diagnostic feedback on their progress and achievement since explicit performance criteria are provided against which they can compare their performances. • Better communication between users of assessment information and educational institutions can be established through the use of various forms of outcome reporting which are couched in performance terms and are hence intelligible to non-specialists.

6. The shift to teacher-led assessment • UK -Teacher assessment given greater importance following review of National Curriculum Assessment -Teacher assessment used for reporting from 2007 in Wales -Government “Excellence and Enjoyment” report (DfES 2003) endorses greater role for teachers in assessment -Tomlinson Report (2002) endorses use of teacher assessment for national reporting

7. The shift to teacher-led assessment (contd) • USA -Widespread adoption of “authentic” TBA in schools & adult education -Use of authentic teacher-conducted assessments in some high stakes contexts (eg Kentucky, Nebraska)

8. Differing theoretical perspectives on TBA: What is the construct? • “Weak” versus “strong” view of TBA: “…in the strong sense, tasks will represent real-world tasks, and performance will be judged on real-world criteria, that is, the fulfilment of the task set” (McNamara 1996)

9. Differing theoretical approaches to TBA (contd) • “In second language performance tests in the weak sense, the focus is on language performance….The candidate is required to perform on a task which may represent tasks he or she may subsequently face in the real world; however, the capacity to perform the task is not the actual focus of the assessment”

10. The problem of generalizability • “While the ‘weak’ view is likely to assess underlying language skills in ways which are relatively broadly generalizable, the ‘strong’ view is likely to produce judgments which are more authentic and relevant to the real life situations towards which candidates may be moving. These judgments about the quality of performance may not, however, be replicable in other contexts” (Wigglesworth 2008).

11. Some other unresolved theoretical issues in TBA The difficulty of “difficulty” • How can task difficulty be defined & measured? • What are the factors affecting task difficulty? • Can task difficulty be modelled and predicted? (Bachman, 2002, 2007; Brindley & Slatyer, 2002; Brown et al, 2002; Elder et al 2002; Ellis, 2008; Robinson, 2001; Skehan, 1998, 2001; Tavakoli, 2009)

12. Educational versus managerial perspectives “ Whereas politicians and government officials tend to see assessment as a tool for implementing and managing policy, teachers and educationists are primarily concerned with ways in which it can be used for the improvement of learning “ (Brindley, 2008)

13. Formative TBA: The educational policy maker perspective • Effective assessment for all pupils should: -recognise what pupils can do and reward achievement -be based on different kinds of evidence -be a valid reflection of what has been taught or covered in class -be reliable in terms of enabling someone else to repeat the assessment and obtain comparable results -be manageable, both in terms of the time needed to complete the task, and in providing results which can be reported or passed on to other teachers (DfES, 2003:2)

14. The formative purpose of TBA: rhetoric or reality? No reference to: • monitoring learners’ emerging language awareness and development: achievement is highlighted • assessment as integrated within instructional discourse. “Taught or covered” suggests one-off measurement focused assessment • formative assessment as an on-going process Leung and Rea-Dickins, 2007)

15. Popular beliefs about testing & assessment • There is a test for every population/purpose • Norm-referenced tests are a fair and objective measure of student ability • Standardized testing raises standards • All tests have pass marks (usually 50%) • Teachers can’t be trusted to do their own assessment

16. The media critics • Formative assessment also embraces a developmental approach to learning, based on the argument that "students develop and learn at different rates and in different ways"… • The result? Instead of pass or fail, student progress or lack of progress is clouded by such politically correct terms as beginning, established, consolidating or emerging, solid, comprehensive. • Instead of students facing regular examinations with consequences for failure, as do those students in stronger performing education systems overseas, students are automatically promoted from year to year, even though many have not mastered the basics (Donnelly 2005)

17. The politicians “The reports I saw allowed for the teacher to assess students from a range of choices – usually, consolidating, sometimes, and not yet’. What kind of nonsense is this? The educational “experts” with whom I seem to be in constant battle, give me the constant refrain of “outcomes assessment”. The ranking of students against one another is opposed by teacher advocates. Try telling that to parents. Worse still, what do they think happens in the real world? (Nelson 2005)

18. “ Politicians are simple people; they like simple people; they like simple choices and clear guidance.” (Yes Minister)

19. The role of assessment literacy • “..training for assessment literacy entails an appropriate balance of technical know-how, practical skills, theoretical knowledge, and understanding of principles but all firmly contextualized within a sound understanding of the role and function of assessment within education and society” (Taylor, 2009)

20. Assessment literacy for all “ …an appropriate level of assessment literacy needs to be nurtured not just among engineers and technicians who are actively involved in test development or research activities, or even among applied linguists and language teachers…but more broadly in the public domain if a better understanding of the function and values of assessment tools and their outcomes is to be realized throughout society” (Taylor, 2009)

21. Assessment and the language teacher: What skills & knowledge do they need? • Teachers should be skilled in choosing assessment methods appropriate for instructional decisions. • Teachers should be skilled in administering, scoring and interpreting the results of both externally-produced and teacher-produced assessment methods

22. Assessment and the language teacher: What skills & knowledge do they need? • Teachers should be skilled in developing, using and evaluating valid student grading procedures which use student assessments. • Teachers should be skilled in communicating assessment results to students, educational decision makers and other concerned stakeholders.

23. Assessment and the language teacher: What skills & knowledge do they need? • Teachers should be skilled in using assessment results when making decisions about individual students, planning teaching, developing curriculum, and institutional improvement. • Teachers should be skilled in recognizing unethical, illegal, and otherwise inappropriate assessment methods and uses of assessment information

24. Components of assessment literacy (Bailey & Brown, Brindley, 2001, Inbar-Lourie, 2008, Taylor, 2009) • The social and political context of assessment • Defining and describing language proficiency • Constructing and evaluating tests and assessment tasks • Assessment in the curriculum • Putting assessment into practice

25. Professional development on assessment: some guiding principles (Brindley 2001) • Involve the whole system • Capitalize on existing practices • Recognise and address the reality and constraints affecting assessment • Encourage a research orientation • Plan for change

26. Teacher involvement in collaborative test/assessment task development • “…the discussion time and the time to sit down with people and discuss something has been very valuable. And then I’m discussing with other people who are also interested in these things. And they ask me about tasks sometimes, they bring me a task and say ‘What do you think about this’?”

27. Teacher involvement in collaborative test/assessment task development “ After doing this project, I realized that setting a test paper is not an easy task. Instead of testing what is easy to test, we have to construct tests that we really discover how successful the learning experiences had been for the students rather than to show in what respects they had been deficient” (Teacher participant cited in Coniam 2009).

28. And in conclusion.. • “We believe that a new breed of assessment literate educators with extensive experience in classrooms will play an important role in implementing assessment policies that truly support student learning”. (Lukin et al 2004)