PRESENTER: TRAN THI HIEU THUY SUPERVISOR: DR. TO THI THU HUONG

1. EVALUATION OF AN END-TERM LISTENING TEST FOR FIRST YEAR MAINSTREAM STUDENTS OF ENGLISH DEPARTMENT – COLLEGE OF FOREIGN LANGUAGES – VNU PRESENTER: TRAN THI HIEU THUY SUPERVISOR: DR. TO THI THU HUONG

2. DESIGN OF THE STUDY • Introduction • Development • Chapter I: Literature Review • Chapter II: The Study • Chapter III: Data Analysis, Discussion of findings and Recommendations • Conclusion

3. Compilation and implementation of “Practise your listening skills” for first-year students of ED: easy-to-follow organization, stimulating contents and useful inputs Realization of the importance of incorporating authentic data into teaching listening (Nunan, 1999) Change in assessment: Listening portfolio (35%) + Mid-term test (15%) + End-term test (50%) Evidences from worldwide studies suggesting the existence of test impacts on teachers and students (Shohamy et.al. (1996), Cheng (1995) and Wantanabe (1996)) Little concern paid to the washback effects of the end-term test on teachers and students of Division 1 - ED Rationale Topic for MA thesis: Evaluation of an end-term listening test for first year mainstream students of English Department – College of Foreign Languages – Vietnam National University

4. Scope of the study Washback effects ofthe end-term listening test of the second semester that K41 students (academic year 2007-2008) sat for on teachers and students.

5. Research questions (1) What are the washback effects of the test on teachers’ teaching content, teaching methodology, attitudes and behaviors? Are they positive or negative? (2) What are the washback effects of the test-taking experience and test results on K41 students in terms of their learning content, learning progress, self-image, motivation, learning attitudes and their relationship with teachers? Are they positive or negative?

6. Chapter I: Literature Review I.1. Language testing in communicative approach I.2. Achievement Language Test I.3. Testing listening comprehension skill

7. Washback effects of a test • (Hughes, 2001, p.1): The effect of testing on teaching and learning is normally referred to as backwash • Synonymous term: washback (Buck (1988, p.17), Shohamy (1992, p.513), Bachman and Palmer (1996, pp.29-35), and so on). • Messick (1996, p.241): For a test to have positive washback, its tasks should be criterion samples – that is, “authentic and direct samples of the communicative behaviors of listening, speaking, reading and writing of the language being learnt”

8. Washback effects of a test Bailey (1999): Washback process may concern: • Teachers • Students • Test developers • Teacher educators • Curriculum planners • Administrators • Language inspectors • Test users • Material developers and publishers • Parents

9. Chapter II: The Study II.1. BACKGROUND TO THE STUDY: THE CONTEXT OF TEACHING AND TESTING ENGLISH LISTENING COMPREHENSION OF FIRST YEAR STUDENTS II.1.1. Standard levels of mastery of Listening comprehension skill for first year students II.1.2. An overview of the second-semester English listening program and its testing in Division 1 – English Department – CFL – VNUH II.1.2.1.The second-semester listening program in Division 1 - English Department – CFL II.1.2.2. The testing of listening comprehension for first year students in English Department - CFL

10. Chapter II: The Study II.2. METHODOLOGY OF THE STUDY II.2.1. A qualitative and quantitative research II.2.2. Subjects II.2.3. Methods and procedures of data collection II.2.4. Methods of data analysis

11. Chapter II: The Study II.2. METHODOLOGY OF THE STUDY II.2.1. A qualitative and quantitative research II.2.2. Subjects II.2.3. Methods and procedures of data collection II.2.4. Methods of data analysis • 50 K41 students • 12 teachers of Division 1 – English Department

12. Chapter II: The Study II.2. METHODOLOGY OF THE STUDY II.2.1. A qualitative and quantitative research II.2.2. Subjects II.2.3. Methods and procedures of data collection II.2.4. Methods of data analysis • Data for the study were collected from 6 sources: • Questionnaire survey for K41 teachers; • Questionnaire survey for K41 students; • Class observations of two classes; • Interviews with two randomly selected teachers; • Interviews with two randomly selected students; • Scores from the second semester end-term listening test.

13. Chapter II: The Study II.2. METHODOLOGY OF THE STUDY II.2.1. A qualitative and quantitative research II.2.2. Subjects II.2.3. Methods and procedures of data collection II.2.4. Methods of data analysis • Two groups of data: teachers’ responses and students responses • Themes ascertained • Qualitative analysis: Text rendering (D’Arbon, 2008) • Quantitative analysis: Frequency, percentage, charts

14. Chapter III: Data Analysis, Discussion of findings and Recommendations RESEARCH QUESTIONS (1) What are the washback effects of the test on teachers’ teaching content, methodology, attitude and behavior? Are they positive or negative? (2) What are the washback effects of the test-taking experience and test results on K41 students on their learning content, learning progress, self-image, motivation, learning attitudes and their relationship with teachers? Are they positive or negative?

15. Teachers’ responses Teachingcontent • Fulfillment of class teaching to the listening syllabus • Types of activities carried out during class time • Teaching materials

16. Teachers’ responses Teaching content • Total commitment of listening teachers to the syllabus • Homework checking and In-class practice: essential parts of the lesson • Taught skills: as suggested in the syllabus • Test covering learnt knowledge and skills • Use of past test papers and IELTS practice tasks as teaching materials • Test mentioning and practicing: increase as the test came nearer

17. Teachers’ responses Teaching methodology • Following stages of a lesson and procedure of carrying out the main tasks • Maintaining sufficient time for classroom activities • Being willing to adjust teaching methodology • Mentioning the end-term test to draw students’ attention to the lesson • Spending more time for test practice and feedback in the last weeks of the semester

18. Teachers’ responses Attitudes and behaviors Teachers’ anticipation of changes on their part when having a detailed report of students’ past test scores

19. Students’ responses Impacts of test-taking experience • Students’ perception of task 2 in the test • Students’ decision on changing learning contents

20. Students’ responses Task 2 in the test

21. Students’ responses Changes in learning contents

22. Students’ responses Impact of test result

23. Findings – Answer to question 1 • Washback effects of the test on teachers: • Not “teaching-to-the-test” • A bit much of teachers’ talk about the test  More evidences of positive washback effects than negative ones

24. Findings – Answer to question 2 • Washback effects of the test on students: complicated • Development of learning strategies • Preparation of study plan • Desire to earn good mark • Evidence of “learning for the test”

25. Recommendations to teachers • Guiding students to effective use of previous test papers • Encouraging students to seek for fun in learning listening • Being more sensitive with the content of teacher talk in class • Using teaching materials wisely • Constructing item-analysis report

26. Recommendations to students • Taking it easy when dealing with the test • Using learning materials effectively • Developing the habit of analyzing weak points and making plans to improve them • Seeking for help and support from teachers and peers

27. Conclusion • Comprehensive responses to research questions • Significant contribution to the deep evaluation of one criterion of a good test

28. THANKYOUFORYOURATTENTION!!!

29. INTRODUCTION • Rationale • Aims and objectives • Scope of the study • Research questions • Research methodology • Significance of the study

30. Aims and objectives (1) Investigating the washback effects that the second end-term listening test has on teachers of listening skill for K41 students; (2) Investigating the washback effects that the second end-term listening test has on K41 students of English department; (3) Evaluating whether such evidences of washback (if any) are positive or negatives to teachers and students; (4) Proposing ways to enhance the positive impacts and to minimize the negative effects of the test on teachers’ teaching and students’ learning.

31. Research methodology • Quantitative and qualitative approach • Data collection tools: - Semi-structured interviews - Questionnaires - Classroom observations • Subjects: • 50 students • 12 teachers

32. Significance of the study • provides a reliable and profound background on the matter • helps the researcher gain more knowledge and skills in this field • offers an insight into test washback, an area rarely investigated in a university educational context • highlights the close relationship between teaching - learning and testing

33. Chapter I: Literature Review I.1. Language testing in communicative approach I.1.1. What is a language test? I.1.2. Testing in Communicative Approach I.1.3. Purposes of language testing I.1.4. Washback effect of a test I.1.4.1. Washback: definitions and types I.1.4.2. Test-takers and washback I.1.4.3. Teachers and washback I.1.4.4. Other participants and washback

34. Language testing • Allen (1974, p.313): “a test is a measuring device which we use when we want to compare an individual with other individuals who belong to the same group.” • Heaton (1988, p.5): tests are considered first as “means of assessing the students’ performance” and then as devices to “motivate students.”

35. Communicative competence Canale (1983, p.339, cited in Tran, 2002, p.5) proposes four dimensions of CC: - Grammatical competence - Sociolinguistic competence - Discourse competence - Strategic competence

36. Purposes of language testing • For teachers • evaluate • reveal teaching effectiveness • provide additional materials • For students • detect strong points and weak points • be a motivation

37. Chapter I: Literature Review I.2. Achievement Language Test I.2.1. Definition I.2.2. Kinds of achievement tests

38. Chapter I: Literature Review I.2. Achievement Language Test I.2.1. Definition I.2.2. Kinds of achievement tests • Hughes (2001, p.10): “…achievement tests are directly related to language courses, their purpose being to establish how successful individual students, groups of students, or the course themselves have been in achieving objectives.”

39. Chapter I: Literature Review I.2. Achievement Language Test I.2.1. Definition I.2.2. Kinds of achievement tests • Two kinds of achievement tests: • - Progress achievement tests • - Final achievement tests

40. Chapter I: Literature Review I.3. Testing listening comprehension skill I.3.1. The nature and sub-skills of listening comprehension I.3.2. Testing listening comprehension

41. Chapter I: Literature Review I.3. Testing listening comprehension skill I.3.1. The nature and sub-skills of listening comprehension I.3.2. Testing listening comprehension • Traditional approach • Alternative approach • McDonough & Shaw (1993): • Processing sounds • Processing meaning

42. Chapter I: Literature Review I.3. Testing listening comprehension skill I.3.1. The nature and sub-skills of listening comprehension I.3.2. Testing listening comprehension Kitao and Kitao (1996, p.1): • Understanding sentences and dialogues: Interpreting meaning, responding to utterances • Task using visual materials: Matching, T/F, Mapping, Drawing • Tasks involving talks and lectures: Summary-filling, Matching, T/F, Short answers, MCQs, Note-taking, etc.

43. Students’ responses Motivation to learn listening skill

44. Teachers’ responses Teaching methodology Frequency of teachers following teaching process and conducting the main task in a procedure

45. Teachers’ responses Teaching methodology Teachers’ reference to the end-term test to draw students’ attention

46. Students’ responses * low-scored group (LSG): five students (<5 points) * average-scored group (ASG): 24 students (5 to 6 points)* high-scored group (HSG): 19 students (7 to 10 points:18 students of 7-8 points and 1 student with 9-10 points)

47. Students’ responses Frequency of activitiesin listening lessons

48. Teachers’ responses Teaching methodology • Process of lesson • Reference to the end-term test

49. Teachers’ responses Students’ past test score • Teachers’ care about students’ past test score • Teachers’ preference of having item-analysis report of students’ past test scores

50. Teachers’ responses Students’ in-class performance • Interest in talks about test-taking techniques and test practice • High appreciation of test practice in-class • High appreciation of the importance of the end-term test