臺東縣 「英語教師評量技巧養成工作坊」 培訓計畫

1. 臺東縣「英語教師評量技巧養成工作坊」培訓計畫臺東縣「英語教師評量技巧養成工作坊」培訓計畫 謝中能 銘傳大學 應用英語學系

2. 測驗之內涵 --測驗之理論與意涵

3. 測驗的定義 A test is a method of measuring a person’sability or knowledge in a given area.

4. 測驗的目的 • 1.Making inferences about test taker’s language ability or making predictions about their capacity for using language to perform further tasks in contexts outside the test itself • 2. Making decisions (selection, diagnostic, placement, progress) about test takers on the basis of their scores, which can inform the decision-makers about their ability or capacity for language use in non-test situation.

5. 測驗與學習的關係(washback effect) The impact of a test on teaching Positive: tests and examinations influence teaching and learning beneficially. Negative: The test fails to reflect the learning principles and/or the course objectives to which they are supposedly related.

6. learning/teaching  evaluation/testing • evaluation/testing  learning/teaching • (negative washback) • Results: frustrating teachers and students

7. 測驗的意義 • Give the students and teachers a chance to evaluate the effectiveness of their learning and teaching.

8. 測驗的要素 Practicality—within the means of financial limitations, time constraints, ease of administration, and scoring and interpretation

9. Reliability: consistency and dependability test reliability: The same test to the same subjects or matched subjects on two different occasions, the test should yield similar results. rater reliability: Different raters are inclined to give similar scores. inter-rater reliability intra-rater reliability

10. Validity: the degree to which the test actually measures what it is intended to measure. --Content validity: clearly defined the achievement --Construct validity: Does this test actually tap into the theoretical construct as it has been defined? “Proficiency” is a construct. “Communicative competence” is a construct.

11. 測驗的用途 Proficiency test: Measuring a student’s control of the language in the light of what they will be expected to do with it in the future performance of a particular task. Diagnostic and placement tests Achievement tests: related directly to classroom lesson or a total curriculum. Aptitude tests

12. 測驗注意事項 Don’t set traps to fail the test takers deliberately. There should be no difference between the course objective and test content. Teachers need to be able to make clear and useful interpretations of test data to understand their students better. Teachers are expected to return the test sheets in the next class.

13. 學習的層次 • Bloom’s taxonomy: Learning is more than just gaining knowledge • 1. knowledge: recall or recognize info • 2. comprehension: understand meaning • 3. application: use or apply knowledge • 4. analysis: interpret knowledge • 5. synthesis: develop new structures • 6. evaluation: assess concepts

14. Recognition v. s. production I’ve been standing here ___ half an hour. A. since B. during C. while D. for I’ve been standing here ___ half an hour. + (produce and use the correct form in real- life situation) - (There are more than one correct answer.) /nearly/almost/over

15. The communicative approach Reflect the culture of a particular country by focusing on context and the use of authentic material Qualitative modes is preferred Each student’s performance is evaluated according to his or her degree of success not in relation to other students.

16. Measuring a student’s control of the language in the light of what they will be expected to do with it in the future performance of a particular task. • (Does the student know enough English to follow a certain university course given in the medium of English?) • In no way related to any syllabus

17. Multiple-choice items High reliability Easy marking Rarely measure communication Never test the ability to communicate in TL Never evaluate actual performance Useful in measuring students’ ability to recognize correct grammatical forms

18. Components of a multiple-choice question Stem: a. an incomplete form He accuses me of __ lies. b. a question According to the writer, what did Tom do? Answer: one absolutely correct Distractors: to distract the majority of poor students .

19. Examples of multiple-choice questions The word “astronauts” is used in the passage to refer to A. travelers in an open liner B. travelers in a space-ship C. travelers in a submarine D. travelers in a balloon == The word “astronauts” is used in the passage to travelers A. an open liner B. a space-ship C. a submarine D. a balloon He began to choke while he was eating the fish. A. die B. cough and vomit C. grow very angry D. be unable to breathe because of something in the windpipe

20. Examples of multiple-choice questions You need a ___ to enter that military airfield. A. permutation B. perdition C. permit D. perspicuity Blackwell started his career as a lawyer in A. 1926 B. 1922 C. 1925 D. 1921

21. Guidelines in multiple-choice questions I. Test one thing at a time I never knew where _____. A. had the boys gone B. have the boys gone C. the boys have gone D. the boys had gone

22. Guidelines in multiple-choice questions II. Each option/distracter should be grammatically correct. Q. Someone who designs houses is a _____. A. designer B. builder C. architect D. plumber Q. The boy’s hobbies referred to in the first paragraph were ________. A. camping and fishing B. tennis and golf C. cycling long distances D. fishing, rowing and swimming

23. Guidelines in multiple-choice questions designing III. The stem should be as brief and clear as possible with no extraneous info or irrelevant clues IV. The stem must be long enough to indicate the basis on which the correct option should be selected. V. A grammatical test item should not contain other grammatical features as difficult as the area being tested, and a vocabulary item should not contain more difficult semantic features in the stem .

24. VI. Items are arranged in rough order of increasing difficulty. • VII. Distracters should not be too difficult nor demand a higher proficiency in the language than the correct option. • Q.You need a ___ to enter that military airfield. • A. permutation B. perdition C. permit • D. perspicuity • VIII. The correct option should appear in each position approximately the same number of times.

25. Tests of vocabulary should avoid difficult grammatical structures. Similarly, tests of grammar should contain only those lexical items which present no difficulty.

26. Tests of grammar & usage “Won’t I need a coat?” “Well, you know how …” A. warm is it today B. today it is warm C. is it warm today D. today is it warm

27. Rearrangement items Good for indirect questions 1. “Won’t I need a coat?” “Well, you know how …. A. it B. today C. warm D. is Good for inversion 2. Everyone’s forgotten…. A. cup B. he C. which D. used Not only …, but he took me to his house. A. me B. he C. did D. meet

28. Rearrangement items III. Good for the order of adjectives/the position of adverbs 1. The police are looking for ….. A. big B. two C. cars D. black 2. Mr. Walker made Ann …. A. her new pen B. to C. show D. me

29. Completion items Good for articles testing Can you see __ sun shining through the clouds? ___ sun is shining today. A. The B. A C. X Good for dialogue testing A: ______________. B: So do I. I generally watch it for an hour every day.

30. Transformation items I haven’t written to you for a long time. It’s a long time __________. In sunny weather I often go for a walk. When the weather ________.

31. Combination items You finish the paper. Then check your answers carefully. (After) Some questions may be difficult. They should be left until later. (which)

32. Testing vocabulary Definition A person who receives and pays out money in a bank A. broker B. accountant C. creditor D. cashier Synonym It’s rained continuously for two whole days. A. without stopping C. regularly B. heavily D. at odd moments

33. Guidelines of item writing 1. If the problem area is located in the options, the stem should be kept simple. 2. If the problem area is in the stem, the options should be simple. 3. Each option should belong to the same word class as the word in the stem. 4. The correct option and the distracters should be at approximately the same level of difficulty. If the correct option is more difficult,===elimination (known answers are not correct) Correct option is easier==too easy for good students to believe

34. 4. All the options should be approximately the same length. 5. Avoid using a pair of synonyms as distracters. The old man was always courteous when anyone spoke to him. A. polite B. glad C. kind D. pleased

35. Listening tests I. phoneme discrimination test A. picture identification (link1) B. Choose the written word which corresponds to the spoken word. spoken: den written: A. Ben B. den C. pen D. ten Good for diagnostic test Cover a wide range of sounds

36. II. Stress and intonation Spoken: I’ve just given THREE books to Bill. Written: I’ve just given THREE books to Bill. ( ) ( ) ( )( ) ( X ) ( ) ( ) ( ) Spoken: Tom’s a fine goalkeeper. Written: The speaker is A. making a straightforward statement B. being very sarcastic C. asking a question

37. III. Statements and dialogues Spoken: I wish you’d done it when I told you. Written: A. I told you and you did it them. B. I didn’t tell you but you did it then. C. It told you but you didn’t do it then. D. I didn’t tell you and you didn’t do it then. (to understand both the grammatical and lexical features)

38. IV. Short response Spoken: Why are you going home? Written: A. At six o’clock. B. Yes, I am. C. To help my mother. D. By bus.

39. V. comprehension through visual materials (link)

40. Oral production tests It is possible for people to produce all the correct sounds but still be unable to communicate effectively and appropriately. Because oral communication is generally rated so highly in language learning, the testing of oral production forms an important part of many language testing programs.

41. Oral production tests I. Reading aloud II. Conversational exchanges A friend of yours has forgotten where he has put his glasses. He cannot see too well without them. What will you say to him? (A. Let me help you to look for them.)

42. Oral production tests III. Using pictures for assessing oral production A. The students are given a picture to study for a few minutes; they are then required to describe the picture in a given time. (advertisements, posters & strip cartoons) B. A pair of students are given almost identical pictures to discuss the differences.

43. Oral production tests IV. The oral interview --a realistic means of assessing in a natural speech situation --not natural but in exam situations --The interviewer should never attempt to note down marks or comments while the student is still engaged in the interview. Pair interview—ensure a pair have similar personalities/levels of language ability

44. Oral production tests V. the short talk Students are allowed several days or only a few minutes to prepare a short talk on a given topic. Students are asked based on the talk either by the examiner or by the audience.

45. Oral production tests VI. Group discussion and role playing One student acts the part of a police officer, another a bus conductor, a third a bus driver, and a fourth a passenger hurrying to visit a sick friend. The passenger hurries to get on the bus and jumps on as it is moving off. The conductor stops the bus and tells him that the bus is full and that he must get off. The passenger can see an empty seat and he begins to argue. The bus is now in the middle of the road and is a danger to other traffic. Act the roles given.

46. Testing reading comprehension I. matching tests A. word matching: assist in developing word recognition speed now bow/not/how/now/mow ever never/over/ever/fewer/even B. sentence matching: Tom is not going to your school. a. Tom is not going to your pool. b. Tom is not coming to your school. c. Tom is going to your school.

47. II. Paraphrasing John is not as tall as Sally, but he’s a little taller than Rick. A. Sally is taller than John and Rick. B. John is not as tall as Rick. C. Sally is taller than John but not as tall as Rick. D. Rick is taller than John and Sally.

48. III. Reading comprehension A. a very short reading extract with one question The president was talking to a young woman in the crowd when Tim suddenly caught sight of a man standing several yards behind her. The man had something in his hand: it was a short stick. Q. What made Tim notice the man in the crowd? A. A. He was very close to Tim. B. The president was talking to him. C. He was standing in front of the woman. D. He was carrying a stick.

49. Guidelines In a proficiency test, the text should contain the type of reading task which will be demanded of the test takers in later real-life situations. If it’s a progress or achievement test, the reading passage should be similar to the type of reading material with which the students have been confronted in their work at school.

50. II. Pictures and sentence matching