口语考官培训

1. 全国国际商务英语考试（二级） 口语考官培训 2011.07.19

2. 1 4 考试模拟测试视频 2 3 考核方式与内容 评分标准开发 6 5 考试流程视频介绍 评分标准解读 口语测试相关概念 目 录

3. 口语测试相关概念 • Rater (alsojudge, marker, scorer, assessors) • The judge or observer who operates a rating scale in the measurement of oral and written proficiency. The reliability of raters depends in part on the quality of their training, the purpose of which is to ensure a high degree of comparability, both inter- and intra-rater. Since raters are human and are therefore subject to individual biases, close attention is paid not only to reliability, but also to analyses of rater bias. (Davies 1999) • A person who assigns a score or rating to a test taker’s oral or written performance on the basis of a set of rating criteria. (Richards 2005)

4. 口语测试相关概念 • Rater training • The preparation of raters for their task of judging performances. The training often takes the form of a workshop in which raters are introduced to the test format and test tasks and the rating criteria, the exemplar performances at each defined level of performance are presented and discussed. Raters are then asked to evaluate a series of performances and to compare their ratings, discussing the grounds for any differences between them. (Davies 1999)

5. Fair Language proficiency Empathetic Sensitive to language 口语测试相关概念 Rater qualities Flexible

6. 口语测试相关概念 • Interlocutor • In language testing the term is used to refer to the interviewer or facilitator of communication in an oral interview. The interlocutor may also be the assessor. (Davies 1999) • A neutral term referring to any person with whom someone is speaking. In language testing, the term is sometimes used to refer to a teacher or other trained person who acts during a test as the person with whom the student or candidate interacts in order to complete a speaking task. (Richards 2005)

7. 口语测试相关概念 • Interlocutor effect (对话人效应， 对话人影响) • The effect of interlocutor behavior in an oral interview test. Variability in interlocutor behavior is a potential source of measurement error.

8. 口语测试相关概念 • Interlocutor effect • Below are examples of different aspects of interlocutor input which may have an effect either on the language behavior of the candidate or on the way in which this behavior is assessed by raters. • The complexity/formality of the language produced by the interlocutor • The number of turns initiated or allocated by him/her • The degree of empathy established with the candidate • The extent to which the interlocutor adheres to the script/guidelines provided.

9. 口语测试相关概念 • Marker Reliability (Weir 2005:34) （评分人信度） • To what extent the markers are • in overall agreement; • ranking a group of students in the same order; • rating individuals at the same level of severity; • consistent in their own judgments during the whole marking process.

10. 口语测试相关概念 • Marker Reliability • Markers need to be consistent in two ways: • each marker needs to be consistent within himself (intra-rater reliability), i.e., given a particular quality of performance, he needs to award the same mark whenever this quality appears. • there needs to be consistency of marking between markers (inter-rater reliability), i.e., one marker will award the same mark as another when confronted with a performance of the same quality.

11. 口语测试相关概念 • Intra-rater reliability (or internal consistency)（评分人内部信度） • means raters agree with themselves, over a period of a few days, about the ratings that they give. (Luoma 2004) • The degree to which an examiner or judge making subjective ratings of ability gives the same evaluation of that ability when he or she makes an evaluation on two or more different occasions. (Richards 2005)

12. 口语测试相关概念 • Inter-rater reliability （评分人间信度） • The degree to which different examiners or judges making different subjective ratings of ability agree in their evaluations of that ability. (Richards 2005) • The level of consensus between two or more independent raters in their judgments of candidates’ performance. (Davies 1999)

13. 口语测试相关概念 • Intra-rater reliability • The extent to which a particular rater is consistent in using a proficiency scale. The consistency of a particular rater can be affected by such factors as • The nature of the scale (inadequately defined scales may cause rater uncertainty which leads to random assignment of scores); • The order in which candidates/test papers are assessed (raters may be normative in their behavior and each new performance may therefore be assessed in relation to the previous one rather than according to the standards defined by the scale); • The ‘stability’ of the rater (raters’ interpretation of the scale may change either because they are tired or distracted after successive marking of a number of test performances or because considerable time has elapsed since their training session or since the previous test administration). (Davies 1999)

14. 口语测试相关概念 • Halo effect（光环效应，成见效应） • The distorting influence of early impressions on subsequent judgments of a subject’s attributes or performance, or the tendency of a rater to let an overall judgment of the person influence judgments on more specific attributes. (Davies 1999)

15. 口语测试相关概念 • Holistic scoring（综合性评分） • (Also global assessment全局性评价, global scoring全局性评分, impressionistic assessment印象性评价, holistic assessment整体性评价) • A method of scoring where a single score is assigned to writing or speaking samples on the basis of an overall impressionistic assessment of the test taker’s performance on a writing or speaking task as a whole. (Richards et al. 2005)

16. 口语测试相关概念 • Holistic scoring（综合性评分） • A type of marking procedures which is common in communicative language testing whereby raters judge a stretch of discourse impressionistically according to its overall properties rather than providing separate scores for particular features of the language produced. (Davies 1999)

17. 口语测试相关概念 • Analytic Scoring （分项评分，分析性评分） • A method of scoring that separates and weights different features of the test taker’s performance on a writing or speaking task and assigns separate scores to each feature. (Richards et al. 2005) • A method of subjective scoring often used in the assessment of speaking and writing skills, where a separate score is awarded for each of a number of features of a task, as opposed to one global score.

18. 口语测试相关概念 • Analytic Scoring • Advantages: • Raters are required to focus on each of the nominated aspects of performance individually, thus ensuring that they are all addressing the same features of the performance. • It allows for more exact diagnostic reporting of literacy or oracy development especially where skills may be developing at different rates. • It leads to greater reliability as each candidate is awarded a number of scores.

19. 口语测试相关概念 • Analytic Scoring • Problems: • The focus on specified aspects of the performance may divert raters’ attention from its overall effect. • The possibility of a halo effect distorting the score due to the number of judgments required. • Time consuming (Davies 1999)

20. 口语测试相关概念 • Accuracy • Pronunciation must be clearly intelligible even if some influences from L1 remain. • Grammatical/lexical accuracy is high though grammatical errors which do not impede communication are acceptable.

21. 口语测试相关概念 • Fluency and Coherence • This refers to the ability to talk with normal levels of continuity, rate and effort and to convey information and to express or justify opinions in coherent, connected speech.

22. 口语测试相关概念 • Approppriacy • The use of language must be generally appropriate to function and to context. • The intention of the speaker must be clear and unambiguous.

23. 口语测试相关概念 • Interactivity • This refers to the ability to take an active part in the development of the discourse, showing sensitivity to turn taking and without undue hesitation.

24. 口语测试相关概念 • Range • A wide range of language must be available to the candidate. Any specific items which cause difficulties can be smoothly substituted or avoided.

25. 口语测试相关概念 • Size • Must be capable of making lengthy and complex contributions where appropriate. • Should be able to expand and develop ideas with minimal help from the interlocutor.

26. 口语测试相关概念 • 口语考官分工 • Assessor A: • Interlocutor • Rater: holistic scoring • Assessor B: Rater: analytic scoring

27. 考核方式与内容

28. 评分标准开发-理论依据

29. BULATS PETS IELTS OPI CET 评分标准开发-实践依据 全国国际商务英语（二级）口试 BEC

30. 评分标准开发-实践依据

31. 评分标准开发-实践依据

32. 评分标准开发-实践依据

33. 评分标准开发-实践依据

34. 评分标准开发-实践依据

35. 评分标准解读 口语考试成绩 （总分50分） 分项评分（满分 25分） 总体评分 （满分 25分） 口试总分为50分，成绩30分及以上为合格

36. 评分标准解读 准确性解读： 词汇 ：基础词汇 一般性商务词汇——discount, sales, business relationship 专业性商务词汇——FOB, insurance, packing, shipment 语法 ： 词性、词序、句式、时态 语音 ： 单词的发音、重读、韵律和语调

37. 分项评分 准确性

38. 评分标准解读 连贯性解读： • 考生能连续地输出意义一致的话语单位；整段话语各个不同命题可以被纳入一个意义框架，即整段话语围绕一个主题展开。

39. 分项评分 连贯性

40. 评分标准解读 得体性解读： 语式 语场 语旨 语域

41. 分项评分 得体性

42. 评分标准解读 互动性解读： 表达 倾听 反馈 互动性

43. 分项评分 互动性

44. 分项评分 商务知识

45. 总体评分

46. 总体评分

47. 总体评分

48. 总体评分

49. 总体评分

50. 口试评分卡