A Study of the Decision-making Behavior of Markers in E-C Sentence Translation Assessment

1. A Study of the Decision-making Behavior of Markers in E-C Sentence Translation Assessment Wen Hui Nie Jianzhong

2. Abstract Second language testing increasingly used rated tasks in place of objectively scored items. An understanding of the raters’ decision-making behavior is of particular importance for both reliability and validity. This paper is intended to investigate the decision-making behavior of experienced raters with similar background in their assessment of an English-Chinese sentence translation task on the basis of quantitative and qualitative analysis. rating scores and raters’ verbal protocol analysis indicated that raters in this study differed significantly in their decision making processes. Keywords: English-Chinese sentence translation; Think-aloud verbal protocol; Decision-making behavior; Inter-rater difference; Translation assessment (1-11)

3. (2-11)

4. Current researches on rater behavior (3-11)

5. ObjectiveTo explore decision-making behavior of experienced raters during the process of translation assessment.

6. Research Design and Methodology (5-11)

9. Results and discussion RQ1: How is the students’ E-C sentence translation assessed by the raters in terms of scores? Figure 4.1 a Frequency analysis on the category of accuracy Table 4.3 ANOVA table for the data of table 4.1 Figure 4.1 b Frequency analyses on the category of expression (8-11)

10. 最后评分 Decide final mark Tentative marking Good First Impression Bad Micro-assessment Macro-assessment Qualitative judgments 1.Task completion 2.Legibility 1.Locating errors 2.Judging gravities of errors Elements (9-11) RQ2 Are there any remarkable differences among raters in their qualitative judgments to the same translation?RQ3 What elements do these raters focus on while marking these two translation tasks and how do they focus on these elements? A model of the decision– making process in translation marking 1 In addition to the extremely good or bad performance, raters usually hold different perceptions to the same performance. 2. In E-C sentence translation assessment, rater showed great difference in deciding on the gravities of errors located.

11. E-C translation elements raters focused on in marking Sentence translation Discourse translation non-criterion elements Layout Spelling mistakes Rater’s affective responses Comparison with other scripts criterion-related elements Comprehension of the source text Mistranslation Omission Incorrect addition Obscurity of language Uncontrolled translation Lexical choice Sentence structure Pauses of sentences Coherence of sentences Literary grace Chinese conventions Fluency in expression Succinctness of language Stylistic inappropriateness Global communicative effect Task completion Error types Error gravity Error frequency non-criterion elements Layout Spelling mistakes Rater’s affective responses Comparison with other scripts criterion-related elements Comprehension of the source text Mistranslation Omission Incorrect addition Uncontrolled translation Lexical choice Chinese conventions Fluency in expression Stylistic inappropriateness Task completion Error types Error gravity Error frequency (10-11)

12. incomplete investigation of protocols

13. THANKS ！