Testing Writing

1. Testing Writing Rio Darmasetiawan 69100077

2. Testing problem • We have to set writing tasks that are properly representative of the population of tasks • The tasks shoudl elicit valid samples of writing • It is essential that the samples of writing can and will be scored validly and reliably

3. Representative tasks • Specify all possible content In order to judge whether the tasks we set are representative of the tasks that we expect students to be able to perform, we have to be clear at the outset just what these tasks are that they should be able to perform.

4. Representative tasks • Include a representative sample of the specified content The more tasks (within reason) that we set, the more representativeof a candidate's ability (the more valid)will be the totality of the samples (of the candidate's ability) we obtain.

5. Elicit a valid sample of writing ability • Set as many separate tasks as is feasible We have to offer candidates as many 'fresh starts' as possible, and each task can represent a fresh start. By doing this, we will achieve greater reliability and so greater validity

6. Elicit a valid sample of writing ability • Test only writing abilityand nothing else This advice assumes that we do not want to test anything other than the ability to write. Therefore, for the sake of validity, we should notset tasks which measure other abilitiessuch as creative, imaginative, etc.

7. Elicit a valid sample of writing ability • Restrict candidates Writing tasks should be well defined:candidates should know what is required of them, and they should not be allowed to go toofar astray. One last thing to say about tasks is that they should not only fit wellwith the specifications, but they should also be made as authentic as possible

8. Ensure valid and reliable scoring • Set tasks which can be reliably scored • Set as many tasks as possible The more scores foreach candidate, the more reliable should be the total score. • Restrict candidates The greater the restrictions imposed on the candidates, the more directly comparable will be the performances of different candidates • Give no choice of tasks Making the candidates perform all tasks also makes comparisons between candidates easier • Ensure long enough samples The samples of writing that are elicited have to be long enough for judgments to be made reliably

9. Ensure valid and reliable scoring • Create appropriate scales for scoring • Holistic scoring Holistic scoring involves the assignment of a single score to a piece of writing on the basis of an overall impression of it. This kind of scoring has the advantage of being rapid. • Analytic scoring Methods of scoring which require a separate score for each of a number of aspects of a task are said to be analytic.

11. Ensure valid and reliable scoring • Calibrate the scale to be used Any scales which is to be used should first be calibrated, meaning collecting samples of performance collected under test conditions, and covering the full range of the scales. • Select and train scorers They should be sensitive to language, have had experience of teaching writing and marking written work. It is also helpful if they have had training in testing.

12. Ensure valid and reliable scoring • Follow acceptable scoring procedures Each task of each student should be scored independently by two or more scorers (as many scorers as possible should be involved in the assessment of each student's work. Seniormember of the team should collate scores and identify discrepancies in scores awarded to the same piece of writing. Nevertheless, once scoring is completed, It is useful to carry out simple statistical analyses to discover if anyone's scoring is unacceptably aberrant