How well did the assessment task do what we wanted it to do?

1. How well did the assessment task do what we wanted it to do? Janina Drazek Manager — Assessment & Comparability, QCAR Queensland Studies Authority

2. Central role of the teacher Skilled diagnosticians, i.e. teachers diagnose and assess against declared standards.

3. Diagnosticians require reliable instruments designed to produce valid information. ASSESSMENT is the teacher’s instrument.

4. Assessment is the purposeful, systematic and ongoing collection of information as EVIDENCE for use in making judgments about student learning. (Education Queensland 2001, Years 1-10 Curriculum framework for Education Queensland schools, Department of Education, Brisbane, p.13) What is assessment?

6. To develop valid and reliable tasks three reflective steps are required: • Evaluation • Modification • Trialling

7. When and how do I EVALUATE? ALL the time. In particular, • once a task has been created do the task yourself. • once you trial the task with learners—give them the possibility of commenting on the task. Using all the information you have gained modify the task or reject entirely.

8. Q.BUT I did all that, now I want to know how do I evaluate whether the task worked? A. Evaluate by applying exactly the same principles used for the development of quality assessments.

9. Sound assessment design Selecting a proper method to: • suit the purpose and the target • elicit the right performance or product • provide enough evidence to support the decision • avoids sources of bias.

10. Evaluating • Did they enjoy the task? • Was it easy/difficult? • Did they understand what to do? • How well did they think they responded to the task? • Ask focused questions about how they did the various sections of the task. Ask the students

11. Evaluating Look at the evidence (student work) • Mark the responses according to the criteria and standards matrix (schema, rubric etc) • Note the range of standards in the responses (consider and analyse) • Do the responses reflect your indicative response? If not, why?

12. Evaluating Review the task against the principles of quality assessment • Credibility–valid and reliable assess what it purports to assess • Intellectual rigor focus on higher order thinking • Authenticity have purpose • User friendliness feasible, appropriate and engaging fair and equitable

14. Evaluating • Did they reflect what is valued in the assessment? • How were they organised? • Were they organised under: Knowing and understanding; Inquiring; Responding; Reflecting (or some other constructs)? • Look at the number of criteria – too many or too few? • Are they sufficiently different from, and independent of, each other? • Did the students understand what was expected? Criteria

15. Evaluating Standards Descriptors • cover a range of performances on a particular criterion • are sufficient in number to enable differentiation of performances • can be seen in student work • are written in positive terms • are written in language that is suitable for students

20. Alignment of Assessment & Curriculum • Intent (What do we want students to learn?) • Enactment (What are the appropriate learning experiences?) • Evidence (What are the contexts in which students demonstrate their learning?) • Judgment (What is being valued?) • Moderation (Is there consistency in the way we apply standards?)

21. Effective assessment design • depends upon a clear and complete explication of the claims one wishes to make on the basis of assessment, the evidence needed to support those claims, the student behaviour that will comprise that evidence, and the tasks needed to elicit those behaviours. That chain of reasoning, once established and documented becomes part of the validity argument in support of the assessment (Matters, 2006:14).

28. QCAR contacts • Website: www.qsa.qld.edu.au/qcar • Email: QCARadmin@qsa.qld.edu.au