Limitations and Future Directions of Tests

1. Limitations and Future Directions of Tests

2. Test Interpretation and Use • A valid test involves valid interpretation and valid use of the test scores • E.g. TOEFL – what was it constructed for? • Making high-stakes admission decisions to North American universities • Can you accurately interpret a score to reflect what it claims it wants to measure? • Will you be able to use the test scores for the purpose the test was constructed? • This issue refers to Consequential Validity

3. Limitation of tests (Bachman) • Subjectivity • In setting of questions as well as grading • Underspecification of domain • Not everything taught or about the subject can be included in a test • Incompleteness (from students’ perspective) • Not everything the student knows can be demonstrated • Indirectness which involves all tests • Imprecision in the grading and the points alloted for each item/question

4. Attaining Acceptable Standards (Linn, Baker, Dunbar, 1991) • Consequences • Does the test have positive consequences? • Fairness • Is the ethnic and cultural background of the student taken into consideration? • Transfer and generalisability • Does the assessment support accurate generalisations about student ability? • Cognitive and linguistic complexity • Does the test require the student to use complex thinking, problem solving and an appropriate level of linguistic complexity?

5. Attaining Acceptable Standards (Linn, Baker, Dunbar, 1991) • Content quality • Is the selected content representative of the current understanding of the construct? • Content coverage • Are the key elements of the curriculum covered? • Meaningfulness • Do the students feel that the tasks are realistic and worthwhile? • Cost and efficiency • Is the information students receive through the test worth the time and money to obtain?

6. Future Directions in Tests and Evaluation

7. Professionalism of teachers • “Informed” assessment by teachers • Need knowledgeable teachers who “systematically observe and selectively document their students’ performance through multiple methods, across diverse contexts, and over time as students participate in meaningful learning activities” (Wolf, 1993: 519)

8. Integration of communicative elements • Communicative – authentic • Authenticity is in implementation NOT design • Communicative tests involve performance, are authentic and are scored on real-life outcomes (Fuller, 2000)

9. Computer based assessments • E.g. TOEFL – TOEFL CBT; TOEFL iBT • Computer adaptive tests • Ease of scoring and delivery • Changes on the part of the test taker? • Validity of computer based tests?

10. Specialised examinations • Literature, ESP, EST • Contextualised – for specific purposes • Often could be more relevant

11. Enhancement of the testing-teaching relationship • Social obligation from tests • Consequential validity – i.e. are the uses of the test score valid? • Tests can be used to drive teaching and learning reform

12. Emphasising the student • Humanistic and constructivist approaches • Independent and autonomous • Relevance of student backgrounds • Feedback and score reporting