1 / 25

Evaluation

This article explains what a test is, its purpose, different types of tests, and how to create a good test. It also discusses the differences between direct and indirect testing and the importance of task-based assessment.

tedl
Download Presentation

Evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluation Peter Strutt Caceres, November 2004

  2. What is a test ? • A test is an activity of gathering information to be used in making educational decisions. • A test is a sample of behaviour, products, or performances from a larger domain of interest. • A test is a task or set of tasks that elicits observable behaviour from the test taker.

  3. What is a test? • A test is a form of measurement. Tests yield scores that represent attributes or characteristics of individuals. • A test measures a particular body of knowledge, skills, abilities, or performances which are of interest to the test user. • A test is a means of evaluating a course of language instruction

  4. What are tests for? • Basically, to make decisions. For example: • Should Cristina be admitted to our university? • Should Miguel be allowed into this section? • What class level should Juan be placed in? • Should Dolores pass or fail the course? • Should we certify Carmen as an air hostess? • Should we pass Sancho Panza onto the second round of interviews?

  5. Types of test • Diagnostic tests are projections used to pinpoint problem areas and to determine if remedial action is required. These may be by self-assessment.

  6. Types of test • Placement tests are used to determine at which level of instruction an individual should start his or her training.

  7. Types of test • Formative tests are administered by classroom teachers throughout the training process to determine mastery of a specific course content and learning objectives.

  8. Types of test • Achievement or Summative tests are closely linked to the course material used in class and test items are written around a sampling of course content.

  9. Types of test • Proficiency tests are designed to assess an individual's overall ability. These tests are not linked to any specific course syllabus or learning objectives. What is measured is the extent and adequacy of an individual's linguistic skills in the broader sense.

  10. Types of test • Psychometric tests attempt to discover a person's capacity, propensity or liability to act, react, experience, or to structure thought or behaviour in particular ways.

  11. What are communicative language tests? What is communicative competence? List your ideas with a colleague.

  12. Communicative language tests are : • Performance-based • Contextualised • Real-world (outcomes) • Measures of production rather than receptive skills

  13. But... • only a small sample of the testee’s language can be collected, and however realistic the tasks may be intended to be, the testee’s performance inevitably reflects the fact that s/he was performing under test conditions.

  14. What makes a good test? • validity i.e. individuals should have an opportunity to demonstrate the skill being assessed (it would be unreasonable to expect a recital of the Highway Code as evidence of an ability to drive a car)

  15. What makes a good test? • face validity i.e. does the test look right? • construct validity i.e. the underlying abilities that the test is designed to measure

  16. What makes a good test? • content validity i.e. a representative sample of the relevant language skills and knowledge • concurrent validity i.e. the extent to which a test yields the same results as other measures of the same skills. • predictive validity i.e. how well the test forecasts future performance

  17. What makes a good test? • Reliablility • inter-rater reliability i.e. making sure that assessors use the same criteria • test-retest reliability i.e. give the students the same test twice to the same group of students

  18. What makes a good test? • split-half reliability i.e. compare the scores of two halves of a test given to the same individual • internal consistency i.e. if various test tasks measure the same ability then a total score of many such tasks should be free from random error

  19. Discrete-point testing 1. Phoneme recognition. 2. True/ False answers. 3. Spelling. 4. Word completion. 5. Grammar items. 6. Multiple choice tests. Integrative testing 1. Cloze tests 2. Dictation 3. Translation 4. Essays and task-based writing tasks 5. Oral interviews and conversation 6. Reading, or other extended samples of real text Testing the 4 skills

  20. Direct vs indirect? • Direct tests are task oriented rather than test oriented, they require the ability to use language in real situations, and they therefore should have a good washback effect. • BUT

  21. Direct vs indirect? • they do call for greater skill and judgement on the part of the test designer and assessors

  22. Direct vs indirect?

  23. Direct vs indirect?

  24. Task-based assessment • Task design • Construct-centred approach : identify the features of general language proficiency that are to be elicited by the task • Work-sample approach : identify the tasks the testee will have to perform in the real world and their interactional properties

  25. Issues • How does the choice of task affect the way the testee performs? • Familiarity of content • Dialogue vs monologue • Complexity • Interactive vs non-interactive • Code complexity & cognitive complexity • What assessment criteria will be used?

More Related