Test development

1. Test development Ginny Price CETL

2. Writing multiple choice items • Write only a few items at a time • Immediately after preparing class lesson or after class discussions • Focus on important points • Spread out work time increasing depth of questions • Build a bank of questions on Angel • Include a few new items each time test is given • Use item analysis to determine worth of test items

3. Anatomy of item • Stem • Responses • Correct response • Three or four plausible distractors

4. stem • Students can read and understand stem • Students can formulate an answer without reading responses • Can be written as an incomplete sentence or a question • Focus on important concept or principle • Trivial details or obscure facts should not be used

5. distractors • Use four responses unless doing so would require using distractors that are implausible • Use student misconceptions and common errors to write plausible distractors • Give a short answer quiz to obtain sample of students’ incorrect responses to use as distractors

6. guidelines • Stem presents problem clearly and concisely • Include a verb in the statement • Contains all information necessary to understand problem • Responses should not contain repetitive words or phrases. • Include these items in the stem • Write correct response first then distractors • Best answer (reduces disagreements) • Agreed upon answer by authorities in field • Correct answer • Facts: anatomy, terminology

7. Guidelines 2 • Avoid giving clues to answers • Measure test taking skills not learning • Reduces test’s validity • Types of clues • Length of responses: correct answer is either longer or shorter than distractors • Verbal association: using similar words in stem and correct response • Grammatical clue: correct answer is only grammatically correct one • Specific determiners: always, never, all, none • Instead use: usually, sometimes, typically, maybe

8. Guidelines 3 • Use positive statements as negative statements confuse students • If negative statements are used the negative word should be bold and in caps • Refrain from using all of the above • too easy • often the correct response • Refrain from using none of the above options • Do not use with negative stem; creates confusing double negative • Does not reinforce correct answer as correct answer is not present • Use only with correct answer format instead of best answer (absolutes) • Can be used to increase item difficulty

9. Guidelines 4 • Each item is independent • No clues in one item for other items • No item dependent on previous item’s response • For responses: use a logical order or randomize • Avoids patterns • Response “A” rarely correct answer; “C” most often correct answer • Cite authority in stem if item controversial • Avoid quoting source verbatim in item • Encourages memorization instead of understanding • Arrange responses in vertical column

10. Concentrations • Purpose of test • Reading material • Understanding material • Synthesizing material • Applying material • Analyzing material • Can adjust weights of items based on purpose • Important concepts and principles in course • Develop novel or thought provoking situations in items • Use students’ misconceptions as distractors

11. techniques • Premise → consequences • Premise + premise → consequence • Analogy • Classification • Proposition 1 & proposition 2 → 1 correct but not 2, 2 correct but not 1, both correct, both incorrect • Context dependent item set

12. Premise → consequences • p → c • A premise (stem) • Followed by consequences (responses) • Evaluate • Example: • What could happen if a liquid is instilled into an animal’s external ear canal without determining the patency of the animal’s tympanic membrane? • Damage to the animal’s hearing • What must be examined before instilling substances into an animal’s external ear canal? • Patency of the tympanic membrane

13. Premise + premise → consequence • p + p → c • Premise + premise (stem) • Followed by a consequence (responses) • Evaluate • Example: • If an animal presented with a body temperature of 104.0°F and they walked to the hospital with their owner on a 90°F day it can be concluded the hyperthermia was caused by • Environmental factors • Stress • Infection • Shock

14. analogy • a : b = c : d • a is to b as c is to d • Precise relationships between two sets of words • Analysis • Example: • Operant conditioning is to voluntary as classical conditioning is to • Involuntary • Association • Habituation • Salivating

15. Classification • Classifies terms, names and statements • Comprehension • Example • Pavlov would be classified as a • Behaviorist • Physiologist • Biologist • Sociologist

16. p1 p2 → 1 correct but not 2, 2 correct but not 1, both correct, both incorrect • Two propositions are presented in the stem. It must be decided whether both are true, neither are true, A but not B is true or B but not A is true. • Example: • A fearful dog with passive defense reflexes comes into the hospital for an exam. This dog will exhibit (A) appeasement body language (B) agonistic body language. • A. A but not B • B. B but not A • C. neither A nor B • D. both A and B

17. Context dependent item set • A stimulus is used to generate a set of questions • Written scenario, graph, chart, map, picture, table, article • Measures high level cognitive skills: comprehension, analysis and application • Use a novel stimulus to gain an idea of student’s quality of learning • Example on next slide

18. Content dependent item The purpose of the porous tape in this picture is to • prevent the catheter from sliding out of the bladder • allow sutures to be placed attaching the tape to the skin • prevent the animal from chewing out the catheter • allow the technician to better monitor catheter length when inserting it

19. Resources • Jacobs. L.C. & Chase, C.I. (1992). Developing and using tests effectively: A guide for faculty.