Multiple-Choice Items

1. Multiple-Choice Items Glen Hammond, RRC, CAE, CIE

2. Multiple Choice Items 5:50 • Are selection/recognition/choice items • are composed of a question or an incomplete statement [STEM] and a list of alternative responses, [OPTIONS]; Incorrect responses are DISTRACTERS or FOILS. The correct answer is the KEY. Multiple-Choice Items

3. Example 1 5:50 • Which of the following is a selection-type item? • multiple choice • extended essay • completion • restricted essay Multiple-Choice Items

4. Example 2 • The highest level of Bloom’s Cognitive Taxonomy listed here is • synthesis. • knowledge. • application. • analysis. Multiple-Choice Items

5. M.-C. Parts & Functions p. 50 • Stem – set task or problem to be performed • Alternatives – suggest responses • Distracters – appearing plausible to uninformed • Key – correct answer – should be apparent to those who know course well Multiple-Choice Items

6. Directions 5:51 • For each of the following stems, select the correct or best answer from among the choices presented. Place the letter of your choice in the blank in front of the question [circle the letter, underline the answer, mark the corresponding letter or space on your answer sheet]. Multiple-Choice Items

7. Considerations 5:51 • They limit testing to what a student should do rather than what one might do if asked to supply own response. • They cannot generally measure problem-solving skills. We get the answers, but not the process the student has gone through to arrive at the answer. Multiple-Choice Items

8. Considerations 5:51 • They cannot test organization and presentation of ideas. Use essay for those items • They cannot test synthesis as cannot any other recognition format, as synthesis is an individual or group creation. • They are difficult to construct and make believable. Multiple-Choice Items

9. M-C may be used to: 5:51 • measure many different learning outcomes. • present a list of similar choices for alternatives. • ensure one idea only is being tested. • reducing chances of successfully guessing the correct answer: .25 as opposed to .5. • reduce the incidence of patterns of answers as might happen in T-F. Multiple-Choice Items

10. Use M-C to test: 5:51 • knowledge of facts, methods, terminology, principles, procedures, systems, frameworks • interpretation or comprehension of information • application of information or generalizations • analysis or evaluation [limited degree] Multiple-Choice Items

11. M-C Construction Rules 5:51-2 • Test important learning outcomes. • Present a single clearly-formulated task in the stem of the question. • State the stem in clear, simple language. • Put as much of the wording as possible in the stem. • State the stem in the positive form, whenever possible. Multiple-Choice Items

12. M-C Construction Rules • Emphasize NEGATIVE PHRASING whenever it is used in the stem. Preferred form is: All of the following are synonyms for curriculum EXCEPT ---. • Ensure that the intended answer is correct or clearly the best answer. • Make all alternatives grammatically consistent with the stem of the item and parallel in form. Multiple-Choice Items

13. M. C. Construction Rules • Avoid verbal clues that might enable students to select the correct answer or to eliminate an incorrect alternative. • similarity of wording in both the stem and correct answer. • stating the correct answer in textbook language or stereotyped phraseology. • including absolute terms in the distracters: never, always, totally, without exception. Multiple-Choice Items

14. M. C. Construction Rules • including two responses that are all-inclusive: all of the above or none of the above. • including two responses that have the same meaning. Multiple-Choice Items

15. M. C. Construction Rules • Make the distracters plausible and attractive to the uninformed: • Use common misconceptions as distracters. • State the alternatives in the language of the student. • Use “good-sounding” words [accurate, important, key, etc.] in the distracters as well as in the correct answer. • Make the distracters similar to the correct answer both in length & in complexity of wording. Multiple-Choice Items

16. M. C. Construction Rules • Use extraneous clues in the distracters, such as stereotypical phrasing, scientific sounding answers, and verbal associations with the stem. • Make the alternatives homogeneous, but in doing so beware of fine discriminations that are educationally significant. Multiple-Choice Items

17. M. C. Construction Rules • Vary the relative length of the correct answer to eliminate length as a clue. • Avoid using the alternative “all of the above” and use “none of the above” with extreme caution. • Control the difficult of the item problem in the stem, changing the alternatives, or increasing the homogeneity of the options. Multiple-Choice Items

18. M. C. Construction Rules • Vary the position of the correct answer in random order. You may do this by opening a book anywhere and using the numbers on the right as a guide: 1 = A; 3 = B; 5 = C; 7 = D; 9 = E. • Ensure each item is independent of the other items in the test. Multiple-Choice Items

19. M. C. Construction Rules • Use an efficient item format: alternatives listed below each other; letters rather than numbers for indicating different alternatives; if stem is a phrase, begin alternatives with lower-case letters. Multiple-Choice Items

20. Uses – Observing Directly • discriminate and make correct choices • comprehend concepts, principles, and generalizations • make judgments about choices • infer & reason • compute • interpret new information • apply information & knowledge Multiple-Choice Items

21. Only Minimally Useful for Inferring: • recall with minimum prompts • producing & expressing unique ideas • articulating explanations and giving examples • organizing personal thoughts • displaying thought processes or patterns of reasoning Multiple-Choice Items

22. Advantages of • versatility in assessing a variety of instructional objectives • reduction of opportunities for “bluffing” or “dressing up the answer” • focus on reading and thinking and not on writing under examination conditions • reduction of chances for obtaining correct answer by blind guessing Multiple-Choice Items

23. Advantages of • utility in obtaining “diagnostic information” when distracters are based on common student errors and misconceptions. Multiple-Choice Items

24. Criticisms of • no opportunity to express own thoughts • items may focus on trivia • penalize those who see “outside the box” • reduction but not elimination of guessing • belief that they lead to “standardized” conceptions -> oversimplification of ideas Multiple-Choice Items

25. Not Recommended for: • problems or questions requiring a single-word or single-number answer • computational problems in general • questions with only two plausible answers • items for which writing the answer doesn’t take any longer than marking an answer to a multiple-choice item. Multiple-Choice Items

26. Criticize this item: • 50. multiple choice • An undetailed example is a. always used singly, b. ineffective, c. used to generate a sharing experience with the audience, d. usually lengthy. Multiple-Choice Items

27. Key • unnecessary to label each item as to type • incomplete stem: no clear task or problem • avoid tandem [side-by-side] arrangement; use parallel [one below other] format • watch exact phrasing in textbook “a sharing experience” capitalized on clang expressions and rote memory rather than understanding and comprehension Multiple-Choice Items

28. Key • verbal clues: “undetailed” can be “lengthy” • correct answer c is longer than the rest of the items, thus providing a clue • specific determiner in a make that item easy to eliminate Multiple-Choice Items

29. Key • heterogeneous not homogeneous options • how examples are used • judgment about desirability • function of examples • characteristics of an example Multiple-Choice Items

30. Word Distracters So That They: • permit those with the minimum desired degree of knowledge to select the correct answer • do not permit those who lack the prerequisite knowledge to select the correct alternative on the basis of testwiseness or superficial clues. Multiple-Choice Items

31. Making Distracters Plausible • Use the students’ common errors. • Use important-sounding words [significant, accurate] that are relevant to the item stem. • Use words that have verbal associations with the item stem [political, politician]. • Use textbook language or other phraseology that has “the appearance of truth.” Multiple-Choice Items

32. Making Distracters Plausible • Use incorrect answers that are likely to result from student misunderstanding or carelessness [forgets to convert from one unit to another]. • Use distracters that are homogeneous and similar in content to the correct answer [e.g., all distracters are capital cities, prime ministers, etc.] Multiple-Choice Items

33. Making Distracters Plausible • Use distracters that are parallel in form and grammatically consistent with the item’s stem. • Make the distracters similar to the correct answer in length, vocabulary, sentence structure, and complexity of thought. Multiple-Choice Items

34. Key Understanding • Distracters should distract the uninformed, but they should not result in “trick questions” that mislead knowledgeable students [Avoid inserting NOT in a correct answer to make it a distracter!]. Multiple-Choice Items

35. Key Requirements • Note: Parallel in form & Homogeneous in content Multiple-Choice Items

36. Parallel in form & Homogeneous in content Multiple-Choice Items

37. Assignment • 10 regular multiple-choice. Try to ensure that some are at the higher level;s of cognition. Multiple-Choice Items