David DiBattista, Ph.D. Brock University Psychology Department

1. Getting the Most Out of Multiple-choice Questions David DiBattista, Ph.D.Brock UniversityPsychology Department

2. Overview • About multiple-choice testing • Two challenges in MC testing • Addressing the challenges

3. About MC Testing • Most common selected-response format • Permits broad coverage of material • Practical even with large classes

5. About MC Testing • Most common selected-response format • Permits broad coverage of material • Practical even with large classes

6. Some Important Terminology • The stem is a question or incomplete statement that is followed by several alternatives. • One alternative is the correct answer (or key) and the others are distracters. • The correct-answer format is typical, with the best-answerformat less common.

7. Overview • About multiple-choice testing • Two challenges in MC testing • Addressing the challenges

8. Two Challenges in MC Testing Challenge #1 Constructing high-quality MC items takes time, effort, and skill. Challenge #2 MC items are too often used to assess what students remember, with higher-level cognitive skills getting little attention.

9. Overview • About multiple-choice testing • Two challenges in MC testing • Addressing the challenges 1. Constructing high-quality items 2. Assessing higher-level thinking

10. Tips for MC Item Construction • Use question format whenever possible and sentence-completion format only when necessary. In either case, use appropriate punctuation and capitalization.

11. When hungry rats are given small bits of food at fixed intervals, excessive water consumption occurs. What is this excessive consumption called? A. primary drinking B. osmometric drinking C. psychogenic polydipsia D. schedule-induced polydipsia

12. When hungry rats are given small bits of food at fixed intervals, excessive water consumption occurs. This excessive consumption is known as A. primary drinking. B. osmometric drinking. C. psychogenic polydipsia. D. schedule-induced polydipsia.

13. Tips for MC Item Construction • If you use completion format, always place the blank at the very end of the stem.

14. Poor Generally speaking, ___________ testing supports content sampling better than do other testing formats. A. essay B. performance C. multiple-choice D. paper-and-pencil

15. Improved Generally speaking, content sampling is best supported by A. essay testing. B. performance testing. C. multiple-choice testing. D. paper-and-pencil testing.

16. Better still Generally speaking, which type of testing best supports content sampling? A. essay testing B. performance testing C. multiple-choice testing D. paper-and-pencil testing

17. Tips for MC Item Construction • The stem should present the issue under consideration clearly and contain as much information as possible. • Do not include irrelevant information in the stem unless it plays a role in the assessment procedure.

18. Poor South America A. imports coffee from Australia. B. is where the Gobi Desert is found. C. was colonized mostly by people from Spain. D. has a larger population than the United States of America.

19. Even worse South America,which has an area of more than 17 million square kilometres, A. imports coffee from Australia. B. is where the Gobi Desert is found. C. was colonized mostly by people from Spain. D. has a larger population than the United States of America.

20. Better Most of South America was colonized by people from what country? A. Spain B. France C. Holland D. England

21. A distribution of 100 test scores is positively skewed, with a mean of 70 and a standard deviation of 10. John’s test score is 80. What is his standard score? A. -1.0 B. +1.0 C. -10.0 D. +10.0

22. A distribution of 100 test scores is positively skewed, with a mean of 70 and a standard deviation of 10. John’s test score is 80. What is his standard score? A. -1.0 B. +1.0 C. -10.0 D. +10.0

23. Tips for MC Item Construction • Check carefully for spelling errors, giving special attention to distracters. • If you use sentence-completion format, check carefully for grammatical consistency of stem and alternatives.

24. Poor Stamp collectors use stamp hinges to A. pick up stamps. B. fold learge stamps in half. C. mount stamps in albums. D. joining two stamps together.

25. Improved Stamp collectors use stamp hinges to A. pick up stamps. B. fold large stamps in half. C. mount stamps in albums. D. join two stamps together.

26. Better still What do stamp collectors use stamp hinges for? A. picking up stamps B. folding large stamps in half C. mounting stamps in albums D. joining two stamps together

27. Tips for MC Item Construction • Whenever possible, avoid negative wording in the stem, and be sure to emphasize it when it does occur.

28. Poor Which term is not usually associated with Sigmund Freud? A. id B. extinction C. repression D. latent content

29. Better Which term is NOT usually associated with Sigmund Freud? A. id B. extinction C. repression D. latent content

30. Also better Which term is usually associated with behaviourism? A. id B. extinction C. repression D. latent content

31. Tips for MC Item Construction • All distracters should be plausible. • Four alternatives will usually be quite adequate, but the number used is best determined by the number of plausible distracters you can supply.

32. Which river flows through the city of Edmonton? A.North Saskatchewan River B. Athabasca River C. Peace River D. Milk River

33. Which river flows through the city of Edmonton? A. North Saskatchewan River B. Mississippi River C. Amazon River D. Rhine River E. Seine River F. Nile River

34. Tips for MC Item Construction • To generate plausible distracters • Use students’ most common errors on constructed-response tests. • Use distracters that are similar to the correct answer in content, length, and complexity. • Use words that sound important or have associations to the stem. • Use distracters that are true, but do not correctly answer the question.

35. Name the river that flows through the city of Edmonton. Athabasca River What do stamp collectors use stamp hinges for? To join stamps together

36. Tips for MC Item Construction • To generate plausible distracters • Use students’ most common errors on constructed-response tests. • Use distracters that are similar to the correct answer in content, length, and complexity. • Use words that sound important or have associations to the stem. • Use distracters that are true, but do not correctly answer the question.

37. In severe cases of obesity, there may be a substantial increase in the number of adipocytes. What term is used to refer to this increase? A. hyperbole B. hyperplasia C. hypertrophy D. hypertonicity

38. Tips for MC Item Construction • To generate plausible distracters • Use students’ most common errors on constructed-response tests. • Use distracters that are similar to the correct answer in content, length, and complexity. • Use words that sound important or have associations to the stem. • Use distracters that are true, but do not correctly answer the question.

39. In responding to a lengthy survey, a man answers “yes” to every yes-no question asked. It is reasonable to suspect that his responses may be influenced by which of the following? A. response acquiescence B. opportunistic characterization C. the partial reinforcement effect D. the conspicuous agreement predisposition

40. Tips for MC Item Construction • To generate plausible distracters • Use students’ most common errors on constructed-response tests. • Use distracters that are similar to the correct answer in content, length, and complexity. • Use words that sound important or have associations to the stem. • Use distracters that are true, but do not correctly answer the question.

41. Which of the following events caused the Prime Minister of Canada to proclaim the War Measures Act? • Newfoundland was occupied by German • forces in 1940. • B. The October Crisis occurred in 1970. • C. The Quebec Referendum was held in • 1980. • D. The Meech Lake Accord was defeated in • 1990.

42. Tips for MC Item Construction • Avoid patterns in the length and location of correct answers that could provide clues that are unrelated to content. • Balance the answer key so that the correct response appears in each position about the same number of times.

43. Your baseball team has a runner on first base. Under what circumstances might it be a good idea for the batter to lay down a sacrifice bunt? • when it is a night game • B. when the pitcher is a relief pitcher • C. when the batter is a right-handed hitter • D. when it is late in the game, the score is • tied, there are no outs, and the batter is a • weak hitter with less than two strikes

44. In a four-alternative multiple-choice test, about how often should the correct answer appear in each of the four locations? A. 10% of the time B. 25% of the time C. 40% of the time D. 60% of the time

45. Who invented the binaural recording system commonly known as “stereo”? A. Xxxxxxxxxxxxxx B. Xxxxxxxxxxxxxx C. Xxxxxxxxxxxxxx D. Xxxxxxxxxxxxxx

46. Who invented the binaural recording system commonly known as “stereo”? A. Alan Dower Blumlein B. Alan Dower Blumlein C. Alan Dower Blumlein D. Alan Dower Blumlein

47. Tips for MC Item Construction • Do not use “none of the above.”

48. “None of the above” as a distracter Poor Which of these 19th century authors wrote Wuthering Heights? A. Emily Bronte B. George Eliot C. Charles Dickens D. None of the above

49. “None of the above” as a distracter Better Which of these 19th century authors wrote Wuthering Heights? A. Emily Bronte B. George Eliot C. Charles Dickens D. William Thackeray

50. “None of the above” as the key Poor Which of these 19th century authors wrote Middlemarch? A. Jane Austen B. Anne Bronte C. Willkie Collins D. None of the above