Writing effective multiple-choice questions

1. Writing effective multiple-choice questions

2. Use a question format or a blank at the end of the stem. • Poor Item • Improved Item It takes approximately ______ for an earthquake P-wave to travel the first 6500 kilometers after the earthquake occurs. a. 6.5 minutes b. 8.0 minutes c. 10.0 minutes d. 18.5 minutes How long does it take for an earthquake P-wave to travel the first 6500 kilometers after the earthquake occurs? a. 6.5 minutes b. 8.0 minutes c. 10.0 minutes d. 18.5 minutes

3. Avoid absolute descriptors (e.g., always and never). • Poor Item • Improved Item If you find someone lying on the ground face-down and unconscious, you should a. only assist them if there’s someone else available to assist you. b. call 911. c. always check the pulse. d. never roll the person over. If you find someone lying on the ground face-down and unconscious, what is the first thing you should do? a. assist them, if there’s someone else available to assist you. b. call 911. c. check the pulse. d. roll the person over.

4. It is a dead giveaway when all the distractors, save the “key”, have absolutes as part of the question.

5. Present most of the content in the stem rather than in the options; but stem should be clear and focused. • Poor Item • Improved Item Pennsylvania a. is nicknamed the Empire State. b. borders 6 other states. c. was the birthplace of two U.S. presidents. d. has a mountain that is 3500 feet above sea level. What is the nickname for the state of Pennsylvania? a. the Empire State b. the Keystone State c. the Sunshine State d. the Show Me State

6. Focus the students’ attention on the information you want them to actually know.

7. Word the stem positively, not negatively. • Poor Item • ImprovedItem Which of the following is not a common trait of invasive species: a. Tolerance for a range of environmental conditions b. Association with humans c. Localized reproduction d. Fast growth Which of the following is NOT a common trait of invasive species: a. Tolerance for a range of environmental conditions b. Association with humans c. Localized reproduction d. Fast growth

8. Correct answer: B • We tend to teach what IS, not what isn’t. • Students are accustomed to learning what IS. • This format is hard on the reader; does not transfer well from the way information was learned/presented. • The example presented here is the way to highlight the item if you can’t change it. The better strategy is the make the statement positive, make the key (correct answer) a true response, and create plausible distractors around it.

9. Write options that are independent, not overlapping. • Poor Item • Improved Item How long does an annual plant generally live? a. It dies after the first year. b. It lives for many years. c. It lives for more than one year. d. It needs to be replanted each year. How long does an annual plant generally live? a. 1 year b. 2 years c. 3 years d. It blooms without replanting each year for up to 10 years. Taken from: Burton, Sudweeks, Merrill, & Wood, 1991.

10. Order options logically. • Improved Item • Poor Item How many different wing types did the Wright Brothers test before they settled on their successful design? a. 2 b. 200 c. 18 d. 150 How many different wing types did the Wright Brothers test before they settled on their successful design? a. 2 b. 18 c. 150 d. 200

11. Order options: Chronologically • Numerically • Alphabetically • Less cognitive load on the test taker and for someone who knows the answer, there’s no reason to complicate the ordering.

12. Avoid options such as ‘All of the Above,’ ‘None of the Above, ’ and Type K options. • Poor Item • ImprovedItem Julie has 50 jelly beans in a bag and 20% of the jelly beans are red. How many of the jelly beans in Julie’s bag are red? a. 5 b. 10 c. 25 d. None of the above Julie has 50 jelly beans in a bag and 20% of the jelly beans are red. How many of the jelly beans in Julie’s bag are red? a. 5 b. 10 c. 25 d. 35

13. For “All of the above” questions—if test takers know two options are correct, then they know the answer • For “None of the above” questions—If test takers know even oneoption is correct, they can eliminate “None of the above” as an option. • In either case, it’s a partial knowledge issue • Type K (choose several options) questions—Less discriminating, VERY cognitive demanding but hasn’t been substantiated that they are truly “more difficult”. They mainly take a long time to write and take.

14. Keep options as homogeneous as possible, in parallel form, and consistent with the stem. • Poor Item • Improved Item Mark Twain was an _____. a. Navy pilot. b. chemist. c. 19th century architect. d. American author. Mark Twain was a/an _____. a. African author. b. European author. c. Australian author. d. American author.

15. Present plausible distracters. • Poor Item • ImprovedItem Which American city boasts the nation’s largest city park system? a. Pittsburgh b. Missouri c. Dallas d. Denver Which American city boasts the nation’s largest city park system? a. Pittsburgh b. Kansas City c. Dallas d. Denver

16. Keep distracter (and key) lengths similar. • Poor Item • Improved Item In the Mediterranean region, cattle raising is handicapped by: a. Challenges of a hot climate b. Low beef demand among consumers c. Insufficient productive pasturage d. High labor costs for ranchers In the Mediterranean region, cattle raising is handicapped by: a. Hot climate b. Low beef demand c. A lack of sufficient productive pasturage d. High labor costs

17. Provide only 1 ‘correct’ answer. • Poor Item • Improved Item Consider the following equation: The answer, 10, is referred to as the a. mean b. variance c. average d. range Consider the following equation: The answer, 10, is referred to as the a. mean b. variance c. standard deviation d. range 13 + 9 + 11 + 7 4 13 + 9 + 11 + 7 4 = 10 = 10

18. References Haladyna, T. M., Downing, S. M., & Rodriguez, M. C. (2002). A review of multiple-choice item-writing guidelines for classroom assessment. Applied Measurement in Education, 15(3), 309-333. Notar, C. E., Zuelke, D. C., Wilson, J. D., & Yunker, B. D. (2004). The table of specifications: Insuring accountability in teacher made tests. Journal of Instructional Psychology, 31(2), 115-129. Rohrer, D., Taylor, K., & Sholar, B. (2010). Tests enhance the transfer of learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36, 233‑239.