Writing Quality Multiple Choice Questions

1. Writing Quality Multiple Choice Questions Facilitated By Sara Fridley Region 3 Education Service Agency sara.fridley@k12.sd.us

2. QUESTION – ANSWER RELATIONSHIP LITERAL INFERENTIAL

3. Detail/recall Reading Comprehension Vocabulary Chart/graph analysis Problem-solving Inference Predict Author’s purpose Cause & effect Types of QuestionsWhat Are You Assessing?

4. Summative & Formative Assessment • Assessment of Learning • (Summative Assessment) • How much have students learned as of a particular point in time? • Assessment for Learning • (Formative Assessment) • How can we use assessments to help students learn more?

5. Assessment ofLearning • Purpose is • to measure student achievement at a point in time for reporting and accountability • to sort students in rank order • to maximize student learning through standardized tests • Results used primarily by • policy makers & program planners • educators • Parents & students

6. Assessment forLearning • The purpose is • To promote further improvement of student learning during the learning process • To involve students in ongoing assessment of their own achievement • Identify student needs • Help students see and hit a target • Results are used primarily by • Educators – to adjust teaching • Parents & students

7. Which is your purpose? • OF learning • FOR learning

8. Disadvantages of Multiple Choice Questions • Quality items are difficult and time consuming to develop • Tendency for items to focus on low level learning objectives • No credit for partial knowledge • Assessment results may be biased by students’ reading ability and test-wiseness • May overestimate learning due to the ability to utilize an elimination process for answer selection • Does not measure the ability to organize and express ideas

9. With that in mind . . . • Allows for assessment of a wide range of learning objectives from factual to evaluative understanding • Amenable to item analysis • Permits manipulation of difficulty level by adjusting the degree of similarity among response options • Objective nature limits bias in scoring • Easily administered to large numbers of students • Limits assessment bias caused by poor writing skills • AND it is the format used on high-stakes tests

10. General Guidelines for Writing Questions • Keep wording simple and focused • Follow rules of grammar • Eliminate superfluous material • Test mastery of material, not ability to figure out what you’re asking • Avoid clues within items • Grammatical clues • Determiners such as “always” and “never” • Don’t lift wording directly from materials • write in your own words • Highlight critical words such as MOST, LEAST, EXCEPT • they are easily overlooked

11. Parts of the MC Question • Stem • Provides introductory information • Direct question or incomplete statement • Correct Answer • One and ONLY ONE is preferable • Distractors • Incorrect response options

12. General Guidelines for Multiple Choice Questions • State whole question in the stem • Keep responses brief and parallel in: • Length • Use of specific determiners • Grammatical construction • Generality • Avoid repetition in response options • Limit use of “all” or “none of the above” • Be sure there is only one correct or best answer • Best answer items should be worded to ask for BEST

13. More Guidelines • Negative premises are undesirable • may especially disadvantage ESL (English Second Language) students. • The answer to one question should not be obtainable from another question. • Avoid potential sources of bias & distortion • Wording • stimuli (graphics, pictures, charts) • Use at least 3 distractors to reduce the chance of guessing the correct answer. • Avoid using a pattern for the position of the correct response.

14. School Bus — Multiple Choice A school is planning a field trip and will be using school buses to transport the students. A school bus holds 36 students. If 1,128 students are being transported, how many buses are needed? a. 31 c. 32 b. 31.33 d. 36

15. Sample Questions The Good, The Bad, and The Ugly

16. Two cyclists start biking from a trail’s start 3 hours apart. The second cyclist travels at 10 miles per hour and starts 3 hours after the first cyclist who is traveling at 6 miles per hour. How much time will pass before the second cyclist catches up with the first from the time the second cyclist started biking? • 2 hours • 7 ½ hours • 4 hours, 30 minutes • 6 hours • 5 ¾ hours Practice question from ACT Prep booklet

17. If a circle has the diameter of 8, what is the circumference? • 6.28 • 12.56 • 25.13 • 50.24 • 100.48 Practice question from ACT Prep booklet

18. The wound exhibited signs of copious drainage requiring medical intervention.Copious means • Minimal • Clarity • Maximal • Foul Practice question from ACT Prep booklet

19. When New York freed its slaves, Isabella had • no children • five children • an education • three children

20. The concept of the plasma membrane as a simple sieve-like structure is not inadequate to explain the • passage of gases involved in respiration into and out of the cell • ability of the cell to admit selectively some inorganic ions while excluding others • Failure of protein molecules to pass through the membrane • Inability of the cell to use starch without prior digestion

21. Released test questions from the 2003 and 2004 California Standards Tests (CSTs).

22. The details about the princess’s life before her eighteenth birthday imply that she was - • Raised to value only material goods • Indifferent to the feelings of others • Easily bored • Both A and C • Both B and C, but not A Sample question from Standardized Test Practice

23. Purpose of Questions What are you asking the student to know?

24. Detail Where was Susan’s dad stationed overseas? • In Albany, New York • In Adu Jahm, near Baghdad • Among ancient ruins in southern Iraq • East of Iraq, in Iran Sample question from Scholastic Scope

25. Vocabulary – Multiple Meanings The subject of today’s lesson is words with multiple meanings. In which sentence does the word subject mean the same thing as in the sentence above? • Susan’s mother is subject to fainting spells. • In Kenya, high school students take 13 subjects at a time. • If you disobey traffic laws, you may be subject to arrest. • Feel free to change the subject of your report. Sample question from SAT9 Activities (McDougall-Littell).

26. Inference Sample Released test questions from the 2003 and 2004 California Standards Tests (CSTs).

27. Infer/Predict Sample Released test questions from the 2005 Massachusetts Standards Tests (CSTs).

28. Inference The details about the princess’s life before her eighteenth birthday imply that she was - • Raised to value only material goods • Indifferent to the feelings of others • Easily bored • Not interested in getting married Sample question from Standardized Test Practice

29. Comprehension Sample Released test questions from the 2003 and 2004 California Standards Tests (CSTs).

30. Author’s Purpose Released test questions from the 2003 and 2004 California Standards Tests (CSTs).

31. Author’s Purpose Released test questions from the 2003 and 2004 California Standards Tests (CSTs).

32. Application Sample Released test questions from the 2005 Massachusetts Standards Tests (CSTs).

33. Critical Analysis You can conclude from the passage that Epimetheus • did not want to know the contents of the box. • already knew what the box contained. • wanted to know what was in the box. • never did learn the contents of the box. Sample question from SAT9 Activities (McDougall-Littell).

34. Critical Analysis The poem suggests that John Henry dies from • shock. • a broken heart. • disappointment. • pushing himself too hard. Sample question from SAT9 Activities (McDougall-Littell).

35. Reading Comprehension Which of the following questions is answered in the article? • What causes heart damage during prolonged space travel? • Why might space travelers develop kidney stones? • What can space travelers do to avoid becoming depressed? • What will the interior of a passenger space plane look like? Sample question from Standardized Test Practice

36. Write Questions & Peer Review --- 25 minutes • 10-15 minutes • Write 4 questions • Check guidelines • 10-15 minutes • Read passage • Answer questions • Check guidelines • Provide feedback • Edit questions if needed