Multiple Choice Questions

Multiple Choice Questions Centre for Teaching & Learning2009 Email: bmarchessault@georgianc.on.ca nkonich@georgianc.on.ca aboelryk@georgianc.on.ca

Workshop Objectives • List MCQ structures, benefits, issues • Develop good writing tips • Analyze some MCQ problems • Investigate guess-work

Stem Which of the following is a physical change? • a. rusting of a nail • b. burning of wood • *c. freezing of water • d. ignition of gasoline Distracters Answer Alternatives The parts of a multiple choice question

Knowledge and Comprehension Application and Analysis Synthesis and Evaluation Advantages of well designed Multiple-Choice questions • can measure all levels of cognitive ability • simplify test taking/grading allowing wider sampling than essay questions • machine scoring, efficient and accurate • versatile: appropriate for use in many different subject-matter areas • objective measurement - no human judgment in the scoring (huh?) • test banks available from publishers

Measuring COMPREHENSION with a MCQ Objective: Identifies the effect of changing a parameter (rule using). A pendulum consists of a sphere hanging from a string. What will happen to the period of the pendulum if the mass of the sphere is doubled? (Assume that the effects of air friction and the mass of the string are negligible, and that the sphere traces an arc of 20° in a plane as it swings.) a. It will increase. b. It will decrease. *c. It will remain unchanged. d. More information is needed to determine what will happen.

Measuring APPLICATION with a MCQ Objective: Identifies the correct application of principle (problem solving). In the diagram above, parallel light rays pass through a convex lens and converge to a focus. They can be made parallel again by placing a: a. Concave lens at point B. b. Concave lens at point C. c. Second convex lens at point A. d. Second convex lens at point B. *e. Second convex lens at point C.

Measuring ANALYSIS with a MCQ Objective: Analyzes poetry and identifies patterns and relationships. [The poem is included here.] The chief purpose of stanza 9 is to: a. Delay the ending to make the poem symmetrical. b. Give the reader a realistic picture of the return of the cavalry. c. Provide material for extending the simile of the bridge to a final point. *d. Return the reader to the scene established in stanza 1.

Limitations of Multiple-Choice questions • difficult and time consuming to construct • can favor simple recall of facts • place a high degree of dependence on the student’s reading comprehension and the instructor’s writing ability • can be easier than open-ended questions asking the same thing… it is harder to recall an answer than to recognize it. • diagnosis of learning problems hard Why did the student select the wrong answer?

Writing Stems • The Leaning Tower of Pisa … • will fall over in 100 years. • was built on weak soil. • was the site of Galileo’s experiment with gravity. • Why will the Leaning Tower of Pisa fall over? • Because it is getting very old. • Because it was built on weak soil. • Because of Galileo’s gravity experiments from the tower. • Write in a direct, positive format. • Minimize use of negatives (capitalize if needed, i.e. NOT) • Place the bulk of the problem, information or situation in the stem. • Use complete sentences or questions in the stem. • A question is better than an open ended phrase.

Writing Stems • The French flag has three colours. One of them is... • Red • Black • Green • Which of the following is a colour on the French flag? • a. Red • b. Black • c. Green • Stem should not pose more than one problem, although the solution to that problem may require more than one step • Clarity of language is essential • Eliminate unnecessary words or extra info • Ensure grammar is correct • "All of the above" NOTgood, one recognized distracter eliminates it. • Review your test with peers

Writing Stems • Add complexity • Present several interconnected concepts in the initial problem or situation • Present a case • Present a cause and effect situation • Require differentiation through analysis • i.e. Which of the following are correct? STEM with interconnected concepts: McGregor’s economic theory on emerging markets applies to Vietnam’s monetary exchange policy in the following way. CONCEPT 1 CONCEPT 2

When possible, state the stem as a direct question rather than as an incomplete statement Which is the better stem? • Alloys are ordinarily produced by ... • How are alloys ordinarily produced?

Present a definite, explicit and singular question or problem in the stem Which is the better stem? • Psychology... • The science of mind and behavior is called ...

Eliminate excessive verbiage or irrelevant information in the stem Which is a better stem? • While ironing her formal, Jane burned her hand accidentally on the hot iron. This was due to a transfer of heat by... • Which of the following ways of heat transfer explains why Jane's hand was burned after she touched a hot iron?

Include in the stem any word(s) that might otherwise be repeated in each alternative Which is a better stem? • In Canada the leader of a political party is officially chosen by a. the Canadian people b. members of parliament c. the Senate *d. the party delegates at a convention • In Canada the leader of a political party is officially a. chosen by the Canadian people b. chosen by members of parliament c. chosen by the Senate *d. chosen by the party delegates at a convention

Use negatively stated stems sparingly. When used, underline and/or capitalize the negative word Which is a better stem? • Which of the following is not cited as an accomplishment of the Mulroney government? • Which of the following is NOT cited as an accomplishment of the Mulroney government?

Is this a good stem? • What led to the formation of the Provincial Rights Party? • a. The level of federal taxation • b. The demand of provinces for the right to make their own laws • c. The industrialization of the Ontario • d. The corruption of federal legislators on the issue of provincial taxation

Writing alternatives • Write the correct answer first, then the alternatives • How many alternatives? • Avoid copying phrases from your textbook • “Never”, “always”, “all of the above”, “none of the above” and multiples (a,c and d) – be careful! You will never know if students know the correct answer!

Writing alternatives • Plausible alternatives reflect • Errors in logic • Common mistakes • Use parallel sentence structures (and/or homogeneous phrases - using same categories) • Make alternatives internally consistent • Similar in length if possible • Make all alternatives concise

Writing alternatives • Include key words from the correct answer in several alternatives • Organize alternatives in a logical order (shortest to longest, least to most, etc) • Include more than 1 decision-making factor in each alternative • Avoid presenting polar opposites

Writing alternatives • Present a true answer that does not answer the question • Use correct terms incorrectly • Use correct answers from previous in-class tests or practice quizzes as incorrect alternatives

Make all alternatives plausible and attractive to the less knowledgeable or skillful student Which alternatives are better? • What led to the crash of flight ET109? • a. Faulty fan blade • b. Failed electrical system • *c. Bird impact • d. Lightning strike • What led to the crash of flight ET109? • a. Faulty fan blade • b. Failed electrical system • *c. Bird impact • d. Passenger list errors

Make alternatives grammatically parallel with each other, and consistent with the stem Which alternatives are better? • What would do most to advance the application of atomic discoveries to medicine? *a. Standardized techniques for treatment of patients. b. Train the average doctor to apply radioactive treatments. c. Remove the restriction on the use of radioactive substances. d. Establishing hospitals staffed by highly trained radioactive therapy specialists. • What would do most to advance the application of atomic discoveries to medicine? *a. Development of standardized techniques for treatment of patients. b. Training of the average doctor in application of radioactive treatments. c. Removal of restriction on the use of radioactive substances. d. Addition of trained radioactive therapy specialists to hospital staffs.

Make the alternatives mutually exclusive Which alternatives are better? • The daily minimum required amount of milk that a 10 year old child should drink is • a. 1-2 glasses. • *b. 2-3 glasses. • *c. 3-4 glasses. • d. at least 4 glasses. • What is the daily minimum required amount of milk a 10 year old child should drink? • a.1 glass. • b. 2 glasses. • *c. 3 glasses. • d. 4 glasses.

When possible, present alternatives in some logical order (e.g., chronological, most to least, alphabetical) Which alternatives are better? • At 7 a.m. two trucks leave a diner and travel north. One truck averages 42 miles per hour and the other truck averages 38 miles per hour. At what time will they be 24 miles apart?a. 6 p.m. b. 9 p.m. c. 1 a.m. *d. 1 p.m. e. 6 a.m.At 7 a.m. two trucks leave a diner and travel north. One truck averages 42 miles per hour and the other truck averages 38 miles per hour. At what time will they be 24 miles apart? a. 1 a.m. b. 6 a.m. c. 9 a.m. *d. 1 p.m. e. 6 p.m.

Which alternatives are better? The two key problems that led to the 2008 economic collapse were toxic debt instruments and a. banker’s excessive bonuses. *b. the credit crisis. c. international debt. d. currency fluctuations. The two key problems that led to the 2008 economic collapse were toxic debt instruments and a. banker’s excessive bonuses. *b. the credit crisis. c. international debt. d. failure of inter-bank lending. Be sure there is only one correct response to the item same thing

Make alternatives approximately equal in length Which alternatives are better? The most general cause of low individual incomes in Canada is *a. lack of valuable productive services to sell. b. unwillingness to work. c. automation. d. inflation. What is the most general cause of low individual incomes in Canada? *a. Lack of valuable productive services to sell. b. The population's overall unwillingness to work. c. The nation's increased reliance on automation. d. An increasing national level of inflation.

Instructions What & why • Give Clear Instructions Questions 1 - 10 are multiple-choice questions designed to assess your ability to remember or recall basic and foundational pieces of knowledge related to this course. Please read each question carefully before reading the answer options. When you have a clear idea of the question, find your answer and mark your selection on the answer sheet. Please do not make any other marks on this exam sheet. Questions 11 – 20 are multiple-choice questions designed to assess your ability to think critically about the subject. Please read each question carefully before reading the answer options. Be aware that some questions may seem to have more than one right answer, but you are to look for the one that makes the most sense and is the most correct. When you have a clear idea of the question, find your answer and mark your selection on the answer sheet. How

“The Art of Guesswork” • READ, ELIMINATE, RE-READ • Read carefully • Slow down at words which link by causation such as "due to" and "because" or "as a result of' and at words of totality such as "never" or "always." • Eliminate wrong choices • Do not jump to the answer that you think is correct. • Re-read the question, as if you were reading it for the first time. Nowchoose your answer from your remaining answers based on this re-reading.

Guesswork • Rule of thumb: "Pick the longest answer.“ • Way to defeat this strategy: make sure the longest answer is right about a fifth of the time (if there are five alternatives for each question)

Guesswork • Rule of thumb: "Pick the B or C alternative.“ • Way to defeat this strategy: make sure each answer is used the same number of times, in random order.

Guesswork • Rule of thumb: "Never pick an answer which uses the word 'always' or 'never' in it.“ • Way to defeat this strategy: make sure such answers are correct about a fifth of the time

Guesswork • Rule of thumb: "If there are two answers which express opposites, pick one or the other and ignore other alternatives.” • Way to defeat this strategy: sometimes offer opposites when neither is correct.

Guesswork • Rule of thumb: "If in doubt, guess.“ • Way to minimize the impact of this strategy: use five alternatives instead of three or four

Guesswork • Rule of thumb: "Pick the scientific-sounding answer.” • Way to defeat this strategy: use scientific sounding jargon in wrong answers

Guesswork • Rule of thumb: "Don't pick an answer which is too simple or obvious.“ • Way to defeat this strategy: sometimes make the simple, obvious answer the correct one.

Guesswork • Rule of thumb: "Pick a word which you remember was related to the topic.“ • Way to defeat this strategy: when drawing up distracters use terminology from the same area of the text as the right answer, but in distracters use those words incorrectly so the wrong answers are definitely wrong.

Practice • Form teams of two or three • Write 2 versions of a MCQ (any subject): • Problematic version • Ideal version • Present to class and explain

Multiple Choice Questions