Introduction & Training Learning by Student Generated Questions

Introduction & Training Learning by Student Generated Questions Seán Mâtheiken Esther Bergman

Seán Mâtheiken Vascular surgery SpR London Deanery, United Kingdom PhD: Online learning for vascular surgeons (working title) Esther Bergman Anatomy/Embryology Maastricht University Maastricht, The Netherlands PhD: Anatomy Education in the Undergraduate Medical Curriculum (working title) The researchers

The experiment: Study A2 • One of the challenges teachers encounter is how to encourage a deeper understanding of the key areas of the curriculum by students. Students in general, and medical students in particular, are often described as ‘strategic learners’, but in reality become ‘superficial learners’ out of necessity when faced with the seemingly boundless volume of material in today’s curriculum. To help the students, they are often given assignments designed to improve their understanding of the topic they are studying. 3

Multiple Choice Questions (MCQs) • Common complaints of students when answering • Easy to answer • Badly written • Too many questions about details • ... • Well proven that writing questions has many learning benefits – but this is seldom practised • This your chance to show us how it should be done!

Assignment: Writing MCQs • Some help! Prescribed Study Material • Scaffolding • (Instructional) scaffolding is the provision of sufficient support to promote learning when concepts and skills are being first introduced to students. • Correlation with each week’s clinical contact • Quality check

Question Structure • A good multiple choice question contains • Question name/number • Stem • Lead in question • Choices • Indication of correct answer + 3 distracters • Explanation of the choices (answer + distracters) • Sufficiently detailed explanation of the concept • not neccassary to describe each individual (in)correct answer • Literature reference

Question Structure: Example Question name/number • #1. Man cannot get up! • A 65-year-old man has difficulty rising from a seated position and straightening his trunk, but he has no difficulty flexing his leg. • Which of the following muscles is most likely to have been injured? • Gluteus maximus* • Gluteus minimus C. Hamstrings D. Iliopsoas E. Obturator internus STEM Lead-in question Choices with correct answer indicated

Scaffolding: Procedural prompts • Interrogative pronouns • What … • Why … • When … • Who … • Where … • How … • long • often • much • …

Procedural prompts: generic question stems Cause-effect relationships • … related to … • … consequence of … • … caused by … • … connected to … • … depending on … • … an effect of … • … explained by … List -strategy • … compared to … • … the same as … • … an example of … • … the opposite of … • … part of / not part of … • … different from …

Practice session • Read article • What is necessary to diagnose constipation? • Rao et al. (2011) • Try to write some mcq’s • Use scaffolding & procedural prompts • handout

Issues Related to Testwiseness (‘cheating’) • Grammatical cues - one or more distracters do not follow grammatically from the stem • Logical cues - a subset of the options is collectively exhaustive • Absolute terms - terms such as “always” or “never” are in some options • Long correct answer - correct answer is longer, more specific, or more complete than other options • Word repeats - a word or phrase is included in the stem and in the correct answer • Convergence strategy - the correct answer includes the most elements in common with the other options

Grammatical cues • Constipation is a symptom based disorder that is caused by multiple mechanisms. Primary constipation involves three pathophysiological subtypes: slow transit constipation, dyssynergic defecation and constipation-predominant irritable bowel syndrome. • Dyssynergic defecation is characterized by: • difficulty or inability to evacuate stool from the rectum* • discomfort or pain as a prominent feature • prolonged delay of stool transit through the rectum • Grammar is perfect, does not give a clue • If grammar is wrong, it is most of the time an incorrect answer

Long correct answer • A 42-year old man is complaining about constipation for about 3 weeks. • What may be the cause of primary constipation? • Drugs • Eating less fibre • Prolonged delay of stool transit through the colon* • Peritonitis • Correct answers are often the longest ones to make absolutely sure they are correct

Converge strategy • Problems with defecation are a big issue for patients. • Many tests are useful for the diagnosis of dyssynergic defecation. Which of the following is SELDOM used? • Anorectal manometry and endoscopy • Rectal barostat test and colonic manometry • The balloon expulsion test and anorectal manometry* • Anorectal manometry and clinical history • Option b would seem to be wrong, because it is the only one which does not contain anorectal manometry

Issues Related to Irrelevant Difficulty • Options are long, complicated, or double • Numeric data are not stated consistently • Terms in the options are vague (eg, “rarely,” “usually”) • Language in the options is not uniform • Options are in a nonlogical order • “None of the above” is used as an option • Stems are tricky or unnecessarily complicated • The answer to an item is “hinged” to the answer of a related item

General guidelines (NBME) – Some DOs • Make sure the item can be answered without looking at the options OR that the options are 100% true or false. • Include as much of the item as possible in the stem; the stems should be long and the options short. • Write options that are grammatically consistent and logically compatible with the stem; list them in logical or alphabetical order. Write distractors that are plausible and the same relative length as the answer. • Most important of all: Focus on important concepts; don’t waste your time (and bore your reader) with trivial facts.

General guidelines (NBME) – and some DON’TS • Avoid superfluous information. • Avoid “tricky” and overly complex items. • Avoid using absolutes such as always, never, and all in the options • Avoid using vague terms such as usually and frequently. • Minimise negatively phrased items (eg, those with except or not in the lead-in). If you do use a negative stem, use only short (preferably single word) options. • And most important of all: Focus on important concepts; don’t waste time testing trivial facts.

F A Q s • How many questions should I write? • What if I can only think of 2 or 3 answer choices? • I’m not confident about my English grammar. Can Esther translate my questions from Dutch for Seán? • Why should I take part in this experiment? • Can I get Prince Harry to marry me /my sister?

FAQs answered… • How many questions should I write? (NEXT PAGE) • What if I can only think of 2 or 3 answer choices? (We can help) • I’m not confident about my English grammar. Can Esther translate all my questions from Dutch for Seán? (NO) • Why should I take part in this experiment? (Esther and Seán are going to be very famous, which could open many doors for you…) • Can I get Prince Harry to marry me / my sister? (See above)

How many questions should I write??? • As many as needed to cover the all key concepts in the article • Somebody who has not read the article should, by attempting to answer your questions and reading your explanations, have a good idea what the article is about • Remember: your questions should be fit for your peers! And not overly detailed ;) • The purpose of these questions is to teach the content of the article, not just to test the reader • It is very unlikely that you can cover the content of a standard journal article with less than 4 questions...

Basis Group 1-4 Part 1 Student 1 + Student 2 Student 3 + Student 4 Student 5 + Student 6 Student 7 + Student 8 Student 9 + Student 10 Part 2 Student 1 Student 2 Student 3 Student 4 Student 5 Student 6 Student 7 Student 8 Student 9 Student 10 Basis Group 5-8 Part 1 Student 1 Student 2 Student 3 Student 4 Student 5 Student 6 Student 7 Student 8 Student 9 Student 10 Part 2 Student 1 + Student 2 Student 3 + Student 4 Student 5 + Student 6 Student 7 + Student 8 Student 9 + Student 10 Experimental design DUO’s Solo’s Solo’s DUO’s

www.bloodysurgery.com/moodle

www.bloodysurgery.com/HowToRegister (Mind the capital letters - or they will come back to bite you!)

www.bloodysurgery.com/HowToMoodle (Mind the capital letters - or they will come back to bite you!)

Thank you for your attention Questions? Now…or email us… or walk in during our lunchtime Open Door today (from 13:00 hours in the ‘studiezaal’of the Anatomy Department) sean@bloodysurgery.com e.bergman@maastrichtuniversity.nl

Introduction & Training Learning by Student Generated Questions