Written Exam Assessing knows and knows how

1. Written ExamAssessing knows and knows how DepartemenObstetridanGinekologi FKUI-RSCM

2. Three levels of test • Knowledge (recall or recognition of specific information) • Combined comprehension and application • Problem solving (transferring existing knowledge and skills to new situations)

3. Guidelines for Basic Item Content • Clinical vignettes • Focus on key concepts and principles that are essential information • Test material relevant to learning in clinic • Avoid items that only require recall of isolated facts • Avoid esoteric or interesting topics that are not essential

4. Tools Case vignettes should include: • age, gender • site of care • presenting complaint • Duration • patient history • physical findings • +/- diagnostic studies • +/- initial treatment Stems should: • not be completely based on real patients • include reference material when it would be realistic in practice • not use the patient’s or doctor’s own words • not include patients who lie

5. Multiple Choice Questions (MCQs)

6. Introduction • MCQs can assess taxonomically higher-order cognitive processing such as interpretation, synthesis and application of knowledge rather than the test of recall of isolated facts. • Should not be used as a sole assessment method in summative examinations, but alongside other test forms

7. Advantage Disadvantage they test recognition (choosing an answer) rather than recall (constructing an answer) allow for guessing difficult and time-consuming to construct • Can assess a broad range of learner knowledge in a short period of time (time effective) • Appropriately constructed MCQs result in objective testing that can measure knowledge,comprehension application and analysis • reliable, discriminatory, reproducible and cost-effective

8. Writing MCQs • Begins with identifying the most important information or skill that is to be learned • Focus on important and relevant content, avoiding testing the knowledge of medical trivia • Controversial items should be avoided

9. Components of MCQs • The item : entire unit and consists of a stem and several options. • The stem: the question, statement or lead-in to the question. • options or choices : the possible answers • keyed response: The correct option • foils or distractors: The incorrect options

10. Parts of MCQs “item” A 62-year-old woman comes to the physician because of bleeding from the vagina. She states that her last menstrual period came 11 years ago and that she has had no bleeding since that time. She has hypertension and type 2 diabetes mellitus. Examination shows a mildly obese woman in no apparent distress. Pelvic examination is unremarkable. An endometrial biopsy is performed that shows grade I endometrial adenocarcinoma Which of the following is the most appropriate next step in management? a. Chemotherapy b. Cone biopsy c. Dilation and curettage d. Hysteroscopy e. Hysterectomy * vignette Stem Lead in distractors Option answer

11. Technical Item Flaws Issues Related toTestwiseness • Grammatical cues • Logical cues • Absolute terms - such as always or never • Long correct answer • Word repeats • Convergence strategy

12. Issues Related to Irrelevant Difficulty • Options are long, complicated, or double • Numeric data are not stated consistently • Terms in the options are vague (eg, “rarely,” “usually”) • Language in the options is not parallel • “None of the above” is used as an option • Stems are tricky or unnecessarily complicated • The answer to an item is “hinged” to the answer of a related item

13. General Guidelines for Item Construction • Make sure the item can be answered without looking at the options, (the options are 100% true / false). • Avoid technical Item Flaws • Write distractors that are plausible and the same relative length as the answer.

14. General…. • Avoid using absolutes such as always, never, and all in the options; also avoid using vague terms such as usually and frequently. • Avoid negatively phrased items. If you must use a negative stem, use only short (preferably single word) options. And most important of all: Focus on important concepts; don’t waste time testing trivial facts

17. EXTENDED MATCHING QUESTIONS

18. EXTENDED MATCHING QUESTIONS • Extended Matching items are multiple-choice items organized into sets that use one list of options for all items in the set. A well-constructed Extended-Matching set includes four components (R-type): 1. a theme 2. an option list 3. a lead-in statement 4. at least two item stems

19. Overview of the Steps for Writing Extended Matching Items 1. Identify the theme for the set. 2. Write the lead-in for the set. 3. Prepare the list of options. 4. Write the items. 5. Review the items. Check to make sure that there is only a single “best” answer for each question.

20. Example of an EMQs • Theme: Bleeding per vagina • Options A. Cervicitis G. Endometrial cancer B. Atrophic vaginitis H. Endometrial polyp C. Cervical cancer I. Incomplete miscarriage D. Cervical polyp J. Threatened miscarriage E. Complete miscarriage K. Toxic-shock syndrome F. Ectopic pregnancy • Instructions/Lead-in For each of the patients described below choose the SINGLE most appropriate diagnosis from the list of options above. Each optionmay be used once, more than once or not at all. .

21. Item: • A 62-year-old woman comes to the physician because of bleeding from the vagina. She states that her last menstrual period came 11 years ago and that she has had no bleeding since that time. She has hypertension and type 2 diabetes mellitus. Examination shows a mildly obese woman in no apparent distress. Pelvic examination is unremarkable. Ultrasound revealed thickened endometrium 15 mm, with increase vascularization. • A 25 years old woman comes to emergency because of bleeding from vagina. She states that her last menstrual period came 2 months ago and she already did pack test and the result was positive. On vaginal examination the doctor found that the ostium was open and there was conception tissue in it.

22. Short Answer Questions

23. Introduction • Short-Answer questions require a student to respond to a question by writing a word or short phrase. • Questions can vary in expected student response- from one word to short sentence, and include forms : • complete the sentence • supply the missing line • short descriptive or qualitative answers

24. Advantage Disadvantage Assess a limited sample of the range of content. Are difficult and time consuming to grade. • Assess higher-order or critical thinking skills • Provide authentic experience. • easier to mark. • Evaluate student thinking and reasoning

25. Procedures for setting and marking • Make the questions precise • Direct questions are better than incomplete statements. • If a numerical answer is required, indicate the units and degree of precision required. • Prepare a structured marking sheet • Allocate marks or part-marks for acceptable answer(s). • Be prepared to accept other equally acceptable answers, some of which you may not have predicted.

26. Example of short answer questions A 62-year-old woman comes to the physician because of bleeding from the vagina. She states that her last menstrual period came 11 years ago and that she has had no bleeding since that time. She has hypertension and type 2 diabetes mellitus. Examination shows a mildly obese woman in no apparent distress. Pelvic examination is unremarkable. Ultrasound revealed thickened endometrium 15 mm, with increase vascularization. What is the most likely diagnosis of this patient

27. Thank you…..