More Than Just Reading Aloud: Strategies for Oral Test Administration

1. More Than Just Reading Aloud: Strategies for Oral Test Administration Emergency Cardiovascular Care Update ConferenceLas Vegas, Nevada, June 14, 2008 Tom Schmitz & Mary S. Barber-Schmitz Twin Cities Area Chapter American Red Cross

2. Conflict of Interest Disclosure • Commercial/industry • None to disclose • Potential intellectual conflicts • None to disclose

3. Presentation Composition

4. Types of Learning Objectives • Psychomotor Objectives (physical abilities) • Able to perform effective chest compressions • Able to correctly open the airway • Able to perform continuous cycles of CPR effectively • Cognitive Objectives (purely knowledge) • Knows signals of a heart attack • Knows characteristics of effective chest compressions • Knows sequence for performing skill steps • Affective Objectives (attitudes & beliefs) • Willing to respond in an emergency • Respects other providers • Sincerely intends to wear gloves every time

5. TYPES OF OBJECTIVES Psychomotor Cognitive Affective COMMON METHODS OF ASSESSMENT Skills tests Written tests Generally no formal assessment in classroom Assessing Achievement

6. Multiple-Choice Written Tests • One method of assessing achievement of cognitive learning objectives • The most common method used in national training programs for first aid, CPR, and AED • The type of written test for which our oral testing strategies have been developed

7. Good Things About Multiple-Choice Written Tests • Easy for instructor to administer • Easy for instructor to grade • Familiar to many students • “Objective”(At least in the sense that both the test and the scoring system are standardized.)

8. Written Test Idealism

9. Is This Idealism Justified? • A student fails the written test. • Does this mean, for sure, that the student doesn’t know the material? Of course not.

10. Some Reasons a Student Might Not Pass A Written Test • Didn’t learn the material, because... • Didn’t pay attention in class • Learning disability • Went to a movie instead of studying • Not able to comprehend language in which class was conducted • Illiteracy in primary language • Course improperly conducted

11. Other Reasons a Student Might Not Pass A Written Test • Did learn the material, but… • Can’t read the test (illiteracy) • Can’t read the test (language barrier) • Can’t read the test (left glasses at home) • Is distracted while taking the test • Is exhausted after a full day of class • Is unfamiliar with multiple-choice testing • Is unskilled at taking multiple-choice tests

12. Written Test Idealism

13. Written Test Reality

14. Written Test Reality Good Not Good Not Good Good

15. Oral Testing We would like to be able to detect caseswhere the student has met learning objectives,despite being unable to demonstrate this learningusing a written test. • Oral testing can detect some of these cases • Oral testing cannot detect all of these cases • We must be careful not to falsely “detect” cases where learning has not actually occurred

16. Know The Rules • Training organizations and programs vary on… • Whether oral testing is allowed at all • What forms of oral testing are allowed • Instructors responsible to know and follow applicable rules for each course they teach!

17. Know The Rules • We’re not addressing the rules of any specific program today • Instead, we’ll assume this generic rule: Oral testing is allowed in any form that does not have the effect of wrongly passing those who have not met the cognitive learning objectives • Your actual progam rules may be more restrictive

18. Strategies for Oral Test Administration

19. Plain Old Reading Aloud • Could help... • A student who left glasses at home • A student who is unable to read the written test due to being illiterate

20. Plain Old Reading Aloud • Won’t help much if the problem is... • A student unfamiliar with multiple-choice tests • A student unskilled at multiple-choice tests • A student taking the test in a non-primary language • Not an easy way for anybody to take a multiple-choice test

21. Suggested Strategies for Oral Test Administration • Strategy 1: Give An Introduction • Strategy 2: Consider Answers Separately • Strategy 3: Make It Concrete • Strategy 4: Visual Choices • Additional Tips

22. No national curriculum test questions were harmed in the making of this presentation

23. However, resemblance to actual test questions is not entirely coincidental

24. Strategy 1: Give An Introduction

25. Strategy 1: Give An Introduction You arrive at the scene of a construction accident. Which of the following is part of the scene size-up? • Determining if the victim is unresponsive • Assessing movement and breathing • Checking whether the scene is safe • Looking for non-life-threatening injuries

26. Strategy 1: Give An Introduction • Overcomes some limitations of oral delivery by alerting listener to main topic right from the start • Improves understanding of both the question and the answer choices • Reduces need to reread • Use along with other oral testing strategies • Low risk of giving away answer

27. Strategy 2: Consider Answers Separately

28. Strategy 2: Consider Answers Separately Why is it important for an AED to be used as quickly as possible on an adult victim? • Most adults initially suffer respiratory emergencies. • The victim’s chance of survival decreases greatly if defibrillation is delayed. • Cardiac emergencies are uncommon in adults. • Rescue breathing is the best care for sudden cardiac arrest.

29. Strategy 2: Consider Answers Separately • Strategy is simply to convert answer choices into separate true/false questions • Have student answer each true/false question independently of the others • Low risk of giving away answer

30. Strategy 2: Consider Answers Separately • Works well for many situations • Complex wording of question and/or answers • Non-parallel answer choices • Negative wording in question and/or answers • “All of the above,” “None of the above,” and “Both a and c” answer choices • Overcomes many common problems related to poor multiple-choice test-taking skills

31. Strategy 2: Consider Answers Separately In performing an initial assessment, you should check for all of the following except— • Consciousness • Airway obstruction • Burns • Severe bleeding

32. Strategy 2: Consider Answers Separately Which of the following can be signals of respiratory distress? • Altered level of consciousness • Broken bones • Changes in skin appearance • Both a and c

33. Strategy 2: Consider Answers Separately • Easy to interpret student responses • No need to deal directly with negative wording • No need to even mention “all/none/some of the above” choices • Select choice which matches pattern of student’s T/F answers • If pattern doesn’t match any choice, call attention to which choices are available

34. Strategy 3: Make It Concrete

35. Strategy 3: Make It Concrete In which situation should you provide CPR? • The victim is unresponsive, is not breathing, and has no pulse. • The victim is breathing but is having trouble breathing. • The victim is not breathing but has a pulse. • The victim is unresponsive but is breathing.

36. Strategy 3: Make It Concrete • Works well for many questions about assessment or about recognizing signs & symptoms (e.g., s/s of respiratory distress) • Gesturing to each of the hypothetical victims helps make clear that we’re comparing four separate situations • Low risk of giving away answer • Easy to interpret student response

37. Strategy 4: Visual Choices

38. Strategy 4: Visual Choices When giving abdominal thrusts to a severely choking child, where should you place your hands? • On the center of the breastbone • On the sides of the ribcage • Just above the navel • On the lower end of the breastbone

39. Strategy 4: Visual Choices • Clarifies choices by giving them visually • Show on self, another instructor, or on manikin • Only works well if all choices feasible to demonstrate (hard with nonsense choices) • Take care to demonstrate fairly and avoid nonverbal cues • Variation: Ask open-ended and have student show the answer

40. Additional Tips for Oral Test Administration

41. Additional Tips • Watch out for questions that are intended to test knowledge of terminology—don’t give away • Take care not to just keep “clarifying” question until correct answer is chosen—for oral test to be valid, student must be able to choose the wrong answer

42. Additional Tips • During oral testing, it might or might not be beneficial for student to have a copy of the test and to fill in the answer sheet • It’s really OK to write on the test (and it might help) • Identify chosen answers question by question, but score the test all at once

44. Key Ideas • Even students who learn information may not always pass a written test; oral testing can be useful in such situations • These strategies can make oral testing more useful • Take care not to misuse oral testing to pass those who should not pass • Know and follow rules for your program

45. Credits & Acknowledgements • Thanks to national training organization program staff who provided administrative information and review

46. Mary S. Barber-Schmitz, NREMT-B,American Red Cross Instructor TrainerTwin Cities Area Chapter American Red Cross mbarberschmitz@yahoo.com  612-770-4906 Tom Schmitz, NREMT-B,American Red Cross Instructor TrainerTwin Cities Area Chapter American Red Cross tschmitz@yahoo.com612-872-3237 Presenter Contact Information