Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity

1. Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill Francis S. Nuthalapaty, MD University of South Carolina - Greenville 2009 APGO Faculty Development Seminar

2. Objectives • Demonstrate ARS in action* • Discuss the rational for using Audience Response Systems (ARS) in medical education • Outline ARS question types and best practices • Outline practical concerns regarding use of ARS • Describe available ARS systems

3. Audience Response Systems • Definition: Electronic tool which allows lecturer to poll learners with immediate feedback. • Synonyms: • Classroom response system • Student response system • Personal response system

4. Instructions for Clickers • To vote: Press 1/A, 2/B etc… • Vote Status Light: • Green if your vote was recorded • Orange if not • To change your vote – press another key • Clickers work like an RF remote • Receiver is plugged into USB port

5. 9 of 40 Are you here today? • Yes • No

6. 9 of 40 I am a… • Clerkship director • Program director • Chair • Clerkship coordinator • Other

7. :00 What is your role in education? • Teaching students in the pre-clinical curriculum • Teaching students in the clerkship • Administration (Dean, Clerkship Director) • Other

8. How often do you currently use ARS in your teaching? • Never used ARS before • Rarely (<25% of my lectures) • Often (>50% of my lectures) • Very Often (>75% of my lectures)

9. Roles of ARS • Restarting the attention clock • Average human attention span: 20 minutes • Recall of information decreases after 15-20 minutes • At maximum 20-25% of material presented in standard lecture is retained

10. Is finding lecture material somewhat mundane, starts previewing following lecture for interesting slides Settling into lecture, previewing lecture slides Has renewed sense of interest in topic at hand Listens intently, hoping to catch key summary points

11. Roles for ARS • To increase interactions • To assess learner’s attitudes • To start or focus discussions • To vote after a debate or panel discussion • To assess preparation and ensure accountability • Questions about reading or homework

12. Roles for ARS • To engage learner throughout a lecture • To assess learner’s understanding and misunderstanding of topics during a lecture • To find out more about student attitudes on a topic • To assess need for more explanation or detail • For quizzes or tests • paying attention • taking good notes • preparing for class or labs • keeping up with homework • actively thinking

13. Prevalence of ARS • Used ubiquitously in many disciplines…mathematics, philosophy, psychology, economics • Used in many age groups…K-12, elementary, undergraduate, graduate

14. History • Pradhan et al • RCT of ARS v. standard lecture format on the topic of contraception • 17 ob/gyn residents at UMDNJ • Compared pretest/posttest (6 weeks later) scores • ARS: 21% improvement • Standard lecture: 2% improvement • P = 0.18

15. History • Nayak and Erinjeri • 27 medical students who were instructed to give a presentation at the end of course • From learners viewpoint • Increased self-confidence with understanding material and verbally answering questions • No differences in end of course exam scores • From presenters viewpoint • Made them more comfortable that they were conveying information effectively • Prepared them for future presentations

16. Pros…based on the literature • Instructor benefits • Immediate feedback on learners understanding, identify gaps • Improved retention of knowledge • Technical issues • Decreased error in collecting data because of electronic transmission • Anonymous • Can increase attendance • in one study only if participation was worth at least 10% of grade • Can reduce attrition through the duration of a course (4% vs 8-12%) • Particularly if class was midday

17. Pros…based on the literature • Increased learner satisfaction by improving interactivity and by engaging learner • Student gave instructors and “learning environment” more positive ratings • Allows learner to compare own understanding with others • Increases active participation • Learner must commit to answer, thus increasing attention • Affects active participation in other ways (asking more questions) • Less sleeping during class • Superior to show of hands technique with increased participation

18. Cons • Need initial investment in software and hardware • Program to create questions • “clickers” or other data entry modality • Training and buy-in from faculty • Willingness to participate by students • Some negative responses • Using ARS decreases time for actual delivery of material during the course of lecture • Can be distracting if overused in a single lecture (2-5/50 minute lecture)

19. Question Types • Factual Questions • Conceptual Questions • One-best-answer Questions • Opinion Questions • Data Gathering Questions • Questions Asking for Predictions • Feedback on Teaching

20. Question Types Factual Questions These questions might be used to see if students did the reading, remember important points from prior classes, or have memorized key facts. Vanderbilt Center for Teaching

21. Question Types Conceptual Questions Multiple-choice questions that demonstrate whether or not students understand important concepts and principles. Vanderbilt Center for Teaching

22. Question Types One-best-answer Questions Questions which include multiple answer choices, more than one of which could be argued as correct. Students are asked to select the one best answer from these choices. Also referred to as "partially correct multiple-choice questions" (Burnstein & Lederman, 2006) Vanderbilt Center for Teaching

23. Question Types Opinion Questions Evaluative and opinion questions which may not have correct answers. Asking these questions can engage students and provoke rich discussions, particularly in response to ethical, legal, or moral issues. Vanderbilt Center for Teaching

24. Question Types Data Gathering Questions Used to quickly gather demographic, opinion, or other data from a class full of students. This is often useful in social science classes that utilize social or economic experiments in the classroom. Vanderbilt Center for Teaching

25. Question Types Questions Asking for Predictions Students can be asked to predict the outcome of an experiment prior to being shown the experiment. This gives the teacher a sense of the students’ preconceptions and increases the surprise value of the experiment when students see just how many of their classmates expected different outcomes. Vanderbilt Center for Teaching

26. Question Types Feedback on Teaching Ask students to provide direct feedback on how well a particular class is going. For example, one might ask students to rate their level of understanding of the lecture thus far on a scale of 1 to 4. Vanderbilt Center for Teaching

27. Question Best Practices • Apply Bloom’s Taxonomy to test concepts rather than recall • Test understanding of jargon separately from understanding of concepts to be sure source of confusion can be clearly identified • Include “I don’t know” to ensure data truly reflects understanding rather than some sample of student guessing

28. Bloom’s Taxonomy

29. Bloom’s Taxonomy • Evaluation: think critically about and defend a position • Judge, Justify, Defend, Criticize, Evaluate • Synthesis: transform, combine ideas to create something new • Develop, Create, Propose, Design, Invent • Analysis: break down concepts into parts • Compare, Contrast, Distinguish • Application: apply comprehension to unfamiliar situations • Apply, Use, Compute, Solve, Predict • Comprehension: demonstrate understanding of ideas, concepts • Restate, Explain, Summarize, Interpret, Describe • Factual Knowledge: remember and recall factual information • Define, List, State, Name, Cite

30. Question Best Practices • Use variety of types of questions • Ask about linking a general concept to a specific example • Ask about linking a familiar idea to a new concept • Ask about a concept yet to be introduced to increase student curiosity • Can ask question twice • First for individuals • Second after student discussion to promote exchange of ideas

31. How would you rate this workshop so far? • Fascinating • Somewhat interesting • Boring • A waste of time

32. Question Writing Exercise

33. 35 yo G0 BMI 40 recently married with long h/o oligoovulation. EMB shows CAH. Propose next step: • Observation, repeat bx in 3 mos • Repeat emb or D and C now • Ultrasound • Medical mgmt: progestin Rx and eval for PCOS • Gyn Onc consult/path review • hysterectomy

34. To use for future conference planning, Palm Beach is a great location for a conference because: • Accessible location • Pleasant weather • Luxurious accomodations • Various recreational activities

35. UNC Experience • Questions should be similar difficulty as those on examinations • Students liked ARS especially if questions provided hints to examination materials • Participation was about 60-70%, not the 100% we anticipated

36. Best Practices • Planning • Be sure to have tech support • Decide how ARS will or will not affect student grades • Define clear objectives for use of ARS in your classroom and write questions with these objectives in mind • First year will take a little more time and effort • Practice when first starting to use ARS

37. Best Practices • Student communication • If used for grading, be sure to have rules about use (i.e. no use of other’s clickers) • Consider throwing out some grades for inadvertent clicking • Consider making it worth 5-10% of total grade (i.e. low stakes, but not too low) • Plan for time to debrief after each question

38. ARS Options • Different technologies • IR transmitters • RF transmitters • Web-based • Smartphone • Presentation software integration • LMS integration

39. ARS Options Turning Point iClicker Interwrite

40. Acknowledgements • Special thanks to: Jake Achey Instructional Technology Technician UNC School of Medicine Medical Students Teaching Lab

41. References • 1. BARBER M, NJUS D. Clicker evolution: seeking intelligent design. CBE Life Sci Educ 2007;6:1-8. • 2. CALDWELL JE. Clickers in the large classroom: current research and best-practice tips. CBE Life Sci Educ 2007;6:9-20. • 3. COLLINS J. Audience response systems: technology to engage learners. J Am Coll Radiol 2008;5:993-1000. • 4. COLLINS LJ. Livening up the classroom: using audience response systems to promote active learning. Med Ref Serv Q 2007;26:81-8. • 5. NAYAK L, ERINJERI JP. Audience response systems in medical student education benefit learners and presenters. Acad Radiol 2008;15:383-9. • 6. PRADHAN A, SPARANO D, ANANTH CV. The influence of an audience response system on knowledge retention: an application to resident education. Am J Obstet Gynecol 2005;193:1827-30. • 7. TURPIN DL. Enhance learning with an audience response system. Am J Orthod Dentofacial Orthop 2003;124:607

42. Session content, including narrated MS Powerpoint slides available at: http://www.obgynknowledgebank.net