Immersive Virtual Characters for Educating Medical Communication Skills

1. Immersive Virtual Characters for Educating Medical Communication Skills T. Bernard, C. Oxendine, D. S. Lind, P. Wagner Dept of Surgical Oncology (Medical College of Georgia) K. Johnsen, A. Raij, R. Dickerson, R. Wells, B. Lok Dept of Comp and Info Science and Eng (College of Eng) M. Cohen, A. Stevens Dept of Surgery (College of Med) J. Cendan, M. Duerson, R. Pauly Dept of Community Health and Family Med (College of Med) R. Ferdig College of Education

2. Previous Work: Technology for Medical Communication Skills • Traditional approaches [Bearman 2003] • Narrative • Problem Solving • Human Patient Simulator [Meurs 1997] • Mechanical simulation • Motors and actuators for I/O • JUST VR [Ponder 2002] • Immersive approach (Stereo Projection) • Trains students to react to emergency situations • Uses human instructor to control system • Virtual Standardized Patient [Hubal 2000] • Commercial Desktop application by RTI • Speech Recognition • Natural Language Engine

3. Project Description • Simulate a standardized patient encounter • Allow repeated interaction with an Immersive, interactive virtual patient in a constrained scenario • Virtual patient, DIANA • Virtual instructor, VIC • To address issues with SPs • Experience diversity • Quality Control • Feedback • Communication skills • No physical diagnosis • Interpersonal Simulator

4. Play Video • Things to look for: interaction modalities

5. System • Low Cost • < $8,000(USD) • COTSComponents • Potential: • Every Hospital

6. Natural Interaction Input • No Keyboard, No Mouse • Speech Recognition • Dragon Naturally Speaking 8 Pro • Accuracy 90+% with 10 minutes training • 70% match to database • Track Communication Cues • Non-Verbal • Track head gaze • Track left hand • Track body lean • Verbal • Inflection • Jargon • Gesture Recognition • Pointing, handshake

7. Natural Interaction Output • DIANA and VIC look at user • Life-size characters • Animation • Hand gestures • Head movement • Perspective-Correct Rendering • Why this works • Does not rely on complete sentences • Constrained scenario • Students trained on specific questions

8. Eight Studies • 2004 • April: Project initiated • August: Prototype (n=7) UF • October: Experts (n=3) UF • December: Pilot Test (n=10) UF • 2005 • June: Two Institutions (n=16) UF/MCG • July: VP vs SP (n=16, n=8) UF/MCG • October: Cultural Bias (n=16) MCG • October: Class Integration (n=33) UF • n = 101 • Testing Centers • Harrell Center at UF • Medical College of Georgia • Diana was in Exam Room #3 (video)

9. Eight Studies

10. VP ≈? SP • How is experiencing an interpersonal scenario with a virtual person similar to – and different from – experiencing an interpersonal scenario with a real person? • Clearly different • But… in what important ways? • Patient-doctor interview provides a constrained scenario • The study asks: • Are post-encounter impressions similar? • Are empathy and other emotions and attitudes similarly expressed? • Which social constructs are followed? • These questions must be explored to: • Determine the extent to which interpersonal scenarios can be simulated with virtual humans • Identify how component technologies need to improve to enable effective interpersonal virtual human systems

11. Overall Performance • Similar Experiences • Same % of participants asked key questions • Out of 12: 6.3 ± 1.7, 5.5 ± 2.1 (α = 0.37) • Same % of participants passed the scenario • SP = 50%, VP = 36% • But… • Some critical items were not asked at the same frequency • Sexual activity (SP = 0.88, VP = 0.44) • Nausea (SP=0.88, VP=0.25) • Related to virtual patient expressiveness

12. Behavioral Measures • Empathy • Empathetic moment – “I’m scared, can you help me?” • Expressed the same % and same # of times (SP = 2.2, VP = 1.3) • But… • Appears less genuine (very robotic) • Conversation flow is “rapid-fire” • Confirmatory phrases statistically different (SP = 20, VP = 3.5)

13. Lessons Learned • Overall experiences similar • Questions asked • Global measures • Education goals met • Students rated educational merits similarly • Students rated difficulty similarly • Global measures of realism do not work • Battery of specific measures more accurate • Practice tool in addition to SPs • Diana rated a 6.36(.85) on a 1 to 10 scale (Average is a 7.47 for real SP’s) • VR works out of the lab

14. Papers • Computer Science • Raij,et al., “Interpersonal Scenarios: Virtual≈Real?”, IEEE Virtual Reality 2006 • Johnsen, et al., “Evolving an Immersive Medical Communication Skills Trainer”, Journal on Presence: Teleoperators and Virtual Environments • Dickerson, et al., “Virtual Patients: Assessment of Synthesized Versus Recorded Speech”, Medicine Meets Virtual Reality 14 • Johnsen, et al., “Experiences in Using Immersive Virtual Characters to Educate Medical Communication Skills”, IEEE Virtual Reality 2005 • Dickerson, et. al., “Evaluating a Script-Based Approach to Simulating Patient-Doctor Interaction”, SCS 2005 Int’l Conf on HCI Adv for Modeling and Sim • Medicine • Stevens, et al., “The Use of Virtual Patients to Teach Medical Students History Taking and Communication Skills“, American Journal of Surgery • Lind and Lok, “The Role of Virtual Patients in Medical Education: Teaching Tool Versus Technological Trend”, FOCUS on Surgical Education • Stevens, et al., “Implementing a Virtual Patient into the Medical School Curriculum at the University of Florida”, Southern Group on Education Affairs 2006 (pres.) • Cohen, et al., “Do Health Professions Students Respond Empathetically to a Virtual Patient?”, Southern Group on Education Affairs 2006 (pres.) • Bernard, et al. “A Multiinstitutional Pilot Study to Evaluate the Use of Virtual Patients to Teach Health Professions Students History-Taking and Communication Skills”, Society of Medical Simulation 2006 • Stevens, et al., “The Use of Virtual Patients to Teach Medical Communication Skills”, Assoc of Surg Educ 2005, Southern Group on Educational Affairs 2005 (pres.)

15. Current Work • Incorporation into the classroom • UF Essentials of Patient Care • n=34 students this year • n=135 next year • MCG Patient Communication • n=200 each year • Study • Real Speech vs. Synethsized Speech • Immersive vs. Non-immersive

16. Do people carry real world biases into the virtual world?

17. Do people carry real world biases into the virtual world?

18. The Use of a Virtual Scenario to Teach Communication Skills for Geriatric Patients

19. Patient Diversity • Measures • Eye gaze • Body Lean • Vocal Inflection • Time • Interruptions • Do students treat different virtual patients similarly?

20. Communication Skills • Fundamental to clinical practice. Patient & physician satisfaction. Patient understanding & adherence. Healthcare outcomes. Malpractice litigation. • Learned skill & experiential learning alone is insufficient.

24. Virtual Objective Structured Exam(VOSCE) NBME

25. UF 1st year Medical Students n = 135 MCG 1st year Medical Students n = 200 EPC courses (Semester 1) ECM courses (Semester 1) Virtual Patient Experience (Early 1st semester) EPC courses (Semesters 2-4) ECM courses (Semesters 2-4) Curriculum integration

26. Virtual Patient Teams T. Bernard, C. Oxendine, D. S. Lind, P. Wagner Dept of Surgical Oncology (Medical College of Georgia) K. Johnsen, A. Raij, R. Dickerson, R. Wells, B. Lok Dept of Comp and Info Science and Eng (College of Eng) M. Cohen, A. Stevens Dept of Surgery (College of Med) J. Cendan, M. Duerson, R. Pauly Dept of Community Health and Family Med (College of Med) R. Ferdig College of Education