420 likes | 430 Views
Dmitriy Babichenko, Jonathan Velez, University of Pittsburgh This presentation was given at the 2017 Serious Play Conference, hosted by the George Mason University - Virginia Serious Play Institute. ScopingSim is an interactive alternate-controller-based serious game that uses off-the-shelf open-source components that can be plugged into virtually any computer and is designed to leverage engaging gaming elements to motivate learners to practice both mechanical and diagnostic aspects of scoping procedures. This presentation will address a number of challenges that we had to overcome in order to develop a useful working prototype, including collecting requirements, underestimating costs, dealing with student developers and continuity of support, setting up experiments to identify models of expertise and feedback mechanisms, and making decisions on whether or not to use VR technologies. Physical medical simulators (mannequins) are widely used for training medical students and medical personnel to perform specialized procedures, hone diagnostic techniques, and improve clinical decision-making skills in critical situations. Such mannequin simulators, however, are often extremely expensive, require development of complex teaching scenarios, support of technical staff, and presence of a clinical expert for debriefing and feedback. To address these issues we began to develop ScopingSim - an interactive alternate-controller-based serious game that uses off-the-shelf open-source components, can be plugged into virtually any computer, and leverages engaging gaming elements to motivate learners to practice both mechanical and diagnostic aspects of scoping procedures. This presentation will address a number of challenges that we had to overcome in order to develop a useful working prototype, including collecting requirements, underestimating costs, dealing with student developers and continuity of support, setting up experiments to identify models of expertise and feedback mechanisms, and making decisions on whether or not to use VR technologies.
E N D
To Scope or Not To Scope Challenges of Gamifying Clinical Procedures Training
Who are we? Dmitriy Babichenko Jonathan Velez Kailani Bailey William O’Toole Ravi Patel ● ● ● ● ●
Acknowledgements While we take full responsibility for any errors and shortcomings of this presentation, we would like to thank the following people for their constant support of this project, for offering their medical, educational, and gamification expertise, and for providing indispensable advice on all aspects of design and implementation. Lorin Grieve, PharmD, Instructor, Pharmacy and Therapeutics, University of Pittsburgh School of Pharmacy John Lutz, Director of Information Technology and Co-Director of Research, Peter M. Winter Institute for Simulation, Education, and Research (WISER) Phillip Lamberty, MD, Director, Medical Intensive Care Unit UPMC Presbyterian Hospital, Director, Pulmonary and Critical Care Ultrasonography, Medical Director, Select Specialty Hospital Pittsburgh Deborah Farkas, PhD, Director of Educational Development, Peter M. Winter Institute for Simulation, Education, and Research (WISER) Timothy Meehan, Student, University of Pittsburgh School of Information Sciences Taylor Winn, Student, iSchool Inclusion Institute (I3) Morgan Freeman, Student, iSchool Inclusion Institute (I3) Michael Depew, Director, iSchool Inclusion Institute (I3) ● ● ● ● ● ● ● ●
Challenges (should you choose to accept them) 1. Structure an in-house game or simulation development project 2. Connect learning objects to system requirements, game mechanics, and technologies 3. Establish a framework that facilitates learner evaluation
What is bronchoscopy? (Medline - https://medlineplus.gov/ency/images/ency/fullsize/23232.jpg)
Motivation #1 - Cost Medical simulators & task trainers are EXPENSIVE (and creepy) 1. 2. 3. High cost of purchase High cost of maintenance Low simulator to student ratio
Motivation #2 - Feedback High-end simulators & task trainers provide limited feedback based on anatomy & physiology ● Difficult to determine whether a learner performed well because of skill / knowledge or just luck ● No user model / learner model ●
Motivation #3 - Debriefing Students cannot practice on their own without supervision of domain expert ● Domain experts’ time is expensive ● Low domain expert to student ratio ● Without debriefing complex procedure simulations are virtually pointless ● Image source: https://cphp.org/critical-incident-debriefing-services/
Motivation #4 - Generalizability ● Common procedure(s) - bronchoscopy, endoscopy, colonoscopy ● Good generalizability ● Number of interested stakeholders Laerdal SimMan (http://www.laerdal.com/us/doc/86/SimMan)
Let’s Roleplay Education/Game Expert IT Expert Medical Expert
Mission Impossible I: Define Requirements
What did the stakeholders want? ● Something that doesn’t require an expensive mannikin & easily distributable ● Provides real-time feedback & just-in-time learning ● Debriefs learner without the help of a domain expert
What did the stakeholders want? (Learning Goals) ● Something that doesn’t require an expensive mannikin & easily distributable Deliberate Practice ● Provides real-time feedback & just-in-time learning - Reinforcement Learning ● Debriefs learner without the help of a domain expert - Reflective Learning
Organize Your Ideas Mission Impossible II
Affinity Diagram Organizes a large number of ideas into their natural relationships Taps a team’s creativity and intuition When to use ○ You are confronted with many facts or ideas in apparent chaos ○ Issues seem too large and complex to grasp ○ Group consensus is necessary ● ● ● http://asq.org/learn-about-quality/idea-creation-tools/overview/affinity.html
Affinity Diagram Procedure Look for ideas that seem to be related in some way. Place them side by side. ● Form notes into “ idea” clusters ● Discuss any patterns, especially reasons for moving controversial notes, and potentially split or merge clusters ● Repeat ● http://asq.org/learn-about-quality/idea-creation-tools/overview/affinity.html
Affinity Diagram Procedure Now it’s your turn http://asq.org/learn-about-quality/idea-creation-tools/overview/affinity.html
Mission Impossible III: Priorities.. Priorities..
Make the Hard Decisions Recognize that each idea in the pool may be competing ideas ○ How will committing to one idea influence the feasibility of other ideas in its cluster? ○ How will committing to one idea influence the feasibility of ideas in other clusters? Analysis & Design ○ Select and prioritize requirements ○ Requirements should support learning objectives ○ Be agile..! Requirements and feasibility can change at any moment ● ●
ScopingSim with Scope Video Version 1 Setareh Sarachi, Pavitraa, Faris Obaid Alotibi, Abhishek Mukherjee, Dimple Varma
ScopingSim with Scope Video Version 2 Siying Zhang, Yue Su, Hanwei Cheng, Zihan Xie
Flex Sensor Ambu Scope Arduino Uno Accelerometer / Gyroscope
Mission Impossible Ghost Protocol: User Modeling
Building the User Model We have no idea what that might look like at this point...
Mission Impossible - Let’s talk about models How do we define performance? ● How do we classify “expertise”? ● When and how do we give feedback? ●
How do we define performance? User model constrained by system design Raw measures: ● x, y, z acceleration Time Spatial position in virtual environment Completed objectives Number of failures ... ● ● ● ● ● ● System design constrained by user model ●
How do we classify “expertise” ? Ask the experts! ○ How is expertise determined in the domain? ○ Pedagogies for knowledge/skill acquisition Qualify expertise from quantified performance ○ Heuristics vs Analytics Choose your pedagogy: ● Fitt and Posner’s Theory of Motor Acquisition (1967) ○ Cognitive → Integrative → Autonomous ● ● Parameterize your raw data: Changes in direction (avg and std dev) Time in motion (avg and std dev) Time at rest (avg and std dev) Ratio of time at rest to time in motion Velocity of motion (avg and std dev) Accomplishment of game objectives … ● ● ● ● ● ● ●
When and how do we give feedback? Feedback elements constrained by system design ○ Environmental cues (e.g., visual, audio, haptic) ○ Implementation examples: mini-map, flash or buzz on environment collision events, servomotor resistance Scaffold learning based on the model of expertise ○ Suggestion mechanisms (helpful or distracting?) ○ Implementation examples: next-step model Contextualize deviations in performance to encourage reflective learning ○ Facilitate debriefing; ask “how” and “why” ○ Detect where learners’ performance can improve and provide enough info for open-ended self-evaluation ● ● ●
Mission Impossible: Where are we today?
Dmitriy Babichenko - dmb72@pitt.edu To Scope or Not To Scope Thank you! Questions/Answers?
Dmitriy Babichenko - dmb72@pitt.edu To Scope or Not To Scope Thank you! Questions/Answers? The Floor is Failure
Dmitriy Babichenko - dmb72@pitt.edu To Scope or Not To Scope Thank you! Questions/Answers?