1 / 22

A Mixed Reality System for Enabling Collocated After Action Review

A Mixed Reality System for Enabling Collocated After Action Review. John Quarles Samsun Lampotang Ira Fischler Paul Fishwick Benjamin Lok. Mixed Reality for Training. Training + Mixed Reality(MR) = Improved Training i.e Military [x], industrial[x], medical [x]

ohanzee-ojeda
Download Presentation

A Mixed Reality System for Enabling Collocated After Action Review

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Mixed Reality System for Enabling Collocated After Action Review John Quarles Samsun Lampotang Ira Fischler Paul Fishwick Benjamin Lok

  2. Mixed Reality for Training • Training + Mixed Reality(MR) = Improved Training • i.e Military [x], industrial[x], medical [x] • MR is used during the training session • After Action Review (AAR) • Playback and review of one’s training session • i.e. AAR in the HPS • Currently: video reviews • helpful for bringing trainees to competency • AAR + MR = ?

  3. After Action Review • Students need: • Repetition, directed instruction, Feedback • Educators need • Assessment Tools • Video Based Review offers: • Assessment Tools and Repetition: • video playback (FF,REW,etc) • instruction and feedback: • Expert must be present during student review

  4. Video Based AAR • Widely used • Limitations • Fixed, exocentric viewpoint • Minimal interaction (Play, Pause, etc) • See only what the camera sees (i.e occlusions are problematic) • Lost information

  5. MR Based AAR • Video review = lost information • Goal: maximize information flow with MR • Advantages of MR systems in general: • Egocentric viewpoint • interactivity – i.e. TUIs • Collocated virtual information (i.e. gaze) • Can we enhance AAR with MR?

  6. Innovations • The Augmented Anesthesia Machine Visualization and Interactive Debriefing System (AAMVID) • MR based AAR system for Anesthesia education • Educator version: AAMVID-E • Student version: AAMVID-S • User evaluation (19 students, 3 educators) • General goal: Evaluate MR for use in AAR

  7. Previous Work • AAR [x],[x],[x] • Recording Virtual Experiences • [x],[x],[x] • AAR of Virtual Human Experiences • IPSViz[x]

  8. Anesthesia Education and MR • The Augmented Anesthesia Machine [x] • merges and abstract simulation of an anesthesia machine with a real machine (magic lens + TUI) • Improves learning

  9. AAR in Anesthesia Education • Currently Video Based • Students perform a test (i.e. a fault test) • Educators and students review and critique performance

  10. AAR in Anesthesia Education • MR system: AAMVID • Based on the AAM • Students (AAMVID-S): • View their own or an expert’s interaction in situ • Follow a directed tutorial guided by the expert’s interaction visualization • Educators (AAMVID-E): • Can visualize and filter aggregate student data • AAMVID addresses video review limitations

  11. Student AAMVID-S System playback controls Gaze visualization interaction visualization

  12. Student AAMVID-S System • 3 Visualization Modes • User Review • View what the student did • Expert Review • View what the expert did • Expert Tutorial • Collocates: • expert’s gaze and interaction event boxes • Real time abstract simulation of the student’s machine • Interaction with the real machine while mimicking expert • Hands-on experience in correcting the fault

  13. AAMVID-S Evaluation • Usability and performance evaluation of AAMVID-S • 19 psychology students: Day 2 (90 min) 1.Testing2. AAR of Tests with AAMVID Day 1 (90 min) Training with AAM

  14. Day 2: Testing • Three Fault Tests • Experimenter caused a problem in the machine • i.e. disabled a the flow of nitrous oxide • Participant had to decide • Whether or not a fault was present • What the fault was (if present) • How to correct the fault (if present)

  15. Metrics • Repeated Measures design: • Performance Measures: • Participants were asked questions (each scored 0-4) • Fault absent or present • What was the fault? • How does the fault affect the patient? • How to correct the fault? • Confidence Measures: • Self reported confidence ratings for above questions (0-4)

  16. Results • Insert graphs that show significant improvements in performance and confidence.

  17. Discussion • Main results: • AAMVID Improved machine fault understanding • Performance question answers improved. • AAMVID increased confidence in fault correction ability • Confidence scores increased • Users prefer Expert Tutorial Mode

  18. Educator AAMVID-E System Interactive Filtering(i.e. on spatial cog data) Aggregate Gaze Heat Mapping

  19. Educator AAMVID-E System Interactive Filtering(i.e. on spatial cog data) Interaction Graph(markov model of class interaction)

  20. Informal Evaluation • 3 Experts in anesthesia education • Dr. Samsun Lampotang • David Lizdas • Dr. Nicholas Gravenstein

  21. Observations • Magic lens vs desktop? • What worked and what didn’t work

  22. Conclusions • Presented AAMVID • Improved student understanding and confidence • Offered needed assessment abilities to educators that were not available before • MR based AAR • can offer visualizations and interactivity that video based AAR cannot

More Related