Cognitive Issues in Virtual Reality

1. Cognitive Issues in Virtual Reality Wickens, C.D., and Baker, P., Cognitive issues in virtual environments, in Virtual Environments and Advanced Interface Design, Barfield and Furness, pp. 514-541. Summarized by Geb Thomas

2. The Main (Cognitive ) Features of VR • 3D viewing vs. 2D viewing • Dynamic vs. static displays • Closed-loop vs. open-loop interaction • Inside-out vs. outside in frame of reference • Multimodal interaction

3. A Comparison

4. Uses of Virtual Reality • What cognitive issues lie behind each application • How do these play into the user’s perceptual strengths and weaknesses

5. On-Line Performance • Direct manipulation capabilities in a remote environment • Principle issues: • Closed-loop perceptual motor performance • Situation awareness • Low workload and cognitive effort

6. Off-line Training and Rehearsal • Lumbar injection • Maneuvering a space craft • Rehearsing a dangerous mission • Key consideration: • Effective transfer of training

7. On-line comprehension • Reaching understanding, comprehension or insight • Scientist interacting with a database • Key: • perceiving relations • perceiving constraints • perceiving constancies

8. Off-line Knowledge Acquisition • Useful for education • Key issue: • knowledge transfer

9. How to Cognitively Engineer VR • Select features that aid and do not disrupt cognitive needs • closed-loop performance may not help on-line performance but is key to understanding • Different parts of the visual system are involved in perceptual-motor coordination and navigation than are involved with perceptual understanding of spatial location • Ambient vs. focal vision

10. The Operator Brings • Wide sensory bandwidth • Limited perceptual bandwidth • Constraints on attention • Constraints on working memory • High level of natural perceptual-motor coordination • Large repertoire of facts and knowledge

11. Search • Find an object of interest • Object of the search may be concrete instance or an abstraction of the rendering in VE • A map often facilitates searching • Minimize map clutter • Flexible frame of reference • Tie or link map and VE

12. Navigation • Challenging because of the removal of constraints • Speed and flexibility can cause loss of situational awareness • May help to partially automate movement, such as logarithmically control speed • Metaphors matter • flight in mazes • “in-hand” for objects

13. Perceptual Biases • Gibson and ecological perception, regularly spaced texture, level surfaces for gradients, slant and optical flow. • Ellis, McGreevy et al. • Virtual space effect because of minification or magnification from FOV • 2D-3D effect perceived rotation of vectors towards viewing plane • Display enhancement

14. Visual Motor Coupling and Manipulation • Gain • Time delay • Control order • Target view decoupling • Field of view

15. Field of View • Wider provides greater situational awareness • Wider distorts perceived position • Wider provide better sense of motion • Wider can promote motion sickness

16. Perception and Inspection • For navigation, spatial relations predominate • For inspection, light, shadow, motion parallax • Schematic figures for learning

17. Learning • Procedural learning • Lower realism • Perceptual motor skill learning • Active participation in control loop is important • Do dynamic simulation help? • Spatial Learning and Navigation Rehearsal • rotating frame can inhibit map building • Active control loop is not always a benefit • Head-mounted displays may be a hindrance • Conceptual learning • Multi-modal, active

18. HF Guidelines in Learning • Consistency • Redundancy • Visual Momentum • Consistent representations • Graceful transitions • Highlight anchors