VE Input Devices

1. VE Input Devices Doug Bowman Virginia Tech

2. Goals and Motivation • Provide practical introduction to the input devices used in VEs • Examine common and state of the art input devices • look for general trends • spark creativity • Advantages and disadvantages • Discuss how different input devices affect interface design

3. Input devices • Hardware that allows the user to communicate with the system • Input device vs. interaction technique • Single device can implement many ITs

4. ITs Human-computer interface User interface software Input devices System Software Output devices User

5. Human-VE interface Env. model Display(s) Simulation loop: -render -check for events -respond to events -iterate simulation -get new tracker data Tracking system Input device(s)

6. Input device characteristics • Degrees of Freedom (DOFs) & DOF composition (integral vs. separable) • Range of reported values: discrete/continuous/hybrid • User action required: active/passive/hybrid • Intended use: locator, valuator, choice, … • Frame of reference: relative vs. absolute • Properties sensed: position, motion, force, …

7. Practical classification system • Desktop devices • Keyboards, 2D mice and trackballs, pen-based tables, joysticks, 6DOF devices for the desktop • Tracking devices • 3D mice • Special-purpose devices • Direct human input

8. Desktop devices: keyboards • Chord keyboards1 • Arm-mounted keyboards2 • “Soft” keyboards (logical devices)

9. Desktop devices: 6-DOF devices • 6 DOFs without tracking • Often isometric • Exs: Fig. 4.4 SpaceBall 5000, SpaceMouse Plus, SpaceOrb

10. Motion Tracking • Critical characteristics • Range, latency, jitter (noise or instability), and accuracy • Different motion trackers • Magnetic • Mechanical • Acoustic • Inertial • Optical • Hybrid

11. Electromagnetic trackers • Exs: Polhemus Fastrak, Ascension Flock of Birds • Most common • Used with conventional monitors (for fishtank VR) Small workbench displays • Transmitter • Receiver(s) • Noisy • Affected by metal objects -> distort the magnetic field

12. Inertial trackers • Exs: Intersense IS-300, Intertrax2 • Less noise, lag • Only 3 DOFs (orientation)

13. Exs: Vicon, HiBall, ARToolkit Advantages accurate can capture a large volume allow for untethered tracking Disadvantages may require light emitting diodes(LEDs) image processing techniques occlusion problem Optical/vision-based trackers

14. Hybrid tracking • Ex: IS-600 / 900 • inertial (orient.) • acoustic (pos.) • additional complexity, cost

15. Tracking devices: eye tracking

16. Tracking devices: bend-sensing gloves • CyberGlove7, 5DT • Reports hand posture • Gesture: • single posture • series of postures • posture(s) + location or motion

17. Tracking devices: pinch gloves • Conductive cloth at fingertips • Any gesture of 2 to 10 fingers, plus combinations of gestures • > 115,000 gestures

18. Case study: Pinch Gloves • Pinch gloves are designed to be a combination device (add a position tracker) • Very little has been done with Pinch Gloves in VEs - usually 1 or 2 gestures for: • Object selection • Tool selection • Travel

19. Characteristics of Pinch Gloves • Relatively low cost • Very light • User’s hand becomes the device • User’s hand posture can change • Allow two-handed interaction • Huge number of possible gestures

20. Characteristics of Pinch Gloves II • Much more reliable than data gloves • Support eyes-off input • Can diminish “Heisenberg effect” • Support context-sensitive gesture interpretation

21. Pinch Gloves in SmartScene13 • Lots of two-handed gestures • Scale world • Rotate world • Travel by “grabbing the air” • Menu selection

22. Pinch Gloves for menus • TULIP system14 • ND hand selects menu, D hand selects item within menu • Limited to comfortable gestures • Visual feedback on virtual hands

23. Pinch Gloves for text input • Pinch Keyboard14 • Emulate QWERTY • Pinch finger to thumb to type letter under that finger • Move/rotate hands to change active letters • Visual feedback

24. 3D mice • Ring Mouse • Fly Mouse • Wand • Cubic Mouse • Dragonfly • …

25. Special-purpose devices: using conductive cloth • Virtual toolbelt • Used to select virtual tools • Good use of proprioceptive cues • Interaction slippers3 • Step on displayed options • Click heels to “go home”

26. Special-purpose devices: Painting Table4

27. Special-purpose devices: ShapeTape11

28. Human input: speech • Frees hands • Allows multimodal input • No special hardware • Specialized software • Issues: recognition, ambient noise, training, false positives, …

29. Human input: Bioelectric Control

30. Human input: Body Sensing Devices

31. More human input • Breathing device - OSMOSE • Brain-body actuated control • muscle movements • thoughts!

32. Treadmills Stationary cycles VMC / magic carpet Walking/flying simulations (use trackers) Locomotion devices

33. UNIPORT • First Locomotion Device For U.S. Army (1994) • Proof-of-concept demonstration • Developed in six weeks • Difficult to change direction of travel • Small motions such as side-stepping are impossible

34. Treadport • Developed in 1995 • Based on a standard treadmill with the user being monitored and constrained by mechanical attachment to the user’s waist • User actually walks or jogs instead of pedaling • Physical movement is constrained to one direction

35. Individual Soldier Mobility Simulator (Biport) • Most sophisticated locomotion device • Designed for the conduct of locomotion studies • Hydraulic-based locomotion driven w/ force sensors at the feet • Safeguards limited responsiveness • Too awkward to operate

36. Omni-Directional Treadmill15,16 • Most recently developed locomotion device for U.S. Army • Revolutionary device that enables bipedal locomotion in any direction of travel • Consists of two perpendicular treadmills • Two fundamental types of movement • User initiated movement • System initiated movement

37. Torus treadmill

38. ODT video

39. Virtual Motion Controller17 • Weight sensors in platform sense user’s position over platform • Step in direction to move that direction • Step further to go faster

40. Walking in place18,19 • Analyze tracker information from head, body, feet • Neural network (Slater) • GAITER project (Templeman) • Shown to be better than purely virtual movement, but worse than real walking20

41. Classification of locomotion devices/techniques

42. Input and output with a single device • Classic example - touch screen • LCD tablets or PDAs with pen-based input • Phantom haptic device • FEELEX haptic device21

43. PDA as ideal VE device?22 • Offers both input and output • Has on-board memory • Wireless communication • Portable, light, robust • Allows text / number input • Can be tracked to allow spatial input

44. Conclusions • When choosing a device, consider: • Cost • Generality • DOFs • Ergonomics / human factors • Typical scenarios of use • Output devices • Interaction techniques

45. Acknowledgments • Joe LaViola, Brown University, for slides and discussions • Ron Spencer, presentation on locomotion devices used by the Army

46. References • [1] Matias, E., MacKenzie, I., & Buxton, W. (1993). Half-QWERTY: A One-handed Keyboard Facilitating Skill Transfer from QWERTY. Proceedings of ACM INTERCHI, 88-94. • [2] Thomas, B., Tyerman, S., & Grimmer, K. (1998). Evaluation of Text Input Mechanisms for Wearable Computers. Virtual Reality: Research, Development, and Applications, 3, 187-199. • [3] LaViola, J., Acevedo, D., Keefe, D., & Zeleznik, R. (2001). Hands-Free Multi-Scale Navigation in Virtual Environments. Proceedings of ACM Symposium on Interactive 3D Graphics, Research Triangle Park, North Carolina, 9-15. • [4] Keefe, D., Feliz, D., Moscovich, T., Laidlaw, D., & LaViola, J. (2001). CavePainting: A Fully Immersive 3D Artistic Medium and Interactive Experience. Proceedings of ACM Symposium on Interactive 3D Graphics, Research Triangle Park, North Carolina, 85-93. • [5] Bowman, D., Wineman, J., Hodges, L., & Allison, D. (1998). Designing Animal Habitats Within an Immersive VE. IEEE Computer Graphics & Applications, 18(5), 9-13. • [6] Hinckley, K., Pausch, R., Goble, J., & Kassell, N. (1994). Passive Real-World Interface Props for Neurosurgical Visualization. Proceedings of CHI: Human Factors in Computing Systems, 452-458. • [7] Kessler, G., Hodges, L., & Walker, N. (1995). Evaluation of the CyberGlove(TM) as a Whole Hand Input Device. ACM Transactions on Computer-Human Interaction, 2(4), 263-283. • [8] LaViola, J., & Zeleznik, R. (1999). Flex and Pinch: A Case Study of Whole-Hand Input Design for Virtual Environment Interaction. Proceedings of the International Conference on Computer Graphics and Imaging, 221-225. • [9] Ware, C., & Jessome, D. (1988). Using the Bat: a Six-Dimensional Mouse for Object Placement. IEEE Computer Graphics and Applications, 8(6), 65-70. • [10] Zeleznik, R. C., Herndon, K. P., Robbins, D. C., Huang, N., Meyer, T., Parker, N., & Hughes, J. F. (1993). An Interactive 3D Toolkit for Constructing 3D Widgets. Proceedings of ACM SIGGRAPH, Anaheim, CA, USA, 81-84.

47. References (2) • [11] Balakrishnan, R., Fitzmaurice, G., Kurtenbach, G., & Singh, K. (1999). Exploring Interactive Curve and Surface Manipulation Using a Bend and Twist Sensitive Input Strip. Proceedings of the ACM Symposium on Interactive 3D Graphics, 111-118. • [12] Froehlich, B., & Plate, J. (2000). The Cubic Mouse: A New Device for Three-Dimensional Input. Proceedings of ACM CHI. • [13] Mapes, D., & Moshell, J. (1995). A Two-Handed Interface for Object Manipulation in Virtual Environments. Presence: Teleoperators and Virtual Environments, 4(4), 403-416. • [14] Bowman, D., Wingrave, C., Campbell, J., & Ly, V. (2001). Using Pinch Gloves for both Natural and Abstract Interaction Techniques in Virtual Environments. Proceedings of HCI International, New Orleans, Louisiana. • [15] Darken, R., Cockayne, W., & Carmein, D. (1997). The Omni-directional Treadmill: A Locomotion Device for Virtual Worlds. Proceedings of ACM Symposium on User Interface Software and Technology, 213-221. • [16] Iwata, H. (1999). Walking About Virtual Environments on an Infinite Floor. Proceedings of IEEE Virtual Reality, Houston, Texas, 286-293. • [17] Wells, M., Peterson, B., & Aten, J. (1996). The Virtual Motion Controller: A Sufficient-Motion Walking Simulator. Proceedings of IEEE Virtual Reality Annual International Symposium, 1-8. • [18] Slater, M., Usoh, M., & Steed, A. (1995). Taking Steps: The Influence of a Walking Technique on Presence in Virtual Reality. ACM Transactions on Computer-Human Interaction, 2(3), 201-219. • [19] Slater, M., Steed, A., & Usoh, M. (1995). The Virtual Treadmill: A Naturalistic Metaphor for Navigation in Immersive Virtual Environments, Virtual Environments '95: Selected Papers of the Eurographics Workshops (pp. 135-148). New York: SpringerWien. • [20] Usoh, M., Arthur, K., Whitton, M., Bastos, R., Steed, A., Slater, M., & Brooks, F. (1999). Walking > Walking-in-Place > Flying, in Virtual Environments. Proceedings of ACM SIGGRAPH, 359-364.

48. References (3) • [21] Iwata, H., Yano, H., Nakaizumi, F., & Kawamura, R. (2001). Project FEELEX: adding haptic surface to graphics. Proceedings of ACM SIGGRAPH, Los Angeles, 469-476. • [22] Watsen, K., Darken, R., & Capps, M. (1999). A Handheld Computer as an Interaction Device to a Virtual Environment. Proceedings of the Third Immersive Projection Technology Workshop. • [23] Zhai, S. (1998). User Performance in Relation to 3D Input Device Design. Computer Graphics, 32(4), 50-54.