3D Haptics and Robotics

1. 3D Haptics and Robotics Krasimir Kolarov Interval Research Corporation HCC Seminar, UC Berkeley, 12/3/98.

2. What is this talk about? • Introduction to Haptics and Haptic Interfaces • Commercial and University Haptics • Force Feedback Devices • 3D Haptics at Interval Research Corp.

3. “Haptic” - an information processing perceptual system that uses inputs from the receptors embedded in the skin, as well as in muscles, tendons and joints (Loomis and Lederman, 1986) “hap.tic (hap’tik) adj. of or having to do with the sense of touch; tactile” (Webster’s New World Dictionary) “haptic interfaces” - devices that measure the motion of, and stimulate the sensory capabilities within, our hands (as used in human interface technology ) Haptic Interfaces

4. Haptics relies on action to stimulate perception. The haptic system can sense and act on the environment while vision and audition have purely sensory nature. Being able to touch, feel, and manipulate objects in an environment, in addition to seeing (and hearing) them, provides a sense of immersion in the environment that is otherwise not possible (Srinivasan, 1995) Unique Characteristics of Haptics

5. Actuators (electrical, hydraulic, pneumatic) Sensors Tactile Feedback Interfaces (sensors, texture, slip, surface temperature) Control of Haptic Interfaces (distributed computation, quality) Physical Modeling (collision detection, surface deformation, mechanical compliance, smoothness, physical construction) Other Topics

6. Enhancement of GUI’s (graphical user interfaces) - enable users to feel where the buttons on their programs are. Computer Games - engaging touch interactions, cost-sensitive market. Simulation for training humans - to perform tasks that require sensorimotor skills (surgery, training for naval personnel). Interaction with computer-generated 3D data - users of CAD/CAM, data visualization and other engineering and scientific applications. Medicine, Entertainment, Telerobotics. Applications

7. Tactile Sensory System - distinguish vibrations up to 1 KHz; detection threshold on smooth glass plate - 2 m high single dot, 0.06 m high grating Kinesthetic Sensory System - bandwidth 20-30 Hz; JND (just noticeable distance) - 2.5o for finger joint, 2o for wrist and elbow, 0.8o for the shoulder Motor System - human bandwidth for limb motions is between 1-10 Hz as a function of the mode of operation Active Touch with all three systems - stiffness > 25N/mm is needed for an object to be perceived as rigid Human Haptics System

8. to measure the position and contact forces (and time derivatives) of the user’s hand (or other body parts). to display contact forces and positions (or their spatial and temporal distributions) to the user. Alerting function - vibrations. Premise: The sense of touching simple shapes could be evoked by programming computers to control electromechanical master devices. We can build devices that give us a sense of feel when controlling remote actions with a high degree of dexterity. Functions of Haptic Interfaces

9. 1. Free motion, in which no physical contact is made with objects in the environment 2. Contact involving unbalanced resultant forces (like pressing an object with a finger pad) 3. Contact involving self-equilibrating forces (like squeezing an object in a pinch grasp) Additional consideration - we can touch, feel and manipulate the objects directly or with a tool Categories of Haptics Interfaces

10. Ground-Based Devices joysticks/ hand controllers Body-Based Devices exoskeletal devices flexible (gloves and suits worn by users) rigid links (jointed linkages affixed to users) Tactile Displays shape changers shape memory actuators pneumatic actuators microelectromechanical actuators vibrotactile electrotactile Currently Available Haptic Interfaces

11. History of Haptic Interfaces

12. MIT AI Lab Salisbury (design of high performance mechanisms and sensors) MIT Human-Machine Systems Lab Srinivasan (understand human haptics, enhance human-machine interaction) Sheridan (tactile and auditory substitution of force feedback for teleoperation) MIT Media Lab Margaret Minsky (tactile feedback from a graphics simulation, home haptics) Plesniak (haptics and holographic systems) Harvard University Rob Howe (tactile display of shape and vibrations) UNC Taylor, Fred Brooks (nanomanipulator force feedback, medical research) CMU Baraff, Vedula (force feedback in interactive dynamic simulation) Haptics Research in Universities

13. Stanford University ME Dept., Cutkosky (force feedback grasping, multi-finger manipulation) CS Dept., Khatib, Ruspini (haptics library, force control, dynamics) CCRMA, O’Modrian (grand piano simulation, haptics for the blind) UC Berkeley Canny (optimum stability of grasp, dynamic simulations) Northwestern University Ed Colgate (dynamically effects like mechanical impedance) Japan Iwata (6 dof stewart platform joystick Haptic Master, mechanical design) University of New Mexico, University of Virginia, University of Colorado, Rutgers University, Georgia Tech, McGill University, Naval Postgraduate School, University of Washington, Simon Fraser University, ... University Research (cont.)

14. SensAble Devices (PhanTom) Immersion Corp. (Impulse Engines, Joysticks) Cybernet Systems Corp. (CyberImpact Joystick, Steering Wheel, Flight Yoke) Microsoft (formerly - EXOS Inc. Power Stick, Surgical Simulator, SAFiRE) Boston Dynamics (Tangible Reality, Interactive Humans) VTT, Finland (virtual prototyping) Interval Research Corp., MERL, GE Corporate R&D, High Techsplanations Inc., Army, Navy, ... Industrial Research and Development

15. Immersion Corporation

16. Cybernet Systems Corp.

17. EXOS Inc.

18. SensAble Technologies Inc.

19. Video on Applications (SensAble) • before that • b&w slides from Web Page • color slides from Hasser’s report • after that color slides from VTT

20. Research Interests Develop a cooperative graphic and haptic interface that allows to manipulate and sculpture 3D objects more effectively

21. Goals and Assumptions • Provide a high level interface to haptic devices that: • Complements existing interactive graphic systems • Works robustly in multi-surface environments • Provide common framework to allow stable and safe haptic control. • Ability to perform tasks that are not possible with the current technology • The combined graphical and haptic interface to 3D objects will allow us much richer and powerful interaction.

22. Capable of displaying a large number of simple polygons at interactive rates (>20,000 polygons at 30Hz) Intersecting polygons & gaps common Topology seldom available (Polygon Soup) Gourand/Phong Shading & Texture Existing Graphic Systems

23. Questions • How can you support a powerful and general set of modeling 3D primitives. • Allow the haptic server to operate with a great amount of autonomy from the host computer and simulate a wide range of virtual environments • Explore issues like latency in manipulating large 3D data sets

24. Basis for Research • Test distance/collision calculation and dynamic simulation. • Building a library to support arbitrary complex rigid objects. • Allow the developer to specify constraints between the objects in the environment and control the motion of objects in the virtual world. • Model the contact forces caused by contact and collisions between the objects in the environment

25. Stanford Students Projects • The Virtual Xylophone. • The Haptic Roaches. • Haptic Exploration of rigid 3D objects.

26. before that - color slides on xylophone, roaches after that color slide on staircase slides from the HL talk goals bounding sphere covering bounding sphere hierarchy simulating smooth surfaces results Video on Roaches

27. How to display (and compute) elemental sensations such as impact, friction, softness, motion and constraint. How to involve more complicated interactions (the Phantom concentrates on forces at the fingertip or tool tip). Those include pressure distribution, temperature and high-frequency vibration. Issues

28. Penalty Based Haptic Systems Bounding Sphere Hierarchy Virtual Proxy Model Surface Properties Force Shading/Texturing HL Library with Application Programmers Interface similar to GL Solutions

29. Bounding Sphere Hierarchy

30. The Virtual Proxy

31. Virtual Proxy Description • A representative object that is constrained by obstacles in the environment • Proxy is reduced to a point (C-space). User definable size of proxy. • Constraint planes locally describe the range of potential proxy motion. • Proxy moves to locally minimize the distance to user’s position • Haptic device physically moves user to proxy’s position

32. Force Shading

33. Haptic System Implementation • HL library, syntax similar to GL • graphic Client/Haptic Server Model • Bounding Sphere Hierarchy - O(log n) growth • > 24,000 polygonal primitives on 200 MHz Pentium - Stiffness 1800 Newtons/meter - > 1000 Hz servo rate

34. Demonstrations

35. “Force and Touch Feedback for Virtual Reality”, Grigore Burdea, Rutgers Univ., 1996, John Wiley & Sons. “The PHANToM User’s Group Workshop Proceeding”, MIT September 1996, 1997, 1998 (published as MIT AI Lab Tech Reports). Haptics Home Page at Northwestern Univ.: http://haptic.mech.nwu.edu/ Ruspini, D., Kolarov, K. and Khatib, O. "The Haptic Display of Complex Graphical Environments", Computer Graphics Proceedings, Annual Conference Series, SIGGRAPH'97, Los Angeles, California, September 1997 Sample References