Teleoperation

1. Teleoperation Interfaces

2. Introduction • Interface between the operator and teleoperator! • Teleoperation interface is like any other HMI • H(mobile)RI = TI • Also a HCI

3. Introduction • mechanical manipulation – ”included” interface • Modern methods • Closed loop teleoperation – MMI • Supervisory control – HCI • Mixed interfaces

4. Interface types, Fong • Direct • Closed loop • Multimodal/multisensor • Multisensor/actuator control • Supervisory control • Intelligent telerobot • Novel • New ones

5. Interface - Direct • Closed loop control • Realtime operator decision making is needed • Operator controls with hand controllers (like onboard) • High bandwith low delay communication

6. Multimodal/multisensor • Complex robot in dynamic situation • Individual actuator control, graphical feedback, coordinated motion • Integrated display with combined sensor information

7. Supervisory control • Remarkable part of the control in the teleoperator end • Teleoperator is capable for more or less autonomous task execution • “task based teleoperation”

8. Control methods Direct Supervisory

9. Novel interfaces • ”novel” is relative • gestures • gazes • brainwaves • muscle movements • WEB interfaces • multimodal • supervisory

10. Telepresence • When sufficient amount of sensor information (vision, sound, force) is brought from the teleoperator site to the operator he or she feels physically present in the teleoperator site • Called also tele-existence • Important information is transferred and dangerous/noise is filtered

11. Virtual presence (reality) • Like tele-presence except the sensor infromation is generated artificially by computer(s) • Simulators • Games • Models

12. Augmented reality • Real information (usually image data) is mixed with additional virtual information • Numerical information, real-time models, etc.

13. Telepresence • Already camera monitor combination creates some level of presence • more sophisticated system is called for in order to call it telepresence • To provide a perfect telepresence, all human senses should be transmitted from the teleoperator site to the operator site • vision, hearing and sense are relatively easy • smell and taste are more complicated

14. Vision • Humans get 90% of their perception • “To see is to believe” • eyes are very complex opto-mechanical systems • FoV is (H)180 deg x (V)120 deg • Focused area only few degrees • Movements from whole area • Extremely difficult system to be imitated

15. Vision • Head tracking • Head following cameras (2-3 DoF) • HMD • => relatively good feeling of presence

16. Hearing • Human area 16 – 20000Hz • Important in telepresence • In case of mobile machine control the noise can be filtered and the important sounds transferred with reasonable volume

17. Touch • The most important human sense • Human touch sensors – mechanoreceptors – are activated by touch, i.e. by pressure on the tissues • Two basic classes • tactile information (“touch”) • kinesthetic information (“force”)

18. Tactile • referring to the sense of contact with the object, mediated by the responses of low-threshold mechanoreceptors innervating the skin (say, the finger pad) within and around the contact region

19. Kinesthetic • referring to the sense of position and motion of limbs along with the associated forces conveyed by the sensory receptors in the skin around the joints, joint capsules, tendons, and muscles, together with neural signals derived from motor commands

20. Force feedback (kinesthetic) • force generated by the teleoperator, usually a manipulator, is fed back to the operator in order to generate a real response in gripping and manipulation tasks • Also in virtual environments • Inbuilt in mechanical manipulators

21. Haptic feedback (tactile) • haptic feedback, the tactile skin sensors have the main role. • tactile sensing of the robot manipulator is fed back to the fingers of the operator • Other possibilities also

22. Vestibular sensors • inside the inner ear • angular acceleration and thus rotation • linear acceleration in the horizontal and vertical plane, i.e. to gravity • => position and movements of the head to be detected • Important in dynamic driving tasks

23. Vestibular feedback • not usually used in teleoperation • not needed and expensive to implement • usually in simulators to create presence • If vision and vestibular sensors mismatch => simulator sickness (=seasickness)

24. Simulator Sickness • similar to motion sickness • difference is that SS can occur without any actual motion of the operator • Symptoms: apathy, general discomfort, headache, stomach awareness, nausea, etc. • encountered especially when HMD type displays are used

25. Simulator Sickness • The most typical reason of SS is the cue conflict • In cue conflict different nerves get different information from the environment • Typically conflict between visual and vestibular inputs • especially when HMD is used and the time lags in vision and control Moving teloperator + =>

26. What is Presence Overview

27. Traditional explanation • Based on tele(virtual)-presence • Sheridan’s definition of telepresence

28. Traditional explanation • Extent of sensory information has a much greater impact than the other two combined. These three factors however cannot describe presence alone. Task variables, such as task difficulty and degree of automation, also are important to presence.

29. Is presence only sensor information? • It’s very easy to be unpresent in a boring situation – like lectures • What about dreaming while sleeping or burying one’s head a book • Presence can be very strong without any sensor information  ”a self generated Virtual Reality” • A book – a film from same book • Maximum effect with hallucinogens

30. 3 modes of presence • Really present, perceiving the existing environment • Tele or virtual presence  presence is transferred/generated by ”cheating” human senses • ”Mental” presence, presence is generated in the mind without (direct) sensor information