Bilateral Teleoperation of Multiple Cooperative Robots over

1. Bilateral Teleoperation of Multiple Cooperative Robots over Delayed Communication Network: Theory Dongjun Lee Mark W. Spong d-lee@control.csl.uiuc.edu, mspong@uiuc.edu Research partially supported by the Office of Naval Research (N00014-02-1-0011 and N00014-05-1-0186), the National Science Foundation (IIS 02-33314 and CCR 02-09202), and the College of Engineering at the University of Illinois.

2. Outline 1. Motivations 2. Problem Formulation 3. Passive Decomposition of Slave Robots 4. Control Design 5. Conclusions Part II: Simulation and Semi-Experiment

3. Motivations Applications: 1. Space Structure Construction/Maintenance - Hubble telescopes, International Space Station,… 2. Remote Construction/Maintenance of Civil Structures - Bridge, Highway, Tall buildings,… 3. Operations in Hazardous Environments - Nuclear plants, Deep water, … Bilateral Teleoperation of Multiple Cooperative Robots Bilateral Teleoperation - Human’s intelligent intervention in uncertain environments Multi-Robot Cooperation - Mechanical strength and dexterity - Robustness and safety

4. 1. Abstraction - human is able to operate only small DOF simultaneously 2. Secure grasping - no dropping of the grasped object 3. Haptic feedback - crucial for manipulation tasks 4. Interaction safety and stability - stably coupled with humans, objects, and environments Challenges and Requirements

5. Outline 1. Motivations 2. Problem Formulation 3. Passive Decomposition of Slave Robots 4. Control Design 5. Conclusions

6. inertia human force Stack-up n-DOF product system (n=n1+n2+…+nN-dimensional) Dynamics of Master and Multiple Slave Robots Dynamics of a single master (m-DOF) Coriolis velocity control Dynamics of multiple slave robots (n1+n2+…+nN-DOF)

7. master’s DOF q1 q3 q2 m-dim. level sets Grasping shape control objective desired (constant) grasping shape Grasping Shape Function: Holonomic Constraints - m-dim. holonomic constraints on the config. space of slave robots (m < n) - assumed to address the internal formation shape for cooperative grasping - smooth and full-rank Jacobian (i.e. smooth submersion) - overall group motion evolving on m-dim. level sets (submanifold)

8. - C&C delay between the master and the slaves - Centralized C&C module for multiple slaves - negligible delays among the slaves - workspaces of slaves are close to each other (e.g. cooperative grasping) Communication and Control (C&C) Structure

9. Semi-Autonomous Teleoperation Architecture Observation: - secure grasping is of foremost importance for safety - the system cannot be completely free from time-delay, i.e. system performance would be compromised in some aspects Semi-autonomous teleoperation: 1. local grasping control - secure grasping immune to communication-delay - autonomous control would be enough due to simplicity of cooperative grasping control objective 2. delayed bilateral teleoperation - communication-delay effect confined in bilateral teleoperation - sluggish response could be taken care of by intelligent humans - delayed teleoperation is relatively well-studied areas

10. Energetic passivity master-port mechanical power total slave-ports mechanical power Energetic Passivity for Safe/Stable Interaction - passive with total master/slave mechanical power as supply rate - stable interaction with any E-passive humans[Hogan]/objects/environments

11. Outline 1. Motivations 2. Problem Formulation 3. Passive Decomposition of Slave Robots 4. Control Design 5. Conclusions

12. behavior of overall group (and grasped object) Locked System internal group coordination (cooperative grasping) Shape System - Can achieve tight/secure grasping regardless of overall group behavior - Ensure secure grasping and interaction stability simultaneously Passive Decomposition of Multiple Slaves Robots Coupling: dropping object!!! The Passive Decomposition [Lee&Li, CDC03] decouples the locked and shape systems from each other while enforcing passivity

13. locked system velocity vL shape system velocity vE Orthogonal Decomposition w.r.t. Inertia Metric Grasping shape function Locked system velocityvL : parallel w.r.t. the level sets of qE: (behavior of grasped object and total group) Shape system velocityvE : orthogonal complement w.r.t. inertia matrix (cooperative grasping) Tangent space decomposition basis of kernel of qE basis of orthogonal space

14. - Shape system ((n-m)-DOF) explicitly represents cooperative grasping shape qE(q) - Locked (m-DOF) system describes overall group behavior - Locked and shape dynamics are similar to usual mechanical systems: - ML(q), ME(q): symmetric and positive-definite - ML(q)-2CL(q,q), ME(q)-2CE(q,q) : skew-symmetric - Coupling is energetically conservative: Passive Decoupling - CLE(q,q) =-CELT(q,q) -> vLTCLE(q,q)qE + qETCELT(q,q)vL=0 - Power and kinetic energy are also decomposed Passive Decomposition of Slave Group Dynamics Original Slave Dynamics Passive Decomposition Decomposed Dynamics

15. Original System Decomposed System passive decoupling Energetic Structure of Decomposed Dynamics - We can decouple the shape system (cooperative grasping) and the locked system (overall group) from each other while enforcing passivity - Desired cooperative grasping and overall group behavior can be achieved simultaneously while enforcing interaction stability

16. Outline 1. Motivations 2. Problem Formulation 3. Passive Decomposition of Slave Robots 4. Control Design 5. Conclusions

17. Scattering-based teleoperation control for decoupled locked system Total Slave Control Local grasping control control for decoupled shape system Passive decoupling Local Grasping Control Grasping Dynamics (Decoupled Shape System) desired grasping shape internal force PD/FF-based Control estimate of internal force - Adjusting qEd, and PD-gains, fixtureless grasping can be achieved for flexible object - Although dynamics is decoupled, other effects (e.g. inertia of object) can still perturb the shape system via the internal force FE: feedforward cancellation is necessary Semi-Autonomous Control Decomposed Dynamics

18. Dynamics of Master Robot and Slave Locked System (both are m-DOF) human/combined external forces control Locked System (decoupled) Shape system (locally controlled) Scattering-Based Teleoperation of Locked System By operating the master robot of manageably small DOF, human operators can tele-control the behavior of the grasped object over the delayed master-slave communication channel while perceiving combined external forces acting on the grasped object and slaves

19. Scattering Variables (Power Decomposition) incident (to comm.) reflected (from comm.) line impedance (user-specific) Impedance Control (PI-Control) Scattering-Based Symmetric Teleoperation Symmetric Scattering-Based Teleoperation: - scattering communication (to passify comm. delays) and impedance (PI) controls - asymptotic position-error convergence proof with Z=Kv (i.e. matching condition [Stramigioli et al, TRA03]) : so far, only boundedness of position-error has been established. - force reflection in static manipulation (negligible acceleration/velocity)

20. Conclusions We propose a control framework for bilateral teleoperation of multiple cooperative robots over delayed master-slave comm. channel: - passive decomposition: the decoupled shape (cooperative grasping) and locked (behavior of the grasped object) systems - local grasping control for the shape system: high precision cooperative grasping regardless of human command/comm. delays - scattering-based bilateral teleoperation of the locked system: human can tele-control behavior of the cooperatively grasped object by operating a small-DOF of the master robot, while perceiving combined force on the slaves and the grasped object over the delayed communication channel - enforce energetic passivity: interaction safety and stability are enhanced Part II will present simulation and semi-experiment results.