Bilateral Teleoperation of Multiple Cooperative Robots over

1. Bilateral Teleoperation of Multiple Cooperative Robots over Delayed Communication Network: Application Dongjun Lee Mark W. Spong Oscar Martinez-Palafox d-lee@control.csl.uiuc.edu, {mspong,pomartin}@uiuc.edu Research partially supported by the Office of Naval Research (N00014-02-1-0011 and N00014-05-1-0186), the National Science Foundation (IIS 02-33314 and CCR 02-09202), and the College of Engineering at the University of Illinois.

2. Outline 1. Review of the Proposed Control Framework 2. Simulation Results 3. Semi-Experimental Results 4. Conclusions

3. Bilateral Teleoperation of Cooperative Multi-Robots Combine advantages of - bilateral teleoperation: human intervention in uncertain environments - multi-robot cooperation: mechanical strength/dexterity & robustness/safety - applications: remote construction/maintenance of space/under-water/civil structures in possibly hazardous environments

4. behavior of overall group (and grasped object) Locked System internal formation shape (cooperative grasping) Shape System Passive decoupling Semi-Autonomous Teleoperation Coupling: dropping object!!! - Passive Decomposition [Lee&Li, CDC03] decomposes slave dynamics into decoupled shape (formation shape) and locked (overall group motion) systems - Local grasping control of decoupled shape system: secure/tight grasping regardless of human command via delayed comm. Channel - Bilateral teleoperation of locked system: by operating the master robot of manageably small DOF, human can tele-control the behavior of the grasped object over the delayed comm. channel while perceiving external forces

5. inertia Dynamics of multiple slave robots (n1+n2+…+nN-DOF) Stack-up n-DOF product system (n=n1+n2+…+nN-dimensional) master’s DOF System Modelling and Grasping Shape Function Dynamics of a single master (m-DOF) velocity Coriolis control human force Grasping Shape Function: Rn→Rn-m grasping shape function describes internal group formation shape desired (constant) grasping shape

6. locked system shape system desired grasping shape Local Grasping Control FF cancellation of internal force: although dynamics is decoupled, other effects (e.g. object’s inertia) can still perturb the shape system through internal force FE Passive Decomposition and Local Grasping Control Decomposed Slave Dynamics Locked system: abstracts overall behavior of multiple slave robots and grasped object passive decoupling Shape system: describes internal group formation of slave robots (i.e. cooperative grasping)

7. Locked System Shape system (locally controlled) Scattering-Based Teleoperation of Locked System Dynamics of Master Robot and Slave Locked System (both are m-DOF) human/combined external forces control Scattering-Based Teleoperation of Locked system: - humans can tele-control the behavior of the grasped object over delayed comm. channel while perceiving external forces acting on the object and slaves - asymptotic position coordination/static force reflection

8. Outline 1. Review of the Proposed Control Framework 2. Simulation Results 3. Semi-Experimental Results 4. Conclusions

9. Three 3-DOF Slave Robots 3-DOF Master agent1 deformable object (no friction) (x,y)-translation yaw rotation agent3 agent2 Simulation Settings Delay 0.5s Delay 0.5s - grasping shape function is defined s.t. three slaves form an equilateral triangle (w/ side length L) whose rotation is specified by the heading of agent 2 - human operator can tele-control the position and rotation of the triangle by operating 3-DOF master robot (translation and yaw) - 10% identification errors for inertias of robots (nominal: m=1kg, I=1kgm2)

10. Simulation: Importance of Decoupling With Passive Decoupling Control Without Passive Decoupling Control - no grasped object (just motion coordination) w/ PD-based grasping control - without decoupling control, grasping shape (i.e. shape system) is perturbed by human command and overall group behavior - slight grasping shape distortion w/ decoupling is due to inertial uncertainty

11. Simulation: Heavy Object Fixtureless Manipulation Without Feedforward Cancellation of Internal Force With Feedforward Cancellation of Internal Force - even if dynamics is decoupled, inertial effect of object (w/ frictionless contact) perturbs cooperative grasping through the internal force FE - this perturbation can be cancelled out by feedforward cancellation of the internal force FE (or also by large enough PD-gains)

12. due to grasping shape deformation good load balance due to grasping rigidity Heavy Object Manipulation: Contact/Human Force - human can perceive the total inertias of the grasped object and the slave robots - human can also perceive sensation of grasping loss - better load-balancing is achieved w/ FF-cancellation of the internal force FE, as grasping shape becomes more rigid

13. Simulation: Force Reflection Three 3-DOF Slave Robots agent1 deformable object agent3 agent2 human force external force due to object’s deformation - external forcing (x-direction) on the grasped object is faithfully reflected to the human operator (i.e. haptic feedback) - load balancing among slaves is degraded as the grasped object is deformed in the rigidly-maintained grasping shape

14. Outline 1. Review of the Proposed Control Framework 2. Simulation Results 3. Semi-Experimental Results 4. Conclusions

15. 2-DOF Master Three 2-DOF Slave Robots agent2 deformable object PHANToM Desktop: constrained on plane (i.e. (x,y)-translation) agent3 agent1 Semi-Experiment Setting Delay 0.5s external force Delay 0.5s - three slave robots: 2-DOF point mass dynamics (only x,y translations) - Phantom Desktop is used as master with its workspace constrained on (x,y)-plane - Grasping shape function: : specifies rotation and shape of the triangle formed by the three slaves

16. human perceives inertias of object/slaves due to object deformation secure/precise grasping w/ FF-term Semi-Experiment: Deformable Object Manipulation - x-directional motion (full-range) w/ fixtureless grasping - grasping security is preserved regardless of human command - human can perceive the combined inertia of slaves and grasped object - increase of some slaves' contact force due to inertia/deformation of object

17. Semi-Experiment: Obstacle Perception human perceives external force due to object deformation Secure/precise grasping w/ FF-term - external force (x-direction) on the grasped object center - force generated by the PI-action in the local impedance controls - object’s deformation again leads in unbalanced load sharing among slaves

18. Conclusions We propose a control framework for bilateral teleoperation of multiple cooperative robots over delayed master-slave comm. channel: - passive decomposition: the decoupled shape (cooperative grasping) and locked (behavior of the grasped object) systems - local grasping control for the shape system: high precision cooperative grasping regardless of human command/comm. delays - scattering-based bilateral teleoperation of the locked system: human can tele-control behavior of the cooperatively grasped object by operating a small-DOF of the master robot, while perceiving combined force on the slaves and the grasped object over the delayed comm. channel - enforce energetic passivity: interaction safety and stability - Semi-experiment and simulation results are presented and validate efficacy of the proposed control framework Possible impacts on emerging or traditional applications: - remote construction/maintenance of space/under-water/civil structures in hostile/hazardous environments