Multi-Robot Teams Separating Teamwork from Taskwork

1. Multi-Robot TeamsSeparating Teamwork from Taskwork Gal A. KaminkaThe MAVERICK GroupComputer Science DepartmentBar Ilan University, Israel

2. Teams of Robots and Agents • Rescue, disaster management • Coordinated search • Mapping • Exploration • Space and science applications • Satellite formations • Cleaning • Vacuum cleaning • Surveillance, Security • Defense, civilian • Hazardous materials • Sensor, ad-hoc networks • Agriculture • Plowing • Medical applications • Hospital deliveries 10-15 years ago: Application-specific solutions

3. Teams of Robots and Agents Entertainment...

4. Then, 10-15 Years ago… • Philosophy, Computer Science Theory • Joint intentions (Cohen & Levesque 1990, 1991) • SharedPlans (Sidner & Grosz 1986, Grosz & Kraus 1996) • Systems needing automated collaboration • Robot foraging, coordinated movement (Parker 1993, 1998) • Simulated helicopter teams (Tambe 1996) Team = Teamwork + Taskwork

5. Teamwork and Taskwork Soccer Taskwork: • Kicking to a target • Turning, tackling • Tracking the ball, goal … Soccer Teamwork: • Allocating players to roles • Synchronizing tactics • Sharing relevant information • …….

7. Teamwork Research Task-Dependent Teamwork General Teamwork

8. MAVERICK Research in teamwork(See elsewhere for plan recognition and social cognition) Task-Specific Teamwork: • Foraging(AIJ 07, ICRA 10) • Coverage(ICRA 05,06,…) • Passing(FSS 01) • Scheduling(SSS 06) • Coordinated movement(AAAI 05,ICRA 06,JOPHA 09) • Patrolling (ICRA 07,AAMAS 08,IJCAI 09) • Multi-robot HRI (ICRA 06) • …. General Teamwork: • Monitoring(JAIR 00,02, JAAMAS, AAMAS 07,ICRA 10) • Fault diagnosis(AAAI 98, IJCAI 03, AAAI 05, AAAI 06) • Synchronization(AAAI 05, ICRA 07) • Allocation(AAMAS 03, AAAI 05, AAMAS 07) • Learning, adaptation (DARS 06) • Maintenance (AAMAS 07) • ….

9. MAVERICK Research in teamwork(See elsewhere for plan recognition and social cognition) Task-Specific Teamwork: • Foraging(AIJ 07, ICRA 10) • Coverage(ICRA 05,06,…) • Passing(FSS 01) • Scheduling(SSS 06) • Coordinated movement(AAAI 05,ICRA 06,JOPHA 09) • Patrolling(ICRA 07,AAMAS 08,IJCAI 09) • Multi-robot HRI (ICRA 06) • …. General Teamwork: • Monitoring(JAIR 00,02, JAAMAS, AAMAS 07,ICRA 10) • Fault diagnosis(AAAI 98, IJCAI 03, AAAI 05, AAAI 06) • Synchronization(AAAI 05, ICRA 07) • Allocation(AAMAS 03, AAAI 05, AAMAS 07) • Learning, adaptation (DARS 06) • Maintenance (AAMAS 07) • ….

10. Individual Movement in Formation

11. Move Formation (a) Position Move End Ready Leader Find Leader Lead Follow BDI robot control:A quick review • Each node is a behavior (a controller) • Nodes arranged hierarchically (to decompose tasks) • Also sequentially • Every behavior • Preconditions • End conditions A behavior graph (a)

12. Teamwork? • Each robot executing own copy of recipe • No agreement • Coordination, but nocollaboration

13. Move Formation (a) Position Move End Ready Leader Find Leader Lead Follow Principle #1:Synchronizing Plan Execution • Agents must achieve mutual belief with teammates • Sidner & Grosz & Kraus 1986,1996; Cohen& Levesque 1990 • Team behaviors: Start together, end together • Jennings 1995,Tambe 96

14. Principle #2:Allocate tasks to team-members Teammates assist each other, take over when failure: • Allocate tasks to members (Task Allocation) • e.g., Shehory & Kraus 1995, Dias et al. 2000. • Anticipate needs of teammates • Proactive Communications (Yen et al. 2001) • Dynamically re-adjust team to address failures • Parker 1998, Tambe 1996, 1997, Kumar et al. 2001

15. Principle #2:Allocate tasks to team-members Teammates assist each other, take over when failure: • Allocate tasks to members (Task Allocation) • e.g., Shehory & Kraus 1995, Dias et al. 2000. • Anticipate needs of teammates • Proactive Communications (Yen et al. 2001) • Dynamically re-adjust team to address failures • Parker 1998, Tambe 1996, 1997, Kumar et al. 2001

16. Move Formation (a) (b) {X,Y,Z} Position Walk End Ready Leader Find Leader Lead Follow {Y,Z} {X} Organization hierarchy, allocated tasks • Behavior Graph (a) • Team Hierarchy (b) (Tambe et al. 2000) • Agents can find out with whom to communicate • Agents also know who responsible for a behavior

17. Behavior graph offers opportunities for protocol use • Conditions trigger synchronous selection • Decomposition triggers allocation Synchronized Start Synchronized End Move Formation Position Walk End Allocation Ready Leader Find Leader Lead Follow Opportunities for interaction

18. BITE: Bar Ilan Teamwork Engine (IAS 04, AAAI 05, ICRA 07, AAMAS 07) • Interaction protocols (behaviors) first-class objects • Allow using different protocols at different times • Insead of e.g., always vote, always ask leader, … Context-dependent coordination policies

19. (c) Wait Tally Announce Vote Ack. Readiness CFP Bid Select Move Formation (a) (b) {X,Y,Z} Position Walk End Ready Leader Find Leader Lead Follow {Y,Z} {X} BITE Components • Behavior Graph (a) • Team Hierarchy (b) • Interaction Behaviors (c)

20. BITE Control Algorithm • Each robot has a copy of the three structures • Each robot executes the following control loop • push initial behavior onto execution stack • repeat • e  current top behavior on stack • if (e can be decomposed) push ALLOCATE(children(e)) • else break • Execute stack until a behavior b terminates • SYNC_TERM(b, children(b)) • e  SYNC_SELECT(b, followers(b)), push e • GOTO 2

21. BITE in Action

22. Lesson: Flexibility matters (I) 80000 3500 70000 70251 3000 2900 65487 65059 60000 58183 2500 54005 53588 52730 2257 50000 2000 1831 Task Time 40000 Protocol Time 1500 30000 1284 1000 20000 836 688 Social Time 500 10000 Task Time 0 0 0 1 2 3 4 5 6 7 Configuration: Same behaviors, different protocols

23. Patrolling Frequency-Based (w/ Elmaliach) Adversarial (w/ Kraus, Agmon) ICRA 07, AMAI 10 AAMAS 08 AAMAS 08, AAMAS 09 IAS 08, IJCAI 09 Perimeter Fence (Border) Area

24. Patrolling Frequency-Based (w/ Elmaliach) Adversarial (w/ Kraus, Agmon) ICRA 07, AMAI 10 AAMAS 08 AAMAS 08, AAMAS 09 IAS 08, IJCAI 09 Perimeter Fence (Border) Area

25. Multi-robot, single zone:Individual (unsynchronized) patrol e b a c d

26. Multi-robot, single zoneSynchronized patrol e b a c d Theorem: Synchronized patrol is better. (response time is at most half-segment.)

27. Multi robots, multi zones (overlap): • Internal points visited by multiple robots • Improved uniformity everywhere except for edges • Low frequency in edges • Important for system design, placement of cameras

28. Responding to break-ins, faults • Detected by the robots or operator • Affected robot is committed • Sends a message to others • Other re-allocate segments • Maintain frequency (now reduced) • Many issues in practice: • Intermittent minor stops vs. events • Noise in the actual velocity of the robots • Turning points, communication ranges

29. Multi-robot, single zone(Fence Break)

30. Frequency function(we know math!) Timep)l,r,p,v,o,s,n,t,d) =

31. Principle #3:Organizational Maintenance Teammates assist each other, take over when failure: • Allocate tasks to members (Task Allocation) • e.g., Shehory & Kraus 1995, Dias et al. 2000. • Dynamically re-adjust team to address failures • Parker 1998, Tambe 1996, 1997, Kumar et al. 2001 • Anticipate needs of teammates • Proactive Communications (Yen et al. 2001)

32. 2 Leader Target of 1 Target of 3 Target of 0 1 3 0 Task: Formation Maintenance Separation-Bearing Formation Control (SBC): • One robot assigned to lead the group (leader) • Others maintain distance and angle to peers (targets) • Each robot monitors single target • Monitoring rules imply a control graph • Leader node: Out degree of 0 • Other nodes: Out degree of 1 • Connected, path from all to leader • Proven stable (e.g., Fierro et al. 2001)

33. Diamond FormationPossible Control Graphs No

34. Generating an Organizational Structure:Control Graph Generation(AAAI 05, ICRA 06) Key idea: Generate a multi-graph containing all possible control-graphs Then, induce a single control graph Multi-graph: • Vertices are robots in formation positions • Directed edge (r1,r2) implies r1 can sense r2. • Multiple edges between a pair of vertices allowed • Correspond to multiple sensing options • According to sensing morphology of robot

35. Sensing Costs: Setting edge weights • A robot can have multiple sensors (edges) • Each sensor has range, field of view, pan angle • Cost associated with use to monitor target • Edge weights correspond to sensor usage costs • Induce minimum-cost organization: • Min weight path from each vertex to leader • Single target shortest paths (Dijkstra)

36. Re-organization: Dynamic Switching of Control Graphs • Called when robot (or more) loses its target • Frequent failures update costs • Recovery algorithm updates control graph Glick and Kaminka, ICRA 2006

37. Formation Results No Reorganization Reorganization

38. Time spent in formation Better Reorganization No Reorganization Significantly higher percent of time (two-tailed t-test, p < 2.40109e−15)

39. Time to complete task No Reorganization Reorganization Better Significantly shorter duration in all cases (two-tailed t-test, p < 5e-13)

40. BITE Revisited Common lessons • Synchronized execution is better for task • Balancing load among homogeneous robots • Improved robustness, bug-free development • 60% reduction in software engineering effort! Is this it?

41. MAVERICK Research Task-Specific Teamwork: • Foraging(ECAI 04) • Coverage (ICRA 05,06) • Passing(FSS 01) • Scheduling(SSS 06) • Coordinated movement(AAAI 05, ICRA 06) • Patrolling (ICRA 07, 08) • …. General Teamwork: • Monitoring(JAIR 00,02, JAAMAS, AAMAS 07) • Fault diagnosis(AAAI 98, IJCAI 03, AAAI 05, AAAI 06) • Synchronization(AAAI 05, ICRA 07) • Allocation(AAMAS 03, AAAI 05, AAMAS 07) • Learning, adaptation (DARS 06) • Maintenance(AAMAS 07) • ….

42. Multi-Robot Coverage:Can we separate teamwork from taskwork? • Goal: Visit each point in an area • Applications: Floor cleaning, mowing, de-mining, painting …. • Many approaches, distinguishing properties: • Off-line/on-line, heuristic/complete, approximate/exact Existing literature: • Typical goal: Increase efficiency • Often, no analytically proven robustness to catastrophic failures • Catastrophic: Complete robot failure • Some systems show robustness empirically

43. MSTC: Multirobot Spanning-Tree Coverage(ICRA 2005, ICRA 2006, ICRA 2006, RAS 08, AMAI 09) Family of multi-robot coverage algorithms • Off-line, on-line • Backtracking, non-backtracking • Allow for heterogeneous robots, fuel/speed constraints Common theme: Guaranteed Robustness Will complete as long as one robot is alive

44. A Spanning-Tree Coverage Single robot [Gabriely and Rimon, 2001] • Robots with coverage tool of size D • Grid terrains, each cell of size D • Switch to coarse grid, each cell of size 4D • Draw Spanning Tree (ST) on the coarse grid • Robot moves along ST path • A Hamiltonian cycle!

45. C B A Non-backtracking MSTC • Initialization phase: Build STC, distribute to robots • Distributed execution: Each robot follows its section • Low risk of collisions Robot B is done! Robot A is done! Robot C is done!

46. C B A Guaranteed robustness • Coverage completed as long as one robot is alive • Teamwork requirement: • Need to know task completion status of others

47. C D B A D D C C B B A A Analysis: Non-backtracking MSTC • Running time = max i k step(i) • Best case: • Worst case: n – k • Unfortunately, common case • General non-backtracking worst case: n - 2(k-1) - 1

48. D D C C B B A A Backtracking MSTC • Similar initialization phase • Robots backtrack to assist others • No point is covered more than twice

49. B A D C B A Backtracking MSTC analysis • Best case: The same • Worst case: k=2 k>2

50. Generating a good spanning tree(believed to be NP-Hard) A A B B C C A B = 28 cells B C = 4 cells C A = 4 cells A B = 12 cells B C = 12 cells C A = 12 cells