Decentralized Mission Planning for Heterogeneous Human-Robot Teams

1. Decentralized Mission Planning for Heterogeneous Human-Robot Teams Sameera Ponda Prof. Jonathan How Department of Aeronautics and AstronauticsMassachusetts Institute of Technology

2. Motivation • Modern day complex missions involve networked teams of heterogeneous agents executing several tasks simultaneously: • Unmanned aerial vehicles (UAVs) – target tracking, surveillance • Human operators – classify targets, monitor status • Ground convoys – rescue operations • Key Research Questions: • How can we coordinate team behavior to improve mission performance? • How should planning strategies evolve as we acquire more information?

3. Problem Statement • Goal: Automate task allocation to improve mission performance • Spatial and temporal synchronization • Reduce costs and improve efficiency • Key Technical Challenges: • Combinatorial decision problem – computationally intractable (NP-hard) • Complex agent modeling & constraints (stochastic, non-linear, time-varying) • Limited resources (bandwidth, fuel, etc) • Dynamic networks and communication constraints • Unknown and dynamic environments Task3 Task1 Task2 Task4 Agent5 Agent1 Agent4 Agent6 Agent2 Task5 Task6 Task9 Agent3 Task8 Task7

4. Planning Approaches • Optimal solution methods are computationally intractable for large problems • Typically use approximation methods • Centralized Planning approach • Mission Control Center (MCC) plans & distributes tasks for all agents • High bandwidth, slow reaction, resource intensive • Recent research in Decentralized Planning • Individual agents make their own plans and coordinate with each other • Faster reaction to local information changes • Trade-off between communication and computation • Key Questions: • What quantities should the agents agree upon? • Information / tasks & plans / objectives / constraints • How do we ensure that the planning is robust to inaccurate information and models? Agent2 Agent1 MCC Agent3 Agent4 Agent1 Agent2 Agent3 Agent4

5. Consensus-Based Bundle Algorithm • Decentralized task allocation approach calledConsensus-Based Bundle Algorithm (CBBA)[Choi, Brunet, How 2009] • CBBA iterates between 2 phases: Bidding & Consensus • Core features of CBBA: • Polynomial-timedecentralized algorithm with provably good approximate solutions • Consensus on task assignments, not information – guaranteed real-time convergence even with inconsistent information 1 Phase 1: Build Bundle & Bid on Tasks (individual agents) 2 Phase 2: Consensus (all agents) All agents consistent? 3 Yes N No • Key extensions to CBBA: • Temporal constraints – Time-windows of validity for tasks • Connectivity issues and constraints • Planning for teams with Humans-in-the-loop

6. CBBA with Time-Windows • In realistic missions, task scores often depend on arrival times and have associated time-windows of validity: • Issue: Planning algorithms usually involve time discretization • Extra planning dimension – computationally intractable! • CBBA extended to include time-windows • Solution does not discretize time! • Preserves convergence properties • Planner decides arrival times, producing • task schedules for agents • Embedded CBBA with Time-Windows • into a real-time system architecture Score Score Score Flat Time-critical Peak-time e.g. rendezvous, special ops e.g. monitor status, security shifts e.g. rescue ops, target tracking Arrival Time Arrival Time Arrival Time

7. CBBA with Time-Windows • CBBA successfully used in real-time fight test environments • Cooperative search, acquisition, and track (CSAT) • Coordination of agents under dynamic network topologies • Further information available online at: http://acl.mit.edu/projects/cbba.html

8. Connectivity: Network Challenges • As agents move around in the environment, expect varying network topologies • Limited communication radius between agents • Potential broken comm links and/or disconnected networks • Main issue: Planner cannot converge with a disconnected network, leading to conflicting assignments • Developed two solution approaches: • CBBA with Relays – Creates relay tasks to ensure connectivity • CBBA with Network Handling Protocols – Protocols to adjust task lists prior to planning Task1 Task3 Task2 Task4 Agent1 Agent5 Disconnected Network Agent2 Agent4 Task9 Agent6 Task6 Agent3 Task5 Task7 Task8

9. Connectivity: CBBA with Relays • Extended CBBA to include relay tasks – (Published in GlobeComm 2010) • Employs underutilized agents as relays • Key feature: Agents use bid info to predict network structure at select times • Guarantees connectivity • Computationally efficient - converges in real-time • CBBA with Relays improves team performance and network connectivity Relay Task

10. Connectivity: Network Protocols • If preventing disconnects is too conservative: Network Handling Protocols to adjust task lists for agents prior to planning – (Published in ACC 2010) • Local Adjustment improves mission performance • with low bandwidth and computation requirements Conflicting Assignments – lower mission scores and wasted fuel Guaranteed Deconfliction– higher mission scores and lower fuel consumption

11. Planning for Human-Robot Teams • Most modern missions involve human-robot teams • Human operators perform several tasks • (e.g. supervisory, target classification, monitoring) • Need to coordinate robotic agents and operators • Main Issue: Operator performance is stochastic • Heterogeneous operator capabilities (“slow” vs. “fast”) • Robustness to uncertainty in team performance • Recent research has explored modeling operators • using probabilistic distributions – [Cummings et al ‘10] • Key Challenge: Incorporate uncertainty into planner to increase robustness Predator UAV Operations – Associated Press Log-Normal Distribution for Operator Target Identification Figure from [D. Southern, Masters Thesis, 2010]

12. Planning for Human-Robot Teams • Consider a time-critical mission with operators performing target classification • As expected vs. actual service times differ, planner performance degrades • Adding a margin of conservatism can mitigate this problem • Tradeoff between late penalties and number of tasks assigned • Simulation Observations: • Performance is best when expected & actual • are close (ridge line) • Steeper drop for overestimating (optimistic) • vs. underestimating (conservative) • Conservative Planning performs better • than Optimistic Planning • Developing a Robust Planning Framework • Explicitly embed PDFs of plan parameters • Adapt as estimates improve Agent Schedule Late! Tasks Time

13. Planning for Human-Robot Teams • Currently performing Human-in-the-loop experiments at Cornell • CBBA used to allocate targetsto agents (MIT) • Image processing and sensor fusion used to update target PDFs (Cornell) • Human-in-the-loop for target classification and PDF updates through HRI (Cornell)

14. Conclusions • Explored strategies to coordinate team behavior to improve mission performance • Extended the Consensus-Based Bundle Algorithm (CBBA) to address the demands of more realistic multi-agent mission planning • Included task time-windows of validity • Addressed connectivity issues and communication constraints • Explored planning for heterogeneous human-robot teams • Current research and expected thesis contributions: • Robust decentralized planning framework • Embed distributions of parameters into planner • Preserve computational tractability and scalability • (e.g. avoid discretization, explore efficient sampling techniques) • Flexible planner structure that adapts to dynamic uncertainty representations • Modular uncertainty representations (Nonparametric Bayesian models, etc) • Modify planning strategy without recomputing all scenarios • Efficient strategies for information consensus to improve planner performance • Decide what information and when to share (e.g. hyperparameter consensus) • Cooperative decentralized strategies to update global distributions