Multi-Vehicle Exploration and Routing in Adversarial Environments

1. Multi-Vehicle Exploration and Routing in Adversarial Environments Eric Feron Farmey Joseph Jerome le Ny Program Review Meeting The Stata Center at MIT June 22, 2005

2. Exploration for UAVs Ambush-aware Routing Exploration with stochastic agents Multi-Vehicle Exploration and Routing in Adversarial Environments

3. A common situation in hostile environments: a person must travel from an origin to a destination on a regular schedule. For example: Military Police drive from base each day to school or hospital to provide security Humanitarian convoy leaves supply base each day to deliver food, medicines, etc. to aid dispersal site Ambassador makes daily trip from home to embassy Must randomize the routing so as to minimize the probability of being ambushed by hostile forces? Determine an optimal mixed strategy for choosing among possible routes between origin and destination Routing

4. Previous Work • Exact solutions have been found for a number of specialized ambush games • [Ruckle, 1983], [Baston & Bostock, 1987] solve a continuous ambush game: Player 1 navigates from one side of a rectangle to the other, Player 2 sets finite-length barriers. • [Woodward, 2003] solves a discretized version of the above game. • These solutions are difficult to generalize to an arbitrary city road network

5. Game is played on a graph with directed edges edges » usable roads nodes » road intersections Player 1 (VIP) chooses a path from origin node to destination node Player 2 (hostile forces) simultaneously chooses k nodes as ambush locations k typically small, dependent on enemy resources If P1 is ambushed at node i, game outcome is ai > 0. ai» level of exposure of node i to an ambush If P1’s path avoids all ambush nodes, game outcome is 0. The VIP Transport Game

6. Assumptions • The game is repeated an infinite number of times • Both players have complete information about the environment, including origin and destination nodes • If Player 1’s path passes through more than one ambush site, then the game outcome is equal to the sum of the outcomes of all ambushes. • All ambushes take place at nodes rather than along edges • Player 2 is intelligent and will place ambushes according to the optimal strategy in response to Player 1’s chosen mixed strategy

7. Problem • Given: • graph G = (N, E) • origin node no • destination node nd • ambush outcome values ai for each node • Find: • Player 1's optimal mixed strategy for choosing among all paths between origin and destination, such that the expected outcome of the game is minimized

8. Naive Approach • Decision variables » number of routes, grows exponentially with size of graph • Determine all possible routes from origin to destination for Player 1 • Determine all possible ambush strategies for Player 2 • Form the game matrix A where Aij = game outcome if P1 chooses route i and P2 chooses ambush strategy j • Solve an LP for P1’s optimal mixed strategy p* • Massive game matrix A • Excessive memory requirements • Not a practical formulation

9. Network Flow Approach Concept flow along an edge = probability of traversing that edge • inflow of 1 at origin • outflow of 1 at destination • flow vector $ mixed strategy: p = (p1, p2, ...) • decision variables: » number of edges << number of routes for k-ambush problem: minimize the max flow (weighted by ai) into any set of k nodes

10. Network Flow Approach Formulation • Minimize • z = max a-weighted flow into any set of k nodes • Subject to: • a-weighted flow into any set of k nodes <= z • flow conservation at each node • flow nonnegativity min z s.t. Dp – 1z·0 Ap = b p ¸ 0 Solve for Player 1’s optimal strategy (p*) and the expected value of the game outcome (z*)

11. Example: Cambridge, MA Scenario • 50 nodes, 91 edges • Fresh Pond to downtown Boston • Major roads only • Estimate of enemy capability: 2 ambushes • Ambush outcome values: ai = min {distance to no, distance to nd}

12. Example: Cambridge, MA Solution • Thickness of line proportional to probability of traversing that edge • Generate random path based on edge probabilities • Solution time: 1.5 sec (Pentium 4, 1.8 GHz, 256 MB RAM)

13. Variations • Multiple VIPs, different origins and destinations, traveling in separate vehicles • multicommodity network flow problem • Multiple VIPs, different origins and destinations, traveling in one vehicle • adjust flow constraints to set inflow = 1 at each origin and destination

14. Multi-Agent Exploration • What kind of agents? • Uncertain/hostile environment • Agents can fail: trade-off cost/reliability • Minimization of budget allocated to mission (assume well defined cost per agent) • Number and type (i.e. cost) of agents is a parameter under control in general

15. Agents for Exploration: Examples • High-cost agents: autonomous vehicles, engineered mobility (UAVs, cars…). Reliable, few agents involved. • Low-cost agents: carried by environment + limited control of mobility. Cheap (batch fabrication, ex MEMS), unreliable individually, but use redundancy.

16. Naturally Mobile Sensors • Ex: rats controlled via brain electrodes (ref: Talwar et al., Nature 417, 2002) • Potential Applications include search and rescue operations and mine clearance

17. Line Exploration Models Some exploration scenarios in 1-D • Deterministic agents dropped randomly on a line. N agents dropped at x1,…,xN. Minimize sum of the distances traveled to completely explore the line. How to choose respective exploration segments, i.e. R1,…,RN?

18. Deterministic Agents • Only one agent should switch direction • Optimal cost is • For uniform distribution of agents, ca=cost/agent: optimal # of agents:

19. Probabilistic Agents (ex: naturally mobile sensors.) Continuous model: speed is uncertain control u(t) Є [-1;1] • Discrete Model: random walk, controlled drift Go right (p>q): Goal: explore the line with 1 prob. agent Equivalent results, easier with BM

20. Control Policies • Closed-loop control: agent has positioning capability, bang-bang controller optimal, always go towards closest extremity • Open-loop control: know at beginning which is closest extremity. Send agent towards that point until it reaches it, then to the other end. No positioning (except starting point) necessary

21. Control Policies • Independently of starting point: (in expectation) • Positioning sensor and optimal controller give little improvement of performance

22. Exploration Time Distribution • Open-loop strategy • An agent starting at b reaches 0 with probability at least 1-ε after at most deviation from expectation deterministic result

23. Agent Design Example • We have infinite number of prob. agents, initially all at b, to explore segment [0,b] • Open-loop policy • All agents have autonomy t0: cost for increasing t0 • Can send them one at a time towards 0. If fail to reach 0, send another one.

24. Agent Design Example • Cost: expected time until 1st success cost for increasing autonomy/reliability

25. Conclusions on Exploration • From 1D ex, expect that design and number of agents used in multi-agent systems should be optimized: • Saturation effect with increasing # of agents • Trade-off between cost of technology and benefits • Optimum depends on specific scenario

26. Extensions • How does this generalize to 2D? • Need simple strategies that are easy to compute and scale up with # of agents. • Exploit random natural mobility as tool for exploration (random sensor networks).

27. UAV Routing • Path planner: one component of high-level mission scheduling. Complexity? • Ex: compute shortest tour for aircraft to visit n sites, with simplified dynamics: • Dubins Traveling Salesman Problem (Savla, Frazzoli and Bullo, 2005)

28. Dubins TSP • Dubins’ vehicle is a model for fixed-wing aircraft, constant speed, maximum turn radius ρ • Given initial and terminal locations and velocity directions (headings), complete characterization of the path, computed in constant time

29. Alternating Algorithm • Based on solution of Euclidean TSP. Keep odd numbered edges (straight lines) and add Dubins’ paths between them

30. Randomized Headings Algorithm • Approximation Algorithm for DTSP • Computes in O(n3) a tour within of the optimum • Choose headings at random (good against adversary!) • Compute all-pair Dubins' distance matrix • solve the TSP problem on resulting asymmetric graph, using a (log n)-approximation algorithm

31. Randomized Headings Algorithm • Euclidean metric not involved • Dmin can be considered fixed lower bound on distance between any two points (coverage radius of on-board sensors) • Better performance for dense sets of points

32. Computational Experiments

33. Improvements • Approximation ratio analysis not tight: based on perturbation analysis for point-to-point paths, need a more global approach. Maybe use analysis of bead-tiling alg. of Savla et al. • Deviation from expectation? (also need global analysis.)

34. Conclusion • Network flow formulation provides an efficient means to rigorously determine the optimal mixed strategy for navigating from origin to destination and avoiding pre-set ambushes • For scenarios where the enemy observes the motion of the VIP and makes a real-time decision on ambush location, need to solve a multi-stage ambush game