UAV Search & Capture of a Moving Ground Target

1. UAV Search & Capture of a Moving Ground Target K. Krishna, D. Casbeer, P. Chandler & M. Pachter AFRL control science Center of excellence MACCCS Review April 19, 2012 Approved for public release; distribution unlimited; case number: 88ABW-2012-2747

2. Scenario UGS Sensor Range Valid Intruder Path UAV Communication Range UGS Communication Range BASE

3. UAV/UGS Framework • UAV engaged in search and capture of intruder on a road network • Intersections in road instrumented with Unattended Ground Sensors (UGSs) • Intruder is a goal oriented random walker • UAV has a speed advantage over the intruder • Passing intruder triggers UGS and the event is time-stamped and stored in the UGS • UAV has no sensing capability • Capture occurs when UAV and intruder areat an UGS location at the same time

4. Manhattan Grid G G S G All edges of the grid are of same length Intruder starts at node S and proceeds towards goal nodes marked G UAV also starts at node S and observes red UGS with delay d Intruder dynamics - randomly move North, East or South but cannot retrace path UAV actions - move North, East or South or Loiter at current location

5. System Dynamics

6. Solution Method • Pose the problem as a POMDP • unconventional POMDP since observations give delayed intruder location information with random time delays! • Use observations to compute the current intruder position probabilities • Dual control problem • choose UAV action that minimizes the uncertainty associated with intruder location (localization) or choose action that gets the UAV closest to the most probable intruder location (“end game”/ capture)

7. A few notes • Assumptions: • The intruder finishes in finite time and • can not retrace his steps (no cycles). • Guaranteed intruder capture is hard because • Incomplete, out-of-sequence, and delayed information (Unconventional POMDP formulation)

8. Worst Case Guaranteed Capture(Perfect Information) Related to control with uncertainty modeled as bounded sets [Witsenhausen 68, Bertsekas and Rhodes 73] Guaranteed capture is related to reachable sets [Bertsekas and Rhodes 71]

9. Full Information Scenario • From end of finite horizon • Find set of UAV “safe’’ locations that guarantee capture • Proceed backwards in time looking for “safe locations” • Under full information, we have a necessary and sufficient condition for guaranteed capture [Bertsekas and Rhodes 71]

10. Full Information strategy • Case 1: Intruder is at a goal • The UAV must be at the same goal location • Case 2: Intruder is not at the goal • Safe locations: • Capture now OR • Ensure that UAV is in a position to capture the intruder in the future regardless of intruder’s moves.

11. Partial and Delayed observation Proceed in a fashion similar to the full information case except using (estimate) set of intruder locations instead of the true (unknown) intruder location.

12. Out of sequence Estimation Process

13. Set of safe UAV locations

14. Backward (DP) recursion Either the UAV is at the current intruder location leading to immediate capture or the UAV can take action such that it goes to a favorable location in the next time step; leading to eventual capture.

15. Sufficient condition The condition is however not necessary for guaranteed capture due to the dual control aspect of the problem.

16. 2x2 Grid Example Problem UAV starting location 1 satisfies sufficient condition (see entry 0 in table)

17. Min-Max Optimal Paths For bottom row delays 1 and 2. Intruder path is orange and corresponding optimal UAV path is in green (dotted circle represents loiter). b2 b1

18. Min-Max Optimal (End Game) For middle row delay 1. Intruder path is orange and corresponding optimal UAV path is in green (dotted circle represents loiter). c1 b1

19. Min-Max Optimal Paths For bottom row delay 3. Intruder path is orange and corresponding optimal UAV path is in green (dotted circle represents loiter). c1 b3 b1

20. Min-Max Optimal Paths For middle row delay 2. Intruder path is orange and corresponding optimal UAV path is in green (dotted circle represents loiter). c1 c2 b1

21. Min-Max Optimal Paths For middle row delay D>0 - Induction argument (to prove optimality) leading to the “End Game” cD-1 cD c1 b1 (D-2) steps

22. Min-Max Optimal Paths For bottom row delay D>2 - Induction argument (to prove optimality) leading to the “End Game” cD-2 c1 bD b1 (D-3) steps

23. Min-Max optimal Steps to capture

24. Conclusions • Sufficient condition for guaranteed capture in partial information case • Currently trying to tighten the gap between necessity and sufficiency • Include possible future observations to reduce the predicted future uncertainty set • An induction argument for guaranteed capture on longer horizons (exploit structure in the graph)

25. Search & Capture Video