Random Disambiguation Paths

1. Random Disambiguation Paths Adaptive Sensing MURI Workshop June 28, 2006 Duke University Al Aksakalli In Collaboration with Carey Priebe&Donniell Fishkind Department of Applied Mathematics and Statistics Johns Hopkins University

2. Outline: • Problem Description • Markov Decision Process Formulation • Simulated Risk Disambiguation Protocol • Computational Experiments • Ongoing Research • Summary and Conclusions

3. Problem Description: Spatial arrangement of detections: true detections , false detections

4. Problem Description: Spatial arrangement of detections: true detections , false detections .29 .11 .72 .26 .61 .23 .39 We only see .59 .72 .89 .68 .83 .13 .64 Assume for all that is the probability that .32 .27

5. Problem Description: Given start and destination .29 .11 .72 .26 .61 .23 .39 .59 start s .72 .89 t destination .68 .83 .13 .64 .32 .27

6. Problem Description: Given start and destination .29 .11 .72 About each detection there is a hazard region , an open disk of fixed radius .26 .61 .23 .39 .59 s .72 .89 t .68 .83 .13 .64 .32 .27

7. Problem Description: Given start and destination .29 .11 .72 About each detection there is a hazard region , an open disk of fixed radius .26 .61 .23 ?? .39 .59 s We seek a continuous curve from to in of shortest achievable arclength .72 ?? .89 t .68 .83 .13 .64 .32 .27

8. Problem Description: Given start and destination .29 .11 .72 About each detection there is a hazard region , an open disk of fixed radius .26 .61 .23 .39 .59 s We seek a continuous curve from to in of shortest achievable arclength .72 .89 t .68 .83 .13 .64 …and we assume the ability to disambiguatedetections from the boundary of their hazard regions. .32 .27

9. Problem Description: Given start and destination .29 .11 .72 About each detection there is a hazard region , an open disk of fixed radius .26 .61 .23 true .39 .59 s We seek a continuous curve from to in of shortest achievable arclength .72 .89 t .68 .83 .13 .64 …and we assume the ability to disambiguate detections from the boundary of their hazard regions. .32 .27

10. Problem Description: Given start and destination .29 .11 .72 About each detection there is a hazard region , an open disk of fixed radius .26 .61 .23 …or false .39 .59 s We seek a continuous curve from to in of shortest achievable arclength .89 t .68 .83 .13 .64 …and we assume the ability to disambiguate detections from the boundary of their hazard regions. .32 .27

11. Problem Description: Given start and destination .29 .11 .72 About each detection there is a hazard region , an open disk of fixed radius .26 .59 s We seek a continuous curve from to in of shortest achievable arclength .89 t .68 .83 .13 .64 …and we assume the ability to disambiguate detections from the boundary of their hazard regions. .32 .27 the rest of the transversal…

12. Definition: A disambiguation protocol is a function # disambiguations allowed cost per disambiguation which detection disambiguated next… …and where the disambiguation performed

13. Example 1: Protocol gives rise to the RDP Length=707.97, prob=.89670 Length=1116.19, prob=.10330

14. Example 2:Protocol gives rise to the RDP (superimposed composite)

15. Random Disambiguation Paths (RDP) Problem: Given , find protocol of minimum .

16. Related work: • Canadian Traveller Problem (CTP): Graph theoretic RDP • Given a finite graph –edges with specific probabilities of being traversable, and a starting and a destination vertex – each edge’s status is revealed only when one of the end points is visited: objective is to minimize expected traversal length • Shown to be #P-hard

17. Markov Decision Process (MDP) formulation: Let be the information vector keeping track of the decision maker’s current knowledge; be the set of all possible disambiguation points RDP Problem can be cast as a K-stage finite horizon MDP with States: Actions: where v is a disambiguation point and i is a hazard region index Rewards: the negative of the shortest path distance between the state vertex and the action vertex minus c, if not going to d - d is an absorbative state for which there is a one-time and very large reward for entering Transitions: governed by ‘s

18. Simulated Risk Protocol: For purpose of deciding next disambiguation point, we pretend that ambiguous disks are riskily traversable… ? ? ? traversal ? ?

19. Risk Simulation Protocol: For purpose of deciding next disambiguation point, we pretend that ambiguous disks are riskily traversable… ? ? ? traversal ? ? is the usual Euclidean length of . is the surprise length of , which is the negative logarithm of the probability that is traversable in actuality.

20. Given undesirability function (henceforth, monotonically non-decreasing in its arguments) and, say,

21. Given undesirability function (henceforth, monotonically non-decreasing in its arguments) and, say, Definition:The simulated risk protocol is defined as dictating that the next disambiguation be at the first ambiguous point of . ? ? ? traversal ? ?

22. Given undesirability function (henceforth, monotonically non-decreasing in its arguments) and, say, Definition:The simulated risk protocol is defined as dictating that the next disambiguation be at the first ambiguous point of . ? ? ? traversal ? ? How to proceed once this disambiguation is performed: update and , decrement , and set the new s to be y.

23. How to navigate in this continuous setting: The Tangent Arc Graph (TAG) is the superimposition/subdivision of all visibility graphs generated by all subsets of disks. • For any undesirability function, is an path in TAG !

24. Linear undesirability functions: • Because of the efficiency in their realization, we will • consider simulated risk protocols generated by linear undesirability • functions for a chosen parameter . • As a further shorthand, denote such a protocol by .

25. How (during the simulation of risk phase) can be affected by :

26. How (during the simulation of risk phase) can be affected by :

27. How (during the simulation of risk phase) can be affected by :

28. How (during the simulation of risk phase) can be affected by :

29. How (during the simulation of risk phase) can be affected by :

30. Example 1: Protocol gives rise to the RDP Length=707.97, prob=.89670 Length=1116.19, prob=.10330

31. Example 2: Protocol gives rise to the RDP (superimposed composite)

32. Lattice Discretization: Discretization via a subgraph of the integer lattice with unit edge lengths:

33. Example:Adapting the simulated risk protocol to lattice discretization:

34. Computational experiments: A 40 by 20 integer lattice is used Each hazard region is a disk with radius 5.5 Disk centers sampled from a uniform distribution of integers in ‘s sampled from uniform distribution on (0,1) Cost of disambiguation is taken as 1.5 For each N, K combination, 50 different instances were sampled Optimal solutions found by solving the MDP model via value iteration

35. Illustration with N=7, K=1: Expected length:

36. Comparison of optimal versus simulated risk: Runtime to find overall optimal (SR-RDP runtime negligible) • Simulated risk found the optimal solution 74% of the time • Overall mean percentage error of simulated risk solutions was less than 1% • For N=7, K=3; VI took more than an hour • for N=10, K=1; VI did not run due to insufficient memory

37. Ongoing Research: Pruning State Space via AO* • Implemented an enhanced version of AO-star algorithm • Preliminary results suggest up to 99% of the state space can be pruned • N=15, K=2 can be solved under 15 mins! • Not practical for K>2: N=15, K=3 takes 10.5 hours!!! • Simulated Risk protocol still seems to perform well

38. Example: Enhanced AO* with N=15, K=2

39. Ongoing Research: Multiple sensors & Neutralization • Deployment of multiple sensors with different accuracy rates & ranges at different costs • Also consider a limited neutralization capability • Develop and solve corresponding Partially Observable Markov Decision Process (POMDP) models

40. Summary and Conclusions • RDP is an important, yet hard mine-countermeasures problem • Obtaining optimal solutions presently not feasible for realistic values of N and K • Simulated risk protocol is a sub-optimal yet efficient algorithm that performed well in computational experiments

41. Q & A