Francesco Amigoni, Nicola Basilico, Nicola Gatti {amigoni,basilico,ngatti}@elet.polimi.it

1. Finding the Optimal Strategies in Robotic Patrolling with Adversaries in Topologically-Represented Environments Francesco Amigoni, Nicola Basilico, Nicola Gatti {amigoni,basilico,ngatti}@elet.polimi.it

2. Robotic Patrolling € € € € A patrollingstrategydetermines the pathfollowedby the robot,usually the nextcelltomoveto

3. Randomized Patrolling Strategies € Randomizedstrategy: the robot determines the nextcellaccordingto a probabilitydistribution The patrollershould adopt an unpredictable patrolling strategy, randomizing over cells and trying to reduce the intrusion risk (Pita et al., AAMAS08)

4. Patrolling Strategies with Adversaries • Considering a model of the adversary (Agmonet al., AAMAS08, Paruchuriet al., AAMAS08) can provide the patrolling robot a larger expected utility than not considering it, i.e., it can lead to better strategies (Amigoniet al., IAT2008) • Model of the adversary can include: its preferences over the possible targets, its knowledge about the patroller’s strategy, …

5. The Problem € Agmonet al., ICRA08 The problem we addressed in this work: finding the optimal randomized patrolling strategy in a arbitrary environment while considering a model of the adversary Ourapproachappliestoenvironmentswitharbitrarytopologygeneralizing (Agmonet al., ICRA08)

6. The Basic Patrolling Model • Time is discrete • Environment: represented by a directed graph,e.g.,a grid of cells or a topological map (Carpinet al., IROS08) • Single patrolling robot • It can move between adjacent nodes • It can detect a possible intruder in its current node • Single intruder • It knows the strategy of the patrolling robot, for example because it can observe the patroller movements before attempting to intrude • It can directlyenteranynode • Penetrationtimediisrequiredtosuccessfully complete anintrusion in a nodei • When attempting to penetrate in a node i at time t, the intruder can be detected during {t,t+1,…,t+ di}

7. The Basic Patrolling Model 1 2 3 4 5 7 8 6 move(10) move(12) move(7) € 12 9 10 13 I I I P … wait enter(13) … … enter(1) P P P 1 timeunit … … … Final States • The indruderentersnodei at timet: • If the patrollerdoesnotvisitcell i in the interval{t,t+1,…,t+ di}the intruder wins • Otherwise the intruder iscaptured and the patrollerwins • The intruder neverenters Utilities • Xi ,Yi (i ∈ {1, 2, …, 13}) : patroller’s and intruder’s utilities when the intruder successfully attacks node i • X0 ,Y0 : patroller’s and intruder’s utilities when the intruder is captured

8. Objective € The proposed method finds the probability distribution over the patroller movements, i.e., given the current node, finding the probability of moving in each adjacent node

9. Solving the Game • Two competing actors: we study their behaviors in a game-theoretical framework • The patrollingproblem can bemodeledas a leader-follower game • Twoplayers • The leader commitsto a strategy • The followerobservessuchcommitment and actsas a best responder • Patrollingstrategy: A = {αi,j}, whereαi,jis the probabilityofdoingmove(j)wheniis the currentnode • The optimal A can be derived by computing the equilibrium of the leader-follower game resorting to a bilevel optimization problem (Conitzer and Sandholm, 2006)

10. Solving Algorithm Step 1: is there any strategy A such that the game will never end? • Single bilinearfeasibilityproblem • If a solutionisfound, itis the best patrollingstrategy and the intruder willneverattempttoenter If the above problem does not admit a solution, Step 2: Optimalpatrollingstrategythatmaximizespatroller’s expected utility Game Model Solvingalgorithm • Wesafely assume that the game will end, i.e., the intruder willenter • Wecompute A suchthat the patroller’s expectedpayoffismaximum • Thisamountsto solve a bilinearoptimizationproblemforeverypossibleactionof the intruder

11. An Example X0 = 1 Y0= -1 X1 = 0.8 Y1 = 0.2 d1 = 7 X1 = 0.8 Y1 = 0.2 d1 = 7 0.774 0.451 0.344 0.676 X5 = 0.5 Y5 = 0.5 d5 = 7 X5 = 0.5 Y5 = 0.3 d5 = 7 0.226 0.127 0.096 0.102 Withthisstrategy the game neverends, i.e., the intruder willneverenter 0.898 0.549 0.529 0.228

12. Another Example X0 = 1 Y0= -1 1 0.546 0.546 0.546 X1 = 0.8 Y1 = 0.2 d1 = 5 X1 = 0.8 Y1 = 0.2 d1 = 5 X5 = 0.5 Y5 = 0.3 d5 = 4 X5 = 0.5 Y5 = 0.3 d5 = 4 1 0.454 0.454 0.454 Withthisstrategy the intruder willtrytoenter in cell 1 when the patrolleris in cell 5, the expected utility of the patrolleris 0.819

13. Model Extensions • Augmentedsensingcapabilities: we introduce the rangeparameter • Synchronizedmultirobotsetting: a single patrollerabletosenseanarbitrary subset ofcells X0 = 1 Y0= -1 X4 = 0.8 Y4 = 0.4 X6 = 0.7 Y6 = 0.5 X12 = 0.8 Y12 = 0.4 expected utility penetrationtime

14. Conclusions and Future Works • Wepresentedanapproachtofindoptimalrandomizedpatrollingstrategies in arbitraryenvironmentswithadversaries • Future Works • Accounting for intruder’s movements and limitedobservationcapabilities • Extending our framework with multiple non-synchronized patrollers