Winning concurrent reachability games requires doubly-exponential patience

1. Winning concurrent reachability games requires doubly-exponential patience Michal Koucký IM ASCR, Prague Kristoffer Arnsfelt Hansen,Peter Bro Miltersen Aarhus U., Denmark

2. W • Player 1 chooses A{t,h} • Player 2 chooses B{t,h} • If • A = B then move one level up, • A  B = t then move to 1st level, • A  B = h then Player 1 loses. 7 Example 6 Entrance fee: $15 Win: $20 5 4 3 2 1

3. Entrance fee: $15 Win: $20 Observation: To break even, you need at least ¾ probability to win. Good news: you can win with probability arbitrary close to 1. Bad news: the expected time to win the game with probability at least ¾ is 1025 years (one move per day). … the age of universe: 1011 years

4. … … … Concurrent reachability games [de Alfaro, Henzinger, Kupferman ’98, Everett ’57] Two players play on a graph of states. At each step they simultaneously (independently) pick one of possible actions each and based on a transition table move to the next state. …

5. Goals: Player 1 wants to reach a specific state or states. Player 2 wants to prevent Player 1 from reaching these states. Strategy of a player: • Memory-less (non-adaptive) – π : states  actions. • Adaptive– π : history  actions. Probabilistic strategy: π gives a probability distribution of possible actions.  Patience of a memory-less strategy π = 1/min non-zero prob. in π … [Everett ’57]

6. Winning starting states: • Sure – Player 1 has a winning strategy that never fails. • Almost-Sure – Player 1 has a randomized strategy that reaches goal with probability 1. • Limit-Sure – For every  > 0, Player 1 has a strategy that reaches goal with probability at least 1 –  .

7. P • Player 1 chooses A{t,h} • Player 2 chooses B{t,h} • If • A = B then move one level up, • A  B = t then move to 1st level, • A  B = h then move to state H. n Purgatoryn n-1 … H 3 2 1

8. Our results Thm:1) For every 0<  < ½ , any -optimal strategy of Player 1 in Purgatoryn is of patience > 1/2n-2 . 2) For every l < n/2 , any (1 – 2-l )-optimal strategy of Player 1 in Purgatoryn is of patience > 22n-l-2. Thm:For every 0<  < ½ and every concurrent reachability game with m>61 actions in total, both players have -optimal strategies with patience < 1/242m.

9. Thm:1) For every 0<  <’, if every -optimal strategy of Player 1 is of patience > t then the expected time to win the game by any ’-optimal strategy of Player 1 can be forced to be Ω( t ).  patience ~ expected time to win • All the results essentially hold also for adaptive strategies Recall: the expected time to win Purgatory7 with probability at least ¾ is 1025 years (one move per day).

10. Algorithmic consequences Three algorithmic questions: • What are *-SURE states? PTIME [dAHK] • What are the winning probabilities of different states?  PSPACE [EY] • What is the (-)optimal strategy?  EXP-EXP-TIME upper-bound [CdAH,…]  EXP-SPACE lower-bound [our results] Cor: Any algorithm that manipulates winning strategies in explicit representation must use exponential space. … explicit representation: integer fractions

11. Player 2 plays t Player 2 plays h  Player 2 plays h P • pi – probability of playing t in state i in -optimal strategy of Player 1. • Claim: 1) 0< pi < 1, for all i. • 2)pi<  , for all i. • 3)p1 ≤ p2 . p3 … pn • 4)pi ≤ pi+1 . pi+2 … pn t n pn pn-1 Purgatoryn t n-1 … t p3 p2 p1 3 t 2 t 1

12. Open problems • Generic algorithm for -optimal strategy with symbolic representation? • How to redefine the game to be more realistic?

13. Goals: Player 1 wants to reach a specific state or states. Player 2 wants to prevent Player 1 from reaching these states. Winning starting states: • Sure – Player 1 has a winning strategy that never fails. • Almost-Sure – Player 1 has a randomized strategy that reaches goal with probability 1. • Limit-Sure – For every  > 0, Player 1 has a strategy that reaches goal with probability at least 1 –  .