How Much Memory is Needed to Win in Partial-Observation Games

1. How Much Memory is Needed to Win in Partial-Observation Games Laurent DoyenLSV, ENS Cachan & CNRS & Krishnendu ChatterjeeIST Austria GAMES’11

2. How Much Memory is Needed to Win in Partial-Observation Games stochastic Laurent DoyenLSV, ENS Cachan & CNRS & Krishnendu ChatterjeeIST Austria GAMES’11

3. Examples • Poker • - partial-observation • - stochastic

4. Examples • Poker • - partial-observation • - stochastic • Bonneteau

5. Bonneteau 2 black card, 1 red card Initially, all are face down Goal: find the red card

6. Bonneteau 2 black card, 1 red card Initially, all are face down Goal: find the red card • Rules: • Player 1 points a card • Player 2 flips one remaining black card • Player 1 may change his mind, wins if pointed card is red

7. Bonneteau 2 black card, 1 red card Initially, all are face down Goal: find the red card • Rules: • Player 1 points a card • Player 2 flips one remaining black card • Player 1 may change his mind, wins if pointed card is red

8. Bonneteau 2 black card, 1 red card Initially, all are face down Goal: find the red card • Rules: • Player 1 points a card • Player 2 flips one remaining black card • Player 1 may change his mind, wins if pointed card is red

9. Bonneteau: Game Model

10. Bonneteau: Game Model

11. Game Model

12. Game Model

13. Game Model

14. Game Model

15. Game Model

16. Observations (for player 1)

17. Observations (for player 1)

18. Observations (for player 1)

19. Observations (for player 1)

20. Observations (for player 1)

21. Observation-based strategy This strategy is observation-based, e.g. after it plays

22. Observation-based strategy This strategy is observation-based, e.g. after it plays

23. Optimal observation-based strategy This strategy is winning with probability 2/3

24. Game Model • This game is: • turn-based • (almost) non-stochastic • player 2 has perfect observation

25. Interaction General case: concurrent & stochastic Player 1’s move Player 2’s move Players choose their moves simultaneously and independently

26. Interaction -player games General case: concurrent & stochastic Probability distribution on successor state Player 1’s move Player 2’s move

27. Interaction Turn-based games Special cases: • player-1 state • player-2 state

28. Partial-observation General case: 2-sided partial observation Two partitions and Observations: partitions induced by coloring

29. Partial-observation General case: 2-sided partial observation Two partitions and Observations: partitions induced by coloring Player 2’s view Player 1’s view

30. Partial-observation Observations: partitions induced by coloring Special case: 1-sided partial observation or

31. Strategies & objective A strategy for Player is a function that maps histories (sequences of observations) to probability distribution over actions.

32. Strategies & objective A strategy for Player is a function that maps histories (sequences of observations) to probability distribution over actions. randomized History-depedent

33. Strategies & objective A strategy for Player is a function that maps histories (sequences of observations) to probability distribution over actions. Reachability objective: Winning probability:

34. Qualitative analysis The following problem is undecidable: (already for probabilistic automata [Paz71]) Decide if there exists a strategy for player 1 that is winning with probability at least 1/2

35. Qualitative analysis The following problem is undecidable: (already for probabilistic automata [Paz71]) Decide if there exists a strategy for player 1 that is winning with probability at least 1/2 Qualitative analysis: • Almost-sure: … winning with probability 1 • Positive: … winning with probability > 0

36. Example 1 Player 1 partial, player 2 perfect

37. Example 1 Player 1 partial, player 2 perfect No pure strategy of Player 1 is winning with probability 1

38. Example 1 Player 1 partial, player 2 perfect No pure strategy of Player 1 is winning with probability 1

39. Example 1 Player 1 partial, player 2 perfect Player 1 wins with probability 1, and needs randomization Belief-based-only randomized strategies are sufficient

40. Example 2 Player 1 partial, player 2 perfect

41. Example 2 Player 1 partial, player 2 perfect To win with probability 1, player 1 needs to observe his own actions. Randomized action-visible strategies:

42. Classes of strategies rand. action-visible Classification according to the power of strategies rand. action-invisible pure

43. Classes of strategies rand. action-visible Classification according to the power of strategies rand. action-invisible pure Poly-time reduction from decision problem of rand. act.-vis. to rand. act.-inv. The model of rand. act.-inv. is more general

44. Classes of strategies Computational complexity Strategy complexity (algorithms) (memory) rand. action-visible Classification according to the power of strategies rand. action-invisible pure

45. Known results Reachability - Memory requirement (for player 1)

46. Known results Reachability - Memory requirement (for player 1) [BGG09] Bertrand, Genest, Gimbert. Qualitative Determinacy and Decidability of Stochastic Games with Signals. LICS’09. [CDHR06] Chatterjee, Doyen, Henzinger, Raskin. Algorithms for ω-Regular games with Incomplete Information. CSL’06. [GS09] Gripon, Serre. Qualitative Concurrent Stochastic Games with Imperfect Information. ICALP’09.

47. Known results Reachability - Memory requirement (for player 1)

48. When belief fails (1/2) Belief-based-only pure strategies are not sufficient, both for positive and for almost-sure winning player 1 partial player 2 perfect

49. When belief fails (1/2) Belief-based-only pure strategies are not sufficient, both for positive and for almost-sure winning player 1 partial player 2 perfect

50. When belief fails (1/2) Belief-based-only pure strategies are not sufficient, both for positive and for almost-sure winning player 1 partial player 2 perfect not winning