440 likes | 519 Views
Partial-information Games with Reachability Objectives. Krishnendu Chatterjee Formal Methods for Robotics and Automation July 15, 2011 . TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A A A A. Games on Graphs.
E N D
Partial-information Games with Reachability Objectives KrishnenduChatterjee Formal Methods for Robotics and Automation July 15, 2011 TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA
Games on Graphs • Games on graphs. • Perfect-information graph games • Zermelo’s theorem about Chess in 1913 • From every configuration • Either player 1 can enforce a win. • Or player 2 can enforce a win. • Or both players can enforce a draw.
Chess: Games on Graph • Chess is a game on graph. • Configuration graph.
Graphs vs. Games Two interacting players in games: Player 1 (Box) vs Player 2 (Diamond).
Game Graphs • A game graph G= ((S,E), (S1, S2)) • Player 1 states (or vertices) S1 and similarly player 2 states S2, and (S1, S2) partitions S. • E is the set of edges. • E(s) out-going edges from s, and assume E(s) non-empty for all s. • Game played by moving tokens: when player 1 state, then player 1 chooses the out-going edge, and if player 2 state, player 2 chooses the outgoing edge.
Strategies • Strategies are recipe how to move tokens or how to extend plays. Formally, given a history of play (or finite sequence of states), it chooses a probability distribution over out-going edges. • ¾: S*S1 D(S). • ¼: S*S2! D(S).
Strategies • Strategies are recipe how to move tokens or how to extend plays. Formally, given a history of play (or finite sequence of states), it chooses a probability distribution over out-going edges. • ¾: S*S1! D(S). • History dependent and randomized. • History independent: depends only current state (memoryless or positional). • ¾: S1! D(S) • Deterministic: no randomization (pure strategies). • ¾: S*S1! S • Deterministic and memoryless: no memory and no randomization (pure and memoryless and is the simplest class). • ¾: S1! S • Same notations for player 2 strategies ¼.
Objectives • Objectives are subsets of infinite paths, i.e., õ S!. • Reachability: there is a set of good vertices (example check-mate) and goal is to reach them. Formally, for a set T if vertices or states, the objective is the set of paths that visit the target T at least once.
Applications: Verification and Control of Systems • Verification and control of systems • Environment • Controller M satisfies property (Ã) E C
Applications: Verification and Control of Systems • Verification and control of systems • Question: does there exists a controller that against all environment ensures the property. C M E satisfies property (Ã) || ||
Applications: Systems for Specification • Synthesis of systems from specification • Input/Output signals. • Automata over I/O that specifies the desired set of behaviors. • Can the input player present input such that no matter how the output player plays the generated sequence of I/O signals is accepted by automata ? • Deterministic automata: Games.
Game Models Applications -synthesis [Church, Ramadge/Wonham, Pnueli/Rosner] -model checking of open systems -receptiveness [Dill, Abadi/Lamport] -semantics of interaction [Abramsky] -non-emptiness of tree automata [Rabin, Gurevich/ Harrington] -behavioral type systems and interface automata[deAlfaro/ Henzinger] -model-based testing [Gurevich/Veanes et al.] -etc.
Reachability Games • Pre(X): given a set X of states, Pre(X) is the set of states such that player 1 can ensure next state in X. X T
Reachability Games • Pre(X): given a set X of states, Pre(X) is the set of states such that player 1 can ensure next state in X. X T
Reachability Games • Pre(X): given a set X of states, Pre(X) is the set of states such that player 1 can ensure next state in X. X T
Reachability Games • Pre(X): given a set X of states, Pre(X) is the set of states such that player 1 can ensure next state in X. • Fix-point X T
Reachability Games • Winning set for a partition: Determinacy • Player 1 wins: then no matter what player 2 does, certainly reach the target. • Player 2 wins: then no matter what player 1 does, the target is never reached. • Memoryless winning strategies. • Can be computed in linear time [Beeri 81, Immerman 81].
Chess Theorem • Zermelo’s Theorem Win1 Win2 Both draw
Why Partial-information • Perfect-information: controller knows everything about the system. • This is often unrealistic in the design of reactive systems because • systems have internal state not visible to controller (private variables) • noisy sensors entail uncertainties on the state of the game • Partial-observation Hidden variables = imperfect information. Sensor uncertainty = imperfect information.
Partial-information Games • A PIG G =(L, A, , O) is as follows • L is a finite set of locations (or states). • A is a finite set of input letters (or actions). • µ L £ A £ L non-deterministic transition relation that for a state and an action gives the possible next states. • O is the set of observations and is a partition of the state space. The observation represents what is observable. • Perfect-information: O={{l} | l 2 L}.
PIG: Example b a a,b b a
New Solution Concepts • Sure winning: winning with certainty (in perfect information setting determinacy). • Almost-sure winning: win with probability 1. • Limit-sure winning: win with probability arbitrary close to 1. • We will illustrate the solution concepts with card games.
Card Game 1 • Step 1: Player 2 selects a card from the deck of 52 cards and moves it from the deck (player 1 does not know the card). • Step 2: • Step 2 a: Player 2 shuffles the deck. • Step 2 b: Player 1 selects a card and view it. • Step 2 c: Player 1 makes a guess of the secret card or goes back to Step 2 a. • Player 1 wins if the guess is correct.
Card Game 1 • Player 1 can win with probability 1: goes back to Step 2 a until all 51 cards are seen, and plays uniformly at random for choosing cards till then. • Player 1 cannot win with certainty: there are cases (though with probability 0) such that all cards are not seen. Then player 1 either never makes a guess or makes a wrong guess with positive probability.
Card Game 2 • Step 1: Player 2 selects a new card from an exactly same deck and puts is in the deck of 52 cards (player 1 does not know the new card). So the deck has 53 cards with one duplicate. • Step 2: • Step 2 a: Player 2 shuffles the deck. • Step 2 b: Player 1 selects a card and view it. • Step 2 c: Player 1 makes a guess of the secret duplicate card or goes back to Step 2 a. • Player 1 wins if the guess is correct.
Card Game 2 • Player 1 can win with probability arbitrary close to 1: goes back to Step 2 a for a long time and then choose the card with highest frequency (choosing uniformly at random till the choice is made). • Player 1 cannot win probability 1, there is a tiny chance that not the duplicate card has the highest frequency, but can win with probability arbitrary close to 1, (i.e., for all ² >0, player 1 can win with probability 1- ², in other words the limit is 1).
Sure winning for Reachability • Result from [Reif 79] • Memory is required. • Exponential memory required. • Subset construction: what subsets of states (knowledge) player 1 can be. Reduction to exponential size perfect-information games. • EXPTIME-complete.
Partial-information Games a b a • In starting play a. • In yellow play a and b at random. • In purple: • if last was yellow then a • if last was starting, then b. • Requires both randomization and memory a b a b a a b a b
Almost-sure winning for Reachability • Result from [CDHR 06, CHDR 07] • Standard subset construction fails: as it captures only sure winning, and not same as almost-sure winning. • More involved subset construction is required. • EXPTIME-complete.
Almost-sure winning for Reachability • Brief Idea of the proof. • Suppose GOD gives us the winning knowledge sets. • Given a knowledge cannot play an action that is unsafe (takes out of the winning knowledge set). • Must play safe actions. • Play all safe actions uniformly at random.
Almost-sure winning for Reachability • Brief Idea of the proof. • Suppose GOD gives us the winning knowledge sets. • What GOD can do we can do by non-determinism. • This gives NEXPTIME algorithm, for EXPTIME algorithm see [CDHR 07].
Limit-sure winning for Reachability • Limit-sure winning for reachability is undecidable [GO 10, CH 10]. • Reduction from the Post-correspondence problem (PCP).
Limit-sure winning for Reachability • The undecidable result is first proved for probabilistic automata. • Probabilistic automata are special case of blind stochastic games. • For partial-information games, stochastic transitions can be simulated by deterministic transition [CDGH 10]. • Consequently we obtain the undecidability result.
Conclusion and Related Problems • Theory of graph games • Perfect-information and partial-information games. • Different solution concepts and different complexity. • Several algorithmic questions open. • Partial-observation stochastic games with pure strategies (Working manuscript of CD 11) • Many surprising results (such as non-elementary complete memory bounds).
Conclusion and Related Problems • Partial information games • Problem with clear practical motivation. • Challenging to establish the right frontier of complexity. • Important generalization of perfect-information games. • Unfortunately, undecidable and also high complexity. • Looking for new definitions (of sub-classes).
Collaborators • Laurent Doyen • Hugo Gimbert • Thomas A. Henzinger • Jean-Francois Raskin
The end Thank you ! Questions ?