Energy Parity Games

1. Energy Parity Games KrishnenduChatterjee and Laurent Doyen

2. Energy Parity Games Energy parity games are infinite two-player turn-based games played on weighted graphs. The objective of the game combines a (qualitative) parity condition with the (quantitative) requirement that the sum of the weights (i.e., the level of energy in the game) must remain positive.

3. Motivation • In the synthesis problem implementations are obtained from winning strategies in games with a qualitative objective. • Games with quantitative objective are natural models in economics, where players have to optimize a real-valued payoff. • There is considerable interest in the design of reactive systems that work in resource-constrained environments, such as embedded systems – that have both a quantitative component (specifying the resource constraint such as power consumption) and a qualitative component (specifying the functional requirement).

4. The Game • Two-player turn-based games • Played for infinitely many rounds • A weight is associated to each edge • A priority is associated to each state • In each round, the player owning the current state chooses an outgoing edge to a successor state.

5. The Game • A play satisfies the parity condition if the least priority occurring infinitely often in the play is even. • The quantitative specification requires that the sum of the weights along the play remains always positive. • The game goals are a-symmetric for player-1 and player-2.

6. Main Algorithmic Question Decide if there exists an initial credit (initial energy level) such that player-1 has a strategy to maintain the level of energy positive while satisfying the parity condition, and if the answer is yes, to compute the minimum such initial credit.

7. Main Results • Exponential memory is sufficient and may be necessary for winning strategies in energy parity games. • The problem of deciding the winner in energy parity games can be solved in NP ∩ coNP. • An algorithm to solve energy parity by reduction to energy games (find minimal initial energy).

8. Definitions: Game graphs Game graphs: A game graph G = <Q,E> consists of a finite set Q of states partitioned into player-1 states Q1 and player-2 states Q2 (i.e., Q = Q1∪Q2), and a set E ⊆ Q×Q of edges such that for all q ∈ Q, there exists (at least one) q′∈ Q such that (q, q′) ∈ E. A player-1 game is a game graph where Q1=Q and Q2=∅.

9. Definitions: Game graphs The subgraph of G induced by S ⊆ Q is the graph <S, E∩(S×S)> (which is not a game graph in general); The subgraph induced by S is a game graph if for all s ∈ S there exist s′∈ S such that (s,s′)∈E.

10. Definitions: Plays and strategies • Each player i chooses the next state iff the current state . • The game results in a play from q0 - i.e. an infinite path ρ=q0q1. . . such that (qi,qi+1)∈E for all i ≥ 0. • The prefix of length n of ρ is denoted by ρ(n)=q0...qn . • A strategy for player-1 is a function σ:Q∗Q1→Q such that (q, σ(ρ·q))∈E for all ρ∈Q∗ and q∈Q1.

11. Definitions: Plays and strategies • An outcomeof σ from q0 is a play q0q1... such that σ(q0...qi) = qi+1 for all i ≥ 0 such that qi∈Q1.

12. Definitions: Objectives An objective for G is a set φ ⊆ Qω. Let p:Q→N be a priority function andw:E→Zbe a weight function. We denote by W the largest weight (in absolute value) according to w. The energy levelof a prefix γ=q0q1...qn of a play is , and the mean-payofof a play ρ = q0q1 . . . is MP(w, ρ) = liminfn→∞n1 · EL(w, ρ(n)). • We denote by Inf(ρ) the set of states that occur infinitely often in ρ.

13. Definitions: Objectives • Parity objectives: The parity objective ParityG(p) = {π ∈ Plays(G) | min{p(q)|q∈Inf(π)} is even } requires that the minimum priority visited infinitely often be even.The special cases of Buchi and coBuchi objectives correspond to the case with two priorities, p : Q → {0, 1} and p : Q → {1, 2} respectively.

14. Definitions: Objectives • Energy objectives: Given an initial credit c0∈N∪{∞}, the energy objective PosEnergyG(c0) = {π ∈ Plays(G) | ∀n≥0 : c0+EL(π(n)) ≥ 0} requires that the energy level be always positive. • Mean-payof objectives: Given a threshold ν∈Q, the mean-payof objective MeanPayoffG(ν) = {π ∈ Plays(G) | MP(π) ≥ ν} requires that the mean-payoff value be at least ν.

15. Definitions: Objectives • Combined objectives: The energy parity objective ParityG(p)∩PosEnergyG(c0 ) and the mean-payof parity objective ParityG(p)∩MeanPayofG(ν) combine the requirements of parity and energy (resp., mean-payof) objectives.

16. Definitions: Winning Strategies • A player-1 strategy σ is winning in a state q for an objective φ if ρ∈φ for all outcomes ρ of σ from q.For energy and energy parity objectives with unspecified initial credit, we also say that a strategy is winning if it is winning for some finite initial credit.

17. Definitions: Initial Credit Problem • The initial credit problem asks, given an energy parity game <G,p,w> and a state q, whether there exists a finite initial credit c0∈N and a winning strategy for player-1 from q with initial credit c0. • The minimum initial credit in a state q0∈Q is the least value of initial credit for which there exists a winning strategy for player-1 in q0. • A strategy for player-1 is optimalin a state q0 if it is winning from q0 with the minimum initial credit.

18. Strategy Complexity • Memory of size 4・n・d・W is sufficient for a winning strategy of player-1 (n states and d priorities). • For player-2, memoryless winning strategies exist. • If player-1 wins, then the minimum initial credit is at most (n−1)・W.

19. Lemma 1 Let G be a player-1 energy parity game with n states. If player-1 wins in G from a state q0, then player 1 has a winning strategy from q0 with memory of size 2·(n−1)·W+1 and initial credit (n−1)·W.

20. Lemma 1 - Example We present a family of player-1 games wherememory of size 2·(n−1)·W+1 and initial credit of (n−1)·W may be necessary.

21. Lemma 1 - Proof • Q1=Q (since it’s a player-1 game). • Player-1 is winning. • Consider an outcome ρ of an optimal strategy for player-1: • Min priority of states in Inf(ρ) is even. • Inf(ρ) is strongly connected. • There exists a suffix ρ’ of ρ that only contains states in Inf(ρ). • Let {Ci} be the cycle decomposition of ρ’. • We will construct a fitting winning strategy.

22. Lemma 1 - Proof • We consider two cases. • First: If EL(Ci)>0 for some cycle Ci, from the starting state q0 reach the cycle Ci and pump the cycle until the energy level reaches 2(n-1)W, then reach a state of Inf(ρ) with the minimal priority and go back to cycle Ci. • Rinse and repeat forever. • Each ‘reach’ consumes at most (n-1)W energy, thus a starting energy of (n-1)W will sustain the energy>0 requirement.

23. Lemma 1 - Proof • Second: if EL(Ci) ≤0 for all cycles Ci. • There must be a k≥1 such that EL(Cj)=0 for all j≥k (or else we lose the energy condition). • Since the parity condition is satisfied in ρ the minimal priority of states in inf(ρ) is visited by some cycle Cj. • Go to cycle Cj from the starting state and pump it forever. • In both cases, player-1 wins with memory size 2·(n−1)·W+1 and initial credit (n−1)·W.

24. Lemma 2 • Let G be an energy parity game, and for each winning state q let v(q)∈N be the minimum initial credit in q. For all outcomes ρ of an optimal strategy σ in G from a winning state q0, if the initial credit is v(q0)+c for c≥0, then the energy level at all positions of ρ where a state q occurs is at least v(q)+c. • i.e.: An optimal strategy behaves the same even if we give it extra initial energy.

25. Lemma 2 - Proof • For all outcomesρofσ, the energylevel for all q in ρ must be at least v(q) (otherwise player-2 can win from q, and therefore wins in the original game in contradiction to σ being a winning strategy). • Since strategies are functions of sequence of states only (and not of their energy level), if we start with energy level v(q0)+c, then the energy level at all positions of an outcome of σ is greater by c than if we had started with energy level v(q0).

26. Lemma 3 For all energy parity games G, memoryless strategies are sufficient for player 2 (i.e., the minimum initial credit for player 1 does not change if player 2 is restricted to play memoryless).

27. Upper Bounds • we give upper bounds on the memory and initial credit necessary for player-1 in energy parity games.

28. Good-for-energy strategy • A strategy σ for player 1 is good-for-energy in state q if for all outcomes ρ=q0q1... of σ such that q0=q, for all cycles γ=qi...qi+k in ρ (where k>0 and qi=qi+k), either EL(γ)>0, or EL(γ)=0 and γ is even (i.e., minimum priority in γ is even).

29. Good-for-energy strategy • We will show that good-for-energy strategies that are memoryless exist.

30. Lemma 4 Let Win be the set of winning states for player1 in an energy parity game. Then, there exists a memoryless strategy for player 1 which is good-for-energy in every state q∈Win.

31. Lemma 4 - proof The definition of good-for-energy strategy in a state q can be viewed as a winning strategy in a finite cycle-forming game from q where the game stops when a cycle C is formed.

32. Lemma 4 - proof By [Memoryless determinacy of parity and mean payoff games, 2004] we know both players have memoryless optimal strategies in this finite cycle-forming game.

33. Lemma 4 - proof Given a state q∈Win where player-1 wins, towards contradictions assume he has no good-for-energy strategy from q. Then he loses the cycle-forming game. Which means player-2 wins the cycle forming game with a memoryless-strategy.

34. Lemma 4 - proof Use player-2’s strategy in the original energy parity game and pump the cycles forever instead of stopping. Then all cycles in the outcome have either negative weight or zero weight and the least priority is odd. Which means player-1 loses the energy parity game in contradiction to q∈Win. Thus: player-1 has a memoryless good-for-energy strategy σq from q.

35. Definitions: Attractor The player-1 attractor of a given set S ⊆ Q is the set of states from which player 1 can force to reach a state in S.

36. Definitions: Attractor Defined as the limit Attr1 (S) of the sequence: A0 = S Ai+1 = Ai ∪ {q ∈ Q1 | ∃(q, q′)∈E : q′∈ Ai } ∪ {q ∈ Q2 | ∀(q, q′)∈E : q′∈Ai} for all i ≥ 0.

37. Definitions: Attractor The subgraph of G induced by Q\Attri(S) is again a game graph (i.e., every state has an outgoing edge). Attractors can be computed in polynomial time.

38. Lemma 5 For all energy parity games G with n states and d priorities, if player 1 wins from a state q0, then player 1 has a winning strategy from q0 with memory of size 4·n·d·W and initial credit (n−1)·W .

39. Theorem 1 (Strategy Complexity) For all energy parity games, the following assertions hold: 1) Winning strategies with memory of size 4·n·d·W exist for player-1 (Lemma 5). 2) Memoryless winning strategies exist for player-2 (Lemma 3).

40. Strategy Complexity To Computational Complexity

41. Computational Complexity We show that the initial credit problem for energy parity games is in NP ∩ coNP.

42. Computational Complexity We show that the initial credit problem for energy parity games is in NP ∩ coNP. This is the upper-bound only, and might be lower since we already use exponential memory for player-1 winning strategies.

43. Lemma 6 Let G be an energy parity game. The problem of deciding, given a state q0 and a memoryless strategy σ, whether σ is good-for-energy in q0, can be solved in polynomial time.

44. Lemma 6 – proof Look at the states in G reachable via the strategy σ. For each state q in this graph, an algorithm for the shortest path problem can be used to check in polynomial time that every cycle through q has nonnegative sum of weights. For the states with odd priority we check that the sum is strictly positive.

45. General Direction • We first establish the NP membership of the initial credit problem for the case of energy parity games with two priorities.

46. Lemma 7 Memoryless strategies are sufficient for player1 to win energy co-Buchi games (i.e., the minimum initial credit for player-1 does not change if player-1 is restricted to play memoryless).

47. Lemma 8 (private case) The problem of deciding, given a state q in an energy Buchi (resp. coBuchi) game G, if there exists a finite initial credit such that player-1 wins in G from q is in NP.

48. Lemma 8 (private case) The problem of deciding, given a state q in an energy Buchi (resp. coBuchi) game G, if there exists a finite initial credit such that player-1 wins in G from q is in NP. For energy coBuchi games we guess all possible memoryless strategies for player-1 (Lemma 7) and check in polynomial time that he wins both the energy game and the coBuchi game (ensuring all cycles are positive and visit only priority-2 states).

49. Lemma 8 – cont’ For energy Buchi games we guess the winning set Win, a memoryless strategy σb in GWIN for the Buchi game and σgfe on Win (exists by Lemma 4).Check in polynomial time that σb enforces a visit to a priority-0 state in Win, that σgfe is good-for-energy (Lemma 6) and that q∈Win.

50. Lemma 9 (general case) The problem of deciding, given a state q in an energy parity game G, if there exists an initial credit such that player 1 wins in G from q is in NP.