Optimality in strategic games, CP nets and soft constraints

1. Optimality in strategic games, CP nets and soft constraints

2. Main aim • To compare the notion of optimality used in many formalisms • To throw the basis for exploiting results in one field and reuse them in the other field • Strategic games • Agent interaction while pursuing their own interest (payoff function) • CP nets • Agent’s qualitative and conditional preferences • Soft constraints • Agent’s quantitative preferences

3. Outline • Strategic games • Relation between CP nets and games • Relation between soft constraints and games

4. Parametrized Strategic games • A set of players 1,.., n • For each player i: • A set of strategies Si • A strict total order >iover Si depending on s-i (a joint strategy of all players but player i): payoff function • Example (prisoner’s dilemma): 2 players, 2 strategies (ci, ni) for each player i

5. Pure Nash equilibria • A strategy si is a best response for i to s-i if si≥i s’i for all s’i in Si • A joint strategy s is a pure Nash equilibrium if each si is a best response to s-i • Also: for all i, for all s’i in Si, si≥i s’i • No player has regrets on the strategy he chose • But there could be better joint strategies if more than one player changed its strategy • In the example, one Nash equilibria (NE): (N1,N2)

6. Pareto efficient joint strategies • No other joint strategy is better or equal for all agents, and better for at least one • Example: • (N1,N2): unique Nash equilibrium • All other joint strategies are Pareto efficient (PE)

7. Dominance between strategies • A strategy si is never a best response for i if it is not a best response to any joint strategy s-i • In the example: for each player i, Ci is never a best response

8. Elimination of dominated strategies • G NBR G’ • G’ subgame of G • For all i, each si in Si-S’i is never a best response for i in G Eliminate from the strategies of each players those that are never a best response

9. Nash equilibria and strategy elimination • If G NBR G’, then s Nash equilibrium of G iff Nash equilibrium of G’ • In the example: Ci is nbr, thus G’ has one row and one column, which is the unique Nash equilibrium 1,1

10. From CP-nets to games • Given a CP-net N, we build the game g(N) • Players: features • Strategies of player i: domain of feature xi • Payoff function of player i: CP table for xi • Given s-i, s’i >i si iff s-i|par(xi) : s’i >i si in the cp table for variable i • Thm: opt(N) = NE(g(N))

11. Fish, white, peaches Main course Fish, red, peaches Fish, white, berries Fish, red, berries Wine meat, red, peaches meat, red, berries meat, white, peaches Fruit meat, white, berries Example – CP net fish>meat peaches > strawberries

12. Main course Wine Fruit CP net  Param. Strategic Game • Three players: • 1 = main course, • 2 = wine, • 3 = fruit • Two strategies for each player: • S1= {meat, fish} • S2={red, white} • S3={peaches, strawberries} fish>meat • Payoff functions • For 1 main course: • fish > meat, always • For 2 wine: • fish, --  white > red • meat, --  red > white • For 3 fruit: • peaches > strawberries, always peaches > strawberries

13. Example: optimals and Nash equilibria • Unique optimal for CP-net: (fish, white, peaches) • Hard constraints: fish, peaches, fish  white, meat  red • For the game: • Meat is nbr for main course • Strawberries is nbr for fruit • Once meat is eliminated, red is nbr for wine • Nash equilibrium: fish, white, peaches

14. From games to CP-nets • Given a game G, we build a CP-net n(G): • Feature xi: player i • Domain of xi: strategies for player i • Parents of xi: all the other features • CP table of xi: s-i: si > s’i if si >i s’i given s-i • Thm.: NE(G) = opt(n(G))

15. Example • Two features: x1, x2 • D(x1)={c1, n1} • D(x2)={c2,n2} • x1 depends on x2 • x2=c2: n1 > c1 • x2=n2: n1 > c1 • x2 depends on x1 • X1=c1: n2 > c2 • X1=n1: n2 > c2 • Hard constraints: • x2=c2 → x1=n1 • x2=n2 → x1=n1 • x1=c1 → x2=n2 • x1=n1 → x2=n2 • Unique solution: x1=n1, x2=n2

16. Reduced CP-nets • If y is a parent of x, but the preference over the domain of x does not depend on y, then we can remove y from the parents of x  eliminate rows • From a CP net N to its reduced version r(N)

17. Two features: x1, x2 D(x1)={c1, n1} D(x2)={c2, n2} x1 depends on x2 x2=c2: n1 > c1 x2=n2: n1 > c1 x2 depends on x1 X1=c1: n2 > c2 X1=n1: n2 > c2 Two features: x1, x2 D(x1)={c1, n1} D(x2)={c2, n2} x1 and x2 independent For x1: n1 > c1 For x2: n2 > c2 Example: reduced CP-net

18. n G n(G) r r(n(G)) Nash equilibria of G = optimals of r(n(G)) CP-net techniques in games • From game G to n(G) • From n(G) to r(n(G)) • Hard constraints for r(n(G)) • Optimals of r(n(G)) = Nash equilibria of G

19. Example • Two features: x1, x2 • D(x1)={c1, n1} • D(x2)={c2,n2} • x1 depends on x2 • x2=c2: n1 > c1 • x2=n2: n1 > c1 • x2 depends on x1 • X1=c1: n2 > c2 • X1=n1: n2 > c2

20. Two features: x1, x2 D(x1)={c1, n1} D(x2)={c2, n2} x1 depends on x2 x2=c2: n1 > c1 x2=n2: n1 > c1 x2 depends on x1 X1=c1: n2 > c2 X1=n1: n2 > c2 Two features: x1, x2 D(x1)={c1, n1} D(x2)={c2, n2} x1 and x2 independent For x1: n1 > c1 For x2: n2 > c2 Example: reduced CP-net

21. Two features: x1, x2 D(x1)={c1, n1} D(x2)={c2, n2} x1 and x2 independent For x1: n1 > c1 For x2: n2 > c2 Hard constraints: x1=n1 x2=n2 Thus x1=n1, x2=n2 unique optimal solution of the CP-net and Nash equilibrium of the game Example: reduced CP-net

22. n G n(G) r Nash equilibrium of G = optimal of r(n(G)) r(n(G)) acyclic linear time Games and acyclic CP-nets • From game G to r(n(G)) • If r(n(G)) is acyclic, then G has one Nash equilibrium, and linear time to find it

23. Optimality in SCSPs, NE in Games • From Games to CSPs • Full power of SCSPs no needed to model NE Game G CP-net n(G) Greco et al.2005 CSP C(G) Optimality Constraints of N(G) Equivalent

24. Optimality in SCSPs, NE in Games • From a SCSP P to a game L(P) • Local Approach • Players: one for each variable • Strategies for a player i: all values in domain of xi • Payoff of player i for joint strategy s: preference for assignment s in constraints involving xi

25. local Example 1: Fuzzy SCSP  game X Y Z • Three players x,y,z • Two strategies a,b • Payoff functions • For x: px(aa-)=0.4, px(ba-)=0.3… • For y: • p(aaa) = min(0.4,0.4) = 0.4 • p(aba) = min(0.1,0.1)=0.1 • ... • Two Nash equilibria: aaa and bbb • Optimal solutions: only bbb (a,a)  0.4 (a,b)  0.1 (b,a)  0.3 (b,b)  0.5 (a,a)  0.4 (a,b)  0.3 (b,a)  0.1 (b,b)  0.5

26. local Example 2: Fuzzy SCSP  game X Y Z • Three players x,y,z • Two strategies a,b • Payoff functions • For x: px(aa-)=0.9, px(ba-)=0.6… • For y: • p(aaa) = min(0.9,0.1) = 0.1 • p(aab) = min(0.6,0.2)=0.2 • ... • Two Nash equilibria: aab and bbb • Optimal solutions: only aab, abb, bab, bbb (a,a)  0.9 (a,b)  0.6 (b,a)  0.6 (b,b)  0.9 (a,a)  0.1 (a,b)  0.2 (b,a)  0.1 (b,b)  0.2

27. Strictly monotonic combination • In general, no relationship between optimal solutions of P and Nash equilibria of L(P) • However, some relationship exist if combination is strictly monotonic • Thm.:Soft CSP P with strictly monotonic combination  Opt(P)  NE(L(P))

28. Classical CSPs  games • Classical constraints are combined via logical and (which is not strictly monotonic) • However, if we consider consistent CSPs, the result holds • Thm.: consistent CSP  Sol(P)  NE(L(P))

29. Optimality in SCSPs, NE in Games • Given an SCSP P, build a game GL(P): • Global mapping • Players = variables • Strategies = domain values • Payoff for player x for strategy s: preference value for that assignment (by looking at all constraints) • Note: same payoff for all players • Theorem: Opt(P)  NE(GL(P)) • Subset relation for all classes of SCSPs

30. Optimality in SCSPs, PE in Games • From a game G to an SCSP L’(G): • Variables = players (n) • Domains = strategies • Semiring = Cartesian product of n semirings • For each variable xi, one constraint involving xi and its neighborhood • pref(t) = (d1,...,dn), where dj = 1j for j  i, and di = F(pi(t)) • F is bijection from the payoffs to preferences in a c-semiring • Thm.: Game G  opt(L’(G)) = PE(G)

31. Example Semiring: weighted x weighted (c1,c2)  (7,0) (c1,n2)  (10,0) (n1,c2)  (6,0) (n1,n2)  (9,0) • Pareto efficient joint strategies: all but (1,1) X1 x2 (c1,c2)  (0,7) (c1,n2)  (0,6) (n1,c2)  (0,10) (n1,n2)  (0,9) • Optimal solutions: • (c,c) with pref. (7,7) • (n,c) with pref. (10,6) • (c,n) with pref. (6, 10)

32. Optimality in SCSPs, PE in Games • From SCSPs to Games • If we use the local mapping • Opt(P) ⊆ PE(L(P)) • If we use global mapping • Opt(P) = PE(L(P))

33. Summary :CP-nets and NE games 1-1 N’ not reduced g r g N reduced g(N) r n n(g(N))

34. Summary: SCSPs and Games • Nash Equilibria • Pareto Efficient = = SCSP Game CSP Game ⊆ x st.m. Game ⊆ Game local local SCSP SCSP global global Game ⊆ Game =

35. References • CP-nets and Soft Constraints • Carmel Domshlak, Steven David Prestwich, Francesca Rossi, Kristen Brent Venable, Toby Walsh: Hard and soft constraints for reasoning about qualitative conditional preferences. J. Heuristics 12(4-5): 263-285 (2006) • C. Boutilier, R. I. Brafman, Carmel Domshlak, H. H. Hoos, and D. Poole. Preference-based constraint optimization with CP-nets. Computational Intelligence, 20(2):137–157, 2004 • Games, CP-nets and Soft Constraints • Georg Gottlob, Gianluigi Greco, Francesco Scarcello: Pure Nash Equilibria: Hard and Easy Games. J. Artif. Intell. Res. (JAIR) 24: 357-406 (2005) • Krzysztof R. Apt, Francesca Rossi, K. Brent Venable,Comparing the notions of optimality in CP-nets, strategic games, and soft constraints, to appear in Annals of Mathematics and Artificial Intelligence.