Comparing the notions of optimality in strategic games and soft constraints (and CP nets)

1. Comparing the notions of optimality in strategic games and soft constraints(and CP nets) K. R. Apt*, F. Rossi**, K. B. Venable** * CWI and Univ. of Amsterdam ** University of Padova, Italy

2. Main aim • To compare the notion of optimality used in many formalisms • To exploit results in one field and reuse them in the other field (ex. computational results) • Strategic games • Agent interaction while pursuing their own interest (payoff function) • CP nets • Agent’s qualitative and conditional preferences • Soft constraints • Agent’s quantitative preferences

3. Outline • CP nets • Strategic games • Relation between CP nets and games • Soft constraints • Relation between soft constraints and games

4. CP statements • Conditional ceteris paribus preference statements • Example: • If there is fish, I prefer white wine to red all else being equal • Written asfish: white wine > red wine • Ceteris paribus interpretation • {fish, white wine, peaches} > {fish, red wine, peaches} • {fish, white wine, peaches} >? {fish, red wine, strawberries}

5. CP-nets • Graphical representation of a set of CP statements • A directed graph over features X1, … , Xn • each feature may depend on some others (its parents): arc from parents to child • A set of conditional preference tables • Each row is a cp statement for Xi given values for its parents • Complete tables: one row for each instatiation of the parents • Total order on domain of Xi Ref.: Boutilier, Brafman, Hoos, Poole,1999.

6. Main course Independent feature Dependent feature Wine Independent feature Fruit CP net: an example fish>meat Conditional Preference Table peaches > strawberries

7. (fish, white wine, peaches) worsening flip (fish, red wine, peaches) CP-net semantics • Worsening flip: changing the value of an attribute in a way that is less preferred in some statement. Example: • An outcome O1 is preferred to O2 iff there is a sequence of worsening flips from O1 to O2 • Ordering induced: preorder • Optimal outcomes: no flip brings to a better outcome

8. Fish, white, peaches Main course Fish, red, peaches Fish, white, berries Fish, red, berries Wine meat, red, peaches meat, red, berries meat, white, peaches Fruit meat, white, berries Example fish>meat peaches > strawberries

9. Finding optimal outcomes in CP nets • From a CP-net to a set of hard constraints: • For each cp statement A: b1>b2, we get the constraint a → b1 (a implies b) • An outcome is optimal for the CP-net iff it is a solution of these constraints • Acyclic CP-net • No cycles in the graph • Linear time: sweep forward in the topological order

10. Strategic games • A set of players 1,.., n • For each player i: • A set of strategies Si • A total order iover Si depending on s-i (a joint strategy of all players but player i): payoff function • Example (prisoner’s dilemma): 2 players, 2 strategies (ci, ni) for each player i

11. Nash equilibria • A strategy si is a best response for i to s-i if sii s’i for all s’i in Si • A joint strategy s is a Nash equilibrium if each si is a best response to s-i • Also: for all i, for all s’i in Si, si i s’i • No player has regrets on the strategy he chose • But there could be better joint strategies if more than one player changed its strategy • In the example, one Nash equilibria (NE): (N1,N2)

12. Pareto efficient joint strategies • No other joint strategy is better or equal for all agents, and better for at least one • Example: • (N1,N2): unique Nash equilibrium • All other joint strategies are Pareto efficient (PE)

13. From CP-nets to games • Given a CP-net N, we build the game g(N) • Players: features • Strategies of player i: domain of feature xi • Payoff function of player i: CP table for xi • Given s-i, s’i >i si iff s-i|par(xi) : s’i >i si in the cp table for variable i • Thm: opt(N) = NE(g(N))

14. Fish, white, peaches Main course Fish, red, peaches Fish, white, berries Fish, red, berries Wine meat, red, peaches meat, red, berries meat, white, peaches Fruit meat, white, berries Example – CP net fish>meat peaches > strawberries

15. Example – players and strategies • Three players: 1 = main course, 2 = wine, 3 = fruit • Two strategies for each player: • 1: meat, fish • 2: red, white • 3: peaches, strawberries

16. Example: payoff functions • For wine: • fish, --  white > red • meat, --  red > white • For main course: • --, --  fish > meat • For fruit: • --, --  peaches > strawberries

17. Example: optimals and Nash equilibria • Unique optimal for CP-net: (fish, white, peaches) • For the game: • Nash equilibrium: (fish, white, peaches)

18. From games to CP-nets • Given a game G, we build a CP-net n(G): • Feature xi: player i • Domain of xi: strategies for player i • Parents of xi: all the other features • CP table of xi: s-i: si > s’i if si >i s’i given s-i • Thm.: NE(G) = opt(n(G))

19. Example • Two features: x1, x2 • D(x1)={c1, n1} • D(x2)={c2,n2} • x1 depends on x2 • x2=c2: n1 > c1 • x2=n2: n1 > c1 • x2 depends on x1 • X1=c1: n2 > c2 • X1=n1: n2 > c2 • Hard constraints: • x2=c2 → x1=n1 • x2=n2 → x1=n1 • x1=c1 → x2=n2 • x1=n1 → x2=n2 • Unique solution: x1=n1, x2=n2

20. Reduced CP-nets • If y is a parent of x, but the preference over the domain of x does not depend on y, then we can remove y from the parents of x  eliminate rows • From a CP net N to its reduced version r(N)

21. Two features: x1, x2 D(x1)={c1, n1} D(x2)={c2, n2} x1 depends on x2 x2=c2: n1 > c1 x2=n2: n1 > c1 x2 depends on x1 X1=c1: n2 > c2 X1=n1: n2 > c2 Two features: x1, x2 D(x1)={c1, n1} D(x2)={c2, n2} x1 and x2 independent For x1: n1 > c1 For x2: n2 > c2 Example: reduced CP-net

22. n G n(G) r r(n(G)) Nash equilibria of G = optimals of r(n(G)) CP-net techniques in games • From game G to n(G) • Hard constraints for r(n(G)) • Optimals of r(n(G)) = Nash equilibria of G

23. n G n(G) r Nash equilibrium of G = optimal of r(n(G)) r(n(G)) acyclic linear time Games and acyclic CP-nets • From game G to r(n(G)) • If r(n(G)) is acyclic, then G has one Nash equilibrium, and linear time to find it

24. Soft Constraints: the c-semiring framework • Variables{X1,…,Xn}=X • Domains{D(X1),…,D(Xn)}=D • Soft constraints • each constraint involves some of the variables • a preference is associated with each assignment of the variables • Set of preferences A • Totally or partially ordered (induced by +) • Combination operator (x) • Top and bottom element (1, 0) • Formally defined by a c-semiring <A,+,x,0,1>

25. Soft constraints • Soft constraint: a pair c=<f,con> where: • Scope: con={Xc1,…, Xck} subset of X • Preference function : f: D(Xc1)x…xD(Xck) → A tuple (v1,…, vk) → p preference • Hard constraint: a soft constraint where for each tuple (v1,…, vk) f (v1,…, vk)=1 the tuple is allowed f (v1,…, vk)=0 the tuple is forbidden

26. Solutions • Solution: assignment to all variables • Preference of a solution: • Combination (via x) of the preferences of the partial assignments given by the constraints • Optimal solutions: those with the highest preference

27. Some instances of soft constraints • Each instance is characterized by a c-semiring <A, +, x, 0, 1> • Classical constraints: <{0,1}, logical or, logical and, 0, 1> • Satisfy all constraints • Fuzzy constraints: <[0,1], max, min, 0, 1> • Maximize the minimum preference • Weighted constraints (N):<N+, min, +, +, 0> • Minimize the sum of the costs

28. X Y Z (a,a)  0.4 (a,b)  0.1 (b,a)  0.3 (b,b)  0.5 (a,a)  0.4 (a,b)  0.3 (b,a)  0.1 (b,b)  0.5 Example • Fuzzy CSPs • maximize the minimum preference • Optimal solution: x=y=z=b, with preference 0.5

29. Combination operator • Extensive (always): for all a,b in A, a x b  a,b • Idempotent: for all a in A, a x a = a • Ex.: max, min, and • Ex. of instances: fuzzy, classical • It is possible to apply soft constraint propagation • Strictly monotonic: for all a,b in A, a x b • Ex.: sum, product • Ex. of instances: weighted • It cannot be idempotent and strictly monotonic at the same time

30. From soft CSPs to games: a local approach • From a soft CSP P to a game L(P) • Players: one for each variable • Strategies for a player i: all values in domain of xi • Payoff of player i for joint strategy s: preference for assignment s in constraints involving xi

31. Example X Y Z • Three players x,y,z • Two strategies a,b • Payoff functions • For x: p(aa-)=0.4, ... (same for z) • For y: • p(aaa) = min(0.4,0.4) = 0.4 • p(aab) = min(0.4,0.3)=0.3 • ... • Two Nash equilibria: aaa and bbb • Optimal solutions: only bbb (a,a)  0.4 (a,b)  0.1 (b,a)  0.3 (b,b)  0.5 (a,a)  0.4 (a,b)  0.3 (b,a)  0.1 (b,b)  0.5

32. Strictly monotonic combination • In general, no relationship between optimal solutions of P and Nash equilibria of L(P) • However, some relationship exist if combination is strictly monotonic • Thm.:Soft CSP P with strictly monotonic combination  Opt(P)  NE(L(P))

33. Classical CSPs • Classical constraints are combined via logical and (which is not strictly monotonic) • However, if we consider consistent CSPs, the result holds • Thm.: consistent CSP  Sol(P)  NE(L(P))

34. A global mapping • Given an SCSP P, build a game GL(P): • Players = variables • Strategies = domain values • Payoff for player x for strategy s: preference value for that assignment (by looking at all constraints) • Note: same payoff for all players • Theorem: Opt(P)  NE(GL(P)) • Subset relation for all classes of SCSPs

35. From strategic games to SCSPs • From a game G to an SCSP L’(G): • Variables = players (n) • Domains = strategies • Semiring = Cartesian product of n semirings • For each variable xi, one constraint involving xi and its neighbourhood • pref(t) = (d1,...,dn), where dj = 1j for j  i, and di = pi(t) • Thm.: Game G  opt(L’(G)) = PE(G)

36. (c1,c2)  (0,7) (c1,n2)  (0,6) (n1,c2)  (0,10) (n1,n2)  (0,9) X1 x2 (c1,c2)  (7,0) (c1,n2)  (10,0) (n1,c2)  (6,0) (n1,n2)  (9,0) Example Semiring: weighted x weighted • Pareto efficient joint strategies: all but (1,1) • Optimal solutions: • (c,c) with pref. (7,7) • (n,c) with pref. (10,6) • (c,n) with pref. (6, 10)

37. Computational issues • If opt(P) = NE(G) or opt(P)=PE(G) • Exploit the SCSPs/CP net machinery and tractability results to compute exactly NEs and PEs • If opt(P)  NE(G) • finding Opt(P) gives a lower approximation of NE(G)

38. Summary of main results • Optimal outcomes in CP nets are Nash equilibria in corresponding strategic games • Same mapping for soft constraints  • In general, no relation between optimals and NE • If strictly monotonic SCSPs  optimals  NE • Global mapping  optimals  NE • From games to SCSPs: • Optimals = Pareto efficient joint strategies