Constraint Satisfaction Problems and Games

1. Constraint Satisfaction Problems and Games S Kameshwaran Oct 22, 2002

2. Outline • Introduction to CSP • Introduction to N-Person Non-cooperative Games • Nash Equilibrium revisited: Mixed Strategies • Mapping CSP to a Game • Tracing Procedure (Evolutionary Process) to find NE

3. CSP • Given a set of variables, find possible values to the variables which simultaneously satisfy a set of constraints • Applications: • Scheduling • Resource allocation • Computational Molecular Biology • Vision…

4. CSP • Given: • X: Variables ( X1, X2, …, Xn) • D: Domain Di for variable Xi • R: Set of constraints r (can be logical) • Find: • Solution: Assignment of value to Xi from its domain Di such that all constraints are satisfied

5. CSP • A constraint is called k-ary constraint if it connects k variables • k(r): arity of constraint r • (d1, d2, …, dn) r: The assignment di to Xi satisfies constraint r • Characteristic Function: r(d1, d2, …, dn) • 1 if (d1, d2, …, dn) r • 0 otherwise

6. CSP • Example: 8 Queens Problem • Place the 8 queens on the chess board such that no queen attacks the others • No two queens should be placed on the same row, or on the same column or on the same diagonal • 8 variables • Xi=j: Queen on ith row is placed on jth column • Constraint r: No two queens should be placed in the same column • Binary Constraint: k(r)=2 • Xi is not equal to Xj

7. CSP • Solution Approaches • Search algorithms • Backtracking • Forward checking • Graph based algorithms • Neural Networks

8. N-Person Non-cooperative Games • N players • Non-cooperative vs. Cooperative: : • Players cannot make binding commitments • Players join and split the gains out of cooperation • Solution concept: Nash Equilibrium

9. N-Person Non-cooperative Games • Normal Form Games • N players • Si=Strategy set of player i (Pure Strategy) • Single simultaneous move: each player i chooses a strategy siSi • Nobody observes others’ move • The strategy combination (s1, s2, …, sN) gives payoff (u1, u2, …, uN) to the N players • All the above information is known to all the players and it is common knowledge

10. Nash Equilibrium • Nash Equilibrium is a strategy combination s*= (s1*, s2*, …, sN*), such that si* is a best response to (s1*, …,si-1*,si+1*,…, sN*), for each i • (s1*, s2*, s3*) is a Nash Equilibrium (3 player game) iff • s1* is the best response of 1, if 2 chooses s2* and 3 chooses s3* • s2* is the best response of 2, if 1 chooses s1* and 3 chooses s3* • s3* is the best response of 3, if 1 chooses s1* and 2 chooses s2* • Note: It is a simultaneous game and nobody knows what exactly the choice of other agents • Nash Equilibrium assumes correct and consistent beliefs

11. Nash Equilibrium: Battle of the Sexes • (Prize Fight, Prize Fight) is a NE: Best responses to each other • (Ballet, Ballet) is a NE: Best responses to each other

12. The Welfare Game • Government wishes to aid a pauper if he searches for work but not otherwise • Pauper searches for work only if he cannot depend on government aid

13. The Welfare Game • (Aid, Try to Work) is not NE: Pauper prefers Be Idle

14. The Welfare Game • (Aid, Try to Work) is not NE: Pauper prefers Be Idle • (Aid, Be Idle) is not NE: Govt prefers No Aid

15. The Welfare Game • (Aid, Try to Work) is not NE: Pauper prefers Be Idle • (Aid, Be Idle) is not NE: Govt prefers No Aid • (No Aid, Be Idle) is not NE: Pauper prefers Try to Work

16. The Welfare Game • (Aid, Try to Work) is not NE: Pauper prefers Be Idle • (Aid, Be Idle) is not NE: Govt prefers No Aid • (No Aid, Be Idle) is not NE: Pauper prefers Try to Work • (No Aid, Try to Work) is not NE: Govt prefers Aid

17. Mixed Strategies • Pure Strategy: Player i chooses strategy sijfrom set Si • Mixed Strategy: Player i chooses strategy sij with probability qij (qij>=0, j qij=1) • Every pure strategy is also a mixed strategy • Payoff in mixed strategies is the expected payoff

18. Mixed Strategies: Advantages • Mathematical point of view: • Convexifies the set: Convex sets are nice to play around as the terrain is well understood

19. Mixed Strategies: Advantages • Mathematical point of view: • Convexifies the set: Convex sets are nice to play around as the terrain is well understood • Existence of Nash Equilibrium for finite games: Kakutani fixed point theorem

20. Mixed Strategies: Advantages • Mathematical point of view: • Convexifies the set: Convex sets are nice to play around as the terrain is well understood • Existence of Nash Equilibrium for finite games: Kakutani fixed point theorem • Practical point of view: • Yes and No (depends on the situation)

21. Mixed Strategies: Interpretation • Expected payoff: • Let payoff with strategy si1 be 1 and si2 be 4 • Mixed strategy (½, ½) gives the expected payoff ½+2=2.5

22. Mixed Strategies: Interpretation • Expected payoff: • Let payoff with strategy si1 be 1 and si2 be 4 • Mixed strategy (½, ½) gives the expected payoff ½+2=2.5 • It means a sure payoff of 2.5 is equivalent to a gamble where the payoffs are 1 and 4, each with probability ½

23. Mixed Strategies: Interpretation • Expected payoff: • Let payoff with strategy si1 be 1 and si2 be 4 • Mixed strategy (½, ½) gives the expected payoff ½+2=2.5 • It means a sure payoff of 2.5 is equivalent to a gamble where the payoffs are 1 and 4, each with probability ½ • The above interpretation will not make sense if the payoff is money • It is true only for utilities

24. Mixed Strategies: Interpretation • Games where multiple strategies can be simultaneously employed • Betting on more than one horse

25. Mixed Strategies: Interpretation • Games where multiple strategies can be simultaneously employed • Betting on more than one horse • Multiple instances of the same game • War Scenario: qij% of pilots use strategy sij

26. Mixed Strategies: Interpretation • Games where multiple strategies can be simultaneously employed • Betting on more than one horse • Multiple instances of the same game • War Scenario: qij% of pilots use strategy sij • Same game repeated infinitely

27. Mixed Strategies: Interpretation • Games where multiple strategies can be simultaneously employed • Betting on more than one horse • Multiple instances of the same game • War Scenario: qij% of pilots use strategy sij • Same game repeated infinitely • For a single game: The probability distribution is the opponents’ estimation of player i’s decision

28. CSP as Games • CSP C=< X, D, R > • Game induced by C: GC=(S1, …, Sn; U1, …,Un) • n = |X| • Si = Di • Ui(d1, …, dn) = rR[i]k(r)r(d1, d2, …, dn) • R[i] = Constraint set that includes variable i • The payoff function counts the number of satisfied constraints connecting that variable, taking every constraint along with its arity • Instead of arity one can use different weights

29. Equilibria and Solutions • Every solution of C is a Nash Equilibrium of GC • For a solution, all constraints are satisfied, so no agent can improve its payoff by assuming a different value • All Nash Equilibriums are not solutions • C may not have a solution but still GC will have a NE

30. Equilibria and Solutions

31. Equilibria and Solutions • Not a Solution

32. Equilibria and Solutions • Not a Solution • Nash Equilibrium: A better solution is not possible by moving a single queen in one move – Non-cooperative

33. Trivia: Non-cooperative Games • A solution better to some agents may be available, but cannot be reached by a decision of single agent alone • Cooperation or non-cooperation depends on the game • Non-cooperation need not be due to conflict in goal, but may be due to communication costs

34. Trivia: Non-cooperative Games • Prisoner’s Dilemma in reality: Should IISc water its garden when there is drought in Mysore? • Consider the following situation: Drought in Mysore but not in Bangalore • Saved water from Bangalore can be transported to Mysore • Decision making of an agent in Bangalore: • If all saves water, his saving will not contribute much • If nobody saves water, his saving will not contribute much • So, better not to save

35. Equilibria and Solutions • The complexity of the CSP depends on its structure • Finding solution to C Finding NE to GC • Complexity of finding NE is not known • It is unlikely to be in P • It is also unlikely to be NP-hard as existence of solution is guaranteed

36. Equilibrium Selection • Tracing Procedure (Evolutionary process) • For agent i, there is a probability distribution (mixed strategy) pi, which the other agents expect that i will use • p=(p1, …, pn) • Assumption • Limited computational power • Agents are Bayesian decision makers • Each agent estimates its best strategy depending on p • Value of p is updated based on the previous outcome

37. Equilibrium Selection • BR(p)=Best response strategy to the distribution p • Synchronous Process • p: Initial distribution • p0= p • pk=  BR(pk-1)+(1- )pk-1 • 0<<=1 • If pk converges then the limit point is NE

38. Equilibrium Selection • Computation of BR(p) for an agent is computationally taxing if it is connected with large number of variables • The process may converge to a NE that may not be a solution • Can be considered as the best possible quasi solution • No general proof that the process will always converge • Susceptible to initial probability distributions

39. Trivia: Solution to 8 Queens Problem • 10 distinct solutions • http://www.math.utah.edu/~alfeld/queens/queens.html

40. References • Equilibrium Theory and Constraint Networks, Francesco Ricci, 1992 • Games and Decisions, Luce and Raiffa, Dover Publications, 1957 • Games and Information: An Introduction to Game Theory, Eric Rasmusen, Basil Blackwell Publishers, 1989

41. Next.. • 25/10/02 Nash Equilibrium: P or NP?