The equilibrium for the Prisoner’s dilemma

1. The equilibrium for the Prisoner’s dilemma • It is best for both to implicate regardless of what the other one does • Implicate is a Dominant Strategy for both • (Implicate, Implicate) becomes the Dominant Strategy Equilibrium • Note: If they might collude, then it’s beneficial for both to Not Implicate, but it’s not an equilibrium as both have incentive to deviate

2. Dominant Strategy Equilibrium • Dominant Strategy Equilibrium: is a strategy combination s*= (s1*, s2*, …,si*,…, sN*), such that si*is a dominant strategy for each i, namely, for any possible alternativestrategy profile s= (s1, s2, …, si , …, sN): • if pi is a utility, then pi(s1, s2,…, si*,…, sN) ≥ pi(s1, s2,…, si,…, sN) • if pi is a cost, then pi(s1, s2, …, si*, …, sN) ≤ pi(s1, s2, …, si, …, sN) • Dominant Strategy is the best response to any strategy of other players • If a game has a DSE, then players will immediately converge to it • Of course, not all games (only very few in the practice!) have a dominant strategy equilibrium

3. A more relaxed solution concept: Nash Equilibrium [1951] • Nash Equilibrium: is a strategy combination s*= (s1*, s2*, …, sN*) such that for each i, si*is a best response to (s1*, …,si-1*,si+1*,…, sN*), namely, for any possible alternativestrategy siof player i • if pi is a utility, then pi(s1*, s2*,…, si*,…, sN*) ≥ pi(s1*, s2*,…, si,…, sN*) • if pi is a cost, then pi(s1*, s2*, …, si*, …, sN*) ≤ pi(s1*, s2*, …, si, …, sN*)

4. Nash Equilibrium • In a NE no player can unilaterally deviate from his strategy given others’ strategies as fixed • Each player has to take into consideration the strategies of the other players • If a game has one or more NE, players need not to converge to it • Dominant Strategy Equilibrium  Nash Equilibrium (but the converse is not true)

5. Nash Equilibrium: The Battle of the Sexes (coordination game) • (Stadium, Stadium) is a NE: Best responses to each other • (Cinema, Cinema) is a NE: Best responses to each other  but they are not Dominant Strategy Equilibria … are we really sure they will eventually go out together????

6. A crucial issue in game theory: the existence of a NE • Unfortunately, for pure strategies games (as those seen so far, in which each player, for each possible situation of the game, selects his action deterministically), it is easy to see that we cannot have a general result of existence • In other words, there may be no any, just one, or many NE, depending on the game

7. A conflictual game: Head or Tail • Player I (row) prefers to do what Player II does, while Player II prefer to do the opposite of what Player I does!  In any configuration, one of the players prefers to change his strategy, and so on and so forth…thus, there are no NE!

8. On the existence of a NE • However, when a player can select his strategy randomly by using a probability distribution over his set of possible pure strategies (mixed strategy), then the following general result holds: • Theorem (Nash, 1951): Any game with a finite set of players and a finite set of strategies has a NE of mixed strategies (i.e., there exists a profile of probability distributions for the players such that the expectedpayoff of each player cannot be improved by changing unilaterally the selected probability distribution). • Head or Tail game: if each player sets p(Head)=p(Tail)=1/2, then the expected payoff of each player is 0, and this is a NE, since no player can improve on this by choosing unilaterally a different randomization!

9. Fundamental computational issues concerned with NE • Finding (efficiently) a mixed/pure (if any) NE • Establishing the quality of a NE, as compared to a cooperative system, namely a system in which agents can collaborate (recall the Prisoner’s Dilemma) • In a repeated game, establishing whether and in how many steps the system will eventually converge to a NE (recall the Battle of the Sexes) • Verifying that a strategy profile is a NE, approximating a NE, NE in resource (e.g., time, space, message size) constrained settings, breaking a NE by colluding, etc... (interested in a Thesis, or even in a PhD?)

10. Finding a NE in mixedstrategies • How do we select the correct probability distribution? It looks like a problem in the continuous… …but it’s not, actually! It can be shown that such a distribution can be found by selecting for each player a best possible subset of pure strategies (so-called best support), over which the probability distribution can actually be found by solving a system of algebraic equations (which are in general exponential in the number of players) In the practice, the problem can be solved by a simplex-like technique called the Lemke–Howson algorithm, which however is exponential in the worst case! Remark: Interestingly, for 2-player zero-sum games the solution can instead be found in polynomial time! • It can be shown that there exist games for which finding a NE in mixed strategies is a hard computational task

11. Is finding a NE in MS NP-hard? • W.l.o.g., werestrictourselves to 2-player games, letus call it2-NASH, and wewonderwhetherfinding a NE in mixedstrategies for 2-NASH (which may be thought in normal form as follows) is NP-hard s2,1 s2,2 … s2,m ____________________ s1,1 p1,p2 _____________________________ s1,2 _____________________________ . . ____________________ s1,n ______________________________

12. Is finding a NE in MS NP-hard? (2) • Recall: a decision problem P is in NP (resp., in coNP)if all its "yes"-instances (resp., “no”-instances) can be solved in polynomial time by a Non-Deterministic Turing Machine (NDTM) [Alternative definition for NP (resp., coNP): set of problems for which a "yes"-instance (resp., a “no”-instance) can be verified in polynomial time by a DTM] • Recall also: a problem P (not necessarily a decision one) is NP-hard if one can reduce in polynomial time any decision problem P’ in NP to it (this means, P’ can be solved in polynomial time on a NDTM by transforming it to P, in such a way that “yes”-instances of P’ maps to “yes”-instances of P, while “no”-instances of P’ maps to “no”-instances of P)  since all the instances of 2-NASH are “yes”-instances, one could solve in polynomial time a “no”-instance of P’on a NDTM by just showing that it does not map to a “yes”-instance of 2-NASH)  It turns out that if 2-NASH would be NP-hard then this would imply that NP = coNP(very hard to believe!)

13. The complexity class PPAD • Definition (Papadimitriou, 1994):PPAD (Polynomial Parity Argument – Directed case) is a subclass of TFNP (Total Function Nondeterministic Polynomial), where existence of a solution is guaranteed by a parity argument. Roughly speaking, PPAD contains all problems whose solution space can be set up as the (non-empty) set of all sinks in a suitable directed graph (generated by the input instance), having an exponential number of vertices in the size of the input, though. • Breakthrough: 2-NASH is PPAD-complete!!!(Chen & Deng, FOCS’06) • Remark: It could very well be that PPAD=PNP, but several PPAD-complete problems are resisting for decades to poly-time attacks (e.g., finding Brouwer fixed points)

14. Finding a NE in pure strategies • Exhaustive search: by definition, it is easy to see that an entry (p1,…,pN) of the payoff matrix is a NE if and only if pi is the maximum ith element of the row (p1,…,pi-1, {p(s):sSi} ,pi+1,…,pN), for each i=1,…,N. • However, with N players, an explicit (i.e., in normal-form) representation of the payoff functions is exponential in N • Exhaustive search for finding a pure NE is then exponential in the number of players (even if it is still polynomial in the input size, but the normal-form representation needs not be a minimal-space representation of the input!). • Alternative cheaper methods are sought: for many games of interest, a NE can be found in poly-time w.r.t. to the number of players (e.g., by using the powerful potential method). But the question is: does there exist a general method which guarantees to be always polynomial in N? • Aswewillsee, the answeris NO (unlessP=NP): in pure strategies, finding a NE is NP-hard for many games of interest (in this case, we can talk about NP-hardnesssince a solutionisnotguaranteed to exist)