A study of Correlated Equilibrium Polytope

1. A study of Correlated Equilibrium Polytope By: Jason Sorensen

2. The set of Correlated Equilibrium is a polytope • Polytope has two equivalent definitions: 1.)The Convex hull of a finite set of points in R^n 2.)The bounded intersection of a finite set of closed half-spaces • The correlated equilibrium constraints are • Nash Equilibrium are Correlated -> non-empty

3. Linear programming • Can find maximal correlated equilibrium by linear programming • Use constraints given, player i maximizing hyperplane is • Easily solved by any LP program for reasonable sized games

4. The Shapley game • 12 (Non-trivial) LP constraints • 6 of them reduce to x1 = x2 = x5 = x6 = x7 = x9 • Reduces original 8 dimensional polytope to 9 dimensions • Unique Nash Equilibrium at xi = 1/9 • This corresponds to LP minimum for all utility hyperplanes

5. The Shapley polytope

6. Chicken or Dare? • Game matrix is: • 3 Nash Equilibria at x3 = 1, x2 = 1, and one mixed • Each Nash Equilibirium is an LP minimum for a utility hyperplane • Can reduce polytope dimension to 3 by utilizing equality constraint

7. The Chicken Polytope

8. Theorem time! • All Nash Equilibrium lie on the boundary of the Correlated Equilibrium • Proof on the board! • … But may not by too great if polytope is not full dimensional (boundary != relative boundary)

9. Are CE always better than NE? • In the two cases we studied, all utility hyperplanes were minimized at an NE • Is this generally the case? • NO! • In fact, there is no way to find NE by linear systems in general

10. Correlated Equilibrium get Bested • Game of “Poker” (?!) • Unique Nash Equilibrium - with irrational coordinates • NE does not occur at a vertex (all vertices are rational) • Value of NE for player one: .890 • Value of worst case CE for player one: .5833 • Value of best case CE for player one: 1.467 • The CE Polytope is full 7 dimensional (not graphable)

11. Putting it all together • We can guarantee convergence to CE by natural (no-regret) learning processes • But CE may not always be better than Nash Equilibrium • Which CE do these “natural learning processes” converge to? • How long does convergence take? • Will investigate further in next 2 weeks

12. A Conundrum • Using learning processes to find optimal responses may result in being “bullied” by opponent (it’s always the smart ones who get picked on) • In chicken or dare, if the opponent is rational and knows you are learning, there is no reason not to always play dare • How do we decide whether to learn or bully for optimal payoffs? • Each “playing strategy” (learning or not) combated against each other in a game results in a certain payoff for each player after convergence • Model this situation as a new game, where each “move” is a learning strategy and learn the optimal strategy • Have we really learned anything?

13. Open Problems • Compare learning strategies to figure out optimal method • Figure out properties of general game polytopes (the number of faces) • In which situations is the polytope full dimensional? • In which situations are the NE the LP minimizing vertices for all utility hyperplanes?