Download
1 / 74
790 likes | 997 Views
Supermodular Games. Terminology. Partial Ordering. A relation is R is an order on a set S if the following three properties are satisfied for all x , y , z S : reflexive ( x R x ) transitive ( x R y R z x R z )
E N D
Partial Ordering A relation is R is an order on a set S if the following three properties are satisfied for all x, y, zS: reflexive (x R x) transitive (x R y R z x R z) antisymmetric (x R y R x x = y) The order is only partial order if there are some elements a, b S such that neither a R b nor b R a. A set S taken together with a partial order is a partially-ordered set.
Our relation of interest Let x and y denote two vectors in Let x y if xk yk for all k = 1,2,…,K Let x> y if x y and there exists some k such that xk> yk Example: y x y> x x = (1,0,0,3), y = (1,2,2,3) No relation x = (1,0,4,3), y = (1,2,2,3)
Two More Relations “meet of x and y” “join of x and y” The meet of x and y is the infimum of x and y The join x and y is the supremum of x and y Example: x = {1,0,0,3}, y = {1,2,2,3} x = {1,0,4,3}, y = {1,2,2,3} Note: if y x then
Sublattice Consider Si to be a subset (maybe convex) of . Form S as where Definition Sublattice A set is a sublattice if it is a partially ordered () subset of and if the operations and are closed on S. (i.e. if s, s* S then s s* S and s s* S) Sublattice? S ={(0,0), (0,0.5), (0.5,0), (1,0), (0,1)} no yes S ={(0,0), (0,1), (1,0), (1,1)} Sublattice property – Every bounded sublattice has a greatest and least element.
Increasing Differences Definition ui(si, s-i) has increasing differences in (si, s-i) if, for all such that In other words, an increase in the strategies of i’s rivals increases the value of playing a high strategy for player i.
Supermodular Function Definition ui(si, s-i) is supermodular in si if for each s-i Note if Si is single-dimensional, this is satisfied with equality.
Equivalent Formulations for all i and Siis a sublattice of A game is supermodular if A game is supermodular if
Best Response Properties Stated without proof
Nash Equilibrium Existence (Tarski) If A is a non-empty, compact sub-lattice of m and f : A A is non-decreasing, then f has a fixed point in A. Note upper-semi-continuity Would have to jump down across y=x, but that violates USC. A A has a fixed point (NE)
NE Properties (Topkis) A supermodular game for which each Aiis compact and each uiis u.s.c. in ai for each a-I, then the set of pure strategy NE is non-empty and contains greatest and least elements (Vives) Further the set of NE form a sub-lattice which is nonempty and complete.
More NE Properties Definition Best Response Dynamic At each stage, one player iN is permitted to deviate from ai to some randomly selected action bi Ai iff and (Milgrom and Roberts) A best response dynamic played on a supermodular game with compact action spaces and u.s.c. objective functions converges to a region bounded the greatest and least elements in the set of NE. If the NE is unique, then the best response dynamic converges to the NE.
Convergence of Adaptive Dynamics • The corollaries to Theorem 8 in [Milgrom_90] show that a smooth supermodular game following an adaptive dynamic process with any timing converges to a region bounded by the Nash equilibrium lattice .
Network description Each radio attempts to achieve a target SINR at the receiving end of its link. System objective is ensuring every radio achieves its target SINR Ad-hoc power control
Generalized repeated gamestage game • Players – N • Actions – • Utility function • Action space formulation gjkfraction of power transmitted by j that can’t be removed by receiving end of radio j’s link Nj noise power at receiving end of radio j’s link
Model identification & analysis • Supermodular game • Action space is a lattice • Implications • NE exists • Best response converges • Stable if discrete action space • Best response is also standard • Unique NE • Solvable (see prelim report) • Stable (pseudo-contraction) for infinite action spaces
Validation Implies all radios achieved target SINR Noiseless Best Response Noisy Best Response
Comments on Designing Networks with Supermodular Games • Scales well • Sum of supermodular functions is a supermodular function • Add additional action types, e.g., power, frequency, routing,..., as long as action space remains a lattice and utilities are supermodular • Says nothing about desirability or stability of equilibria • Convergence is sensitive to the specific decision rule and the ability of the radios to implement it
Potential Games • Existence of a function (called the potential function, V), that reflects the change in utility seen by a unilaterally deviating player. • Cognitive radio interpretation: • Every time a cognitive radio unilaterally adapts in a way that furthers its own goal, some real-valued function increases. () time
Exact Potential Games • Definitions, Existence, Basic Properties • Examples • Path Properties
Exact Potential Games DefinitionExact Potential Game A normal form game whose objective functions are structured such that there exists some function P: A which satisfies the following property for all players: In other words it must be possible to construct a single-dimensional function whose change in value is exactly equal to the change in value of the deviating player.
a2 b2 a1 1,1 0, 0 b1 3, 3 0, 0 Example Potential Game (1/2) Coordination Game u1(a1,a2) - u1(b1,a2) = 1 = V(a1,a2) - V(b1,a2) Note: V is not unique. Consider V’ = V + c where c is a constant. Also note the relation between CG Prop. 2 and V u2(a1,a2) – u2(a1,b2) = 1 = V(a1,a2) - V(a1,b2) u1(b1,b2) - u1(a1,b2) = 3 = V(b1,b2) - V(a1,b2) u2(b1,b2) – u2(b1,a2) = 3 = V(b1,b2) - V(b1,a2)
Example Potential Game (2/2) Coordination Game (In Equilibriums) a2 b2 a1 4,2 -1, 1 b1 2, 1 3,-2 u1(a1,a2) - u1(b1,a2) = 1 = V(a1,a2) - V(b1,a2) The Same Potential!! The Same NE! u2(a1,a2) – u2(a1,b2) = 1 = V(a1,a2) - V(a1,b2) u1(b1,b2) - u1(a1,b2) = 3 = V(b1,b2) - V(a1,b2) u2(b1,b2) – u2(b1,a2) = 3 = V(b1,b2) - V(b1,a2)
a2 b2 a1 1,1 0, 0 b1 3, 3 0, 0 Comments on Second Example Dummy Game Coordination Game Second Game a2 a2 b2 b2 a1 a1 4,2 -1, 1 3,1 -1, 1 b1 b1 2, 1 3,-2 -1,-2 3,-2 As we shall see, this is a property of all exact potential games. Also a potential function for an exact potential game is always equal to the characteristic function (plus a constant) of its constituent coordination game.
EPG Property 1 (Voorneveld) A game G = <N, {Ai}iN , {ui}iN> is an exact potential game iff there exist functions {ci}iN and {di}iN such that • ui = ci + di • <N, {Ai}iN , {ci}iN> is a coordination game • <N, {Ai}iN , {di}iN> is a dummy game Outline of proof: if: The characteristic function of the coordination game is an exact potential function of G Only if: Let P be an exact potential of G. Clearly P forms a coordination game. Now consider a game with objective fcns given by ui – P. As the value of deviating in this game is now 0 at all points, this is a dummy game.
EPG Property 2 The NE of an exact potential game are coincident with the NE of its constituent coordination game. Outline of Proof Any unilateral deviation in a dummy game yields the same payoff. Adding a dummy game D to another game G preserves G’s NE. All exact potential games can be expressed As the sum of a coordination game and a dummy game (EPG Property 1). Therefore the NE of the potential game must be the same as the NE of the coordination game.
EPG Property 3 (Voorneveld) For an EPG, the maximizers of the EPF are NE of the EPG. Outline of Proof The NE of an EPG are the NE of its coordination game (CG). By CG Property 3, the maximizers of its characteristic functions (V) are NE. All EPF can be expressed as V constant c. Since the addition of the constant does not change which tuples yield maximum payoffs, the maximizers of the EPF are coincident with the maximizers of V, thus coincident with the NE of the CG, thus coincident with the NE of the EPG.
EPG Property 4 (Voorneveld) Let the EPG be finite (finite action space, finite player set), then the EPG has at least one pure-strategy NE. Outline of Proof Note these conditions mean that the EPF must have at least one maximum. By EPG Property 4, this must be a NE.
Continuous Action Sets (1/2) EPG Property 5 (Shapley) Let G be a game in which the strategy sets are closed intervals of . Suppose the objective functions are continuously differentiable. A function P is a potential iff P is continuously differentiable and for every i N EPG Property 6 (Shapley) If objective functions are twice differentiable then a game is a EPG iff for every i, j N
Continuous Action Sets (2/2) EPG Properties 1-4 also hold for continuous closed action sets. Proofs follow in exactly the same manner.
Vector Operations Consider the set of EPG {EPG1, EPG2,…,EPGK}, with player set N and action space A, and objective functions , and potential functions . Form a new game, G, with player set N, action space A, and objective functions given by . Then G is an EPG with an EPF given by Note: this means that the set of EPG formed from a particular N and A form a vector topological space (closed under addition and scalar multiplication).
Exact Potential Game Forms • Many exact potential games can be recognized by the form of the utility function
Coordination – Dummy Game As previously stated, all EPG are formed from the sum of a coordination game and a dummy game so this is only here for completeness. Consider a game G = <N, (Ai)iN , {ui}iN> such that ui = ci + di where <N, (Ai)iN , {ci}iN> is a coordination game with characteristic function V(a) and <N, (Ai)iN , {di}iN> is a dummy game. This game has an EPF given by V(a)
Bilateral Symmetric Interaction Game Introduced in Ui A strategic form game where each player’s objective function is a sum of bilateral symmetric interaction (BSI) terms. A BSI term such that for every . The objective function is expressed as An EPF for this game is given by
Self-Motivated Game A strategic form game where each player’s objective function is a function solely of their own action, i.e. This has an EPF given by Note this is not really a game (no interaction), but it is often encountered as a component of more complex games.
Cournot Oligopoly (1/2) Cournot oligopoly characterized by real interval action sets and objective function given by Note that So a potential exists
Cournot Oligopoly (2/2) Now rewrite the objective function as Note that this is just a BSI game. So a potential can be written as
a2 b2 a1 w, w x, y a2 a2 b2 b2 b1 z, z y, x a1 a1 z-x+y, z-x+y z, z-x+y x-z+w-y x-z b1 b1 z-x+y, z z, z x-z 0 Prisoners’ Dilemma
Ordinal Potential Games • Definitions, Existence, Properties • Examples • Applications
Ordinal Potential Games DefinitionOrdinal Potential Game (OPG) A normal form game whose objective functions are structured such that there exists some function P: A which satisfies the following property for all players: In other words it must be possible to construct a single-dimensional function where the sign of the change in value is the same as the sign of the change in value of the deviating player. Note that an EPG also satisfies this definition.
Example Ordinal Potential Game Not a Coordination Game a2 b2 a1 0,0 1,1 Note: P is not unique. Consider P’ = c2P + c1 b1 0,1 2,0 sgn(u1(a1,a2) - u1(b1,a2)) = - = sgn(P(a1,a2) - P(b1,a2)) sgn(u2(a1,a2) – u2(a1,b2)) = - = sgn(P(a1,a2) - P(a1,b2)) sgn(u1(b1,b2) - u1(a1,b2)) = - = sgn(P(b1,b2) - P(a1,b2)) sgn(u2(b1,b2) – u2(b1,a2)) = + = sgn(P(b1,b2) - P(b1,a2))
Properties shared with EPG (Shapley) For an OPG, the maximizers of the OPF are NE of the OPG. An OPG has at least one pure-strategy NE. An finite OPG has FIP. An OPG with continuous bounded action sets has AFIP. A repeated game with the same OPG stage also converges with a better response dynamic.
Cycles Consider a cycle , the sum of the changes in value seen by the deviating players in an OPG is not always 0. a2 • = ((a1, a2), (b1, a2), (b1, b2), (a1, b2), (a1, a2)) b2 a1 0,0 1,1 I(, u) = 2 + 1 + 1 – 1 = 3 b1 0,1 2,0
Weak Improvement Cycles No known simple necessary and sufficient condition like the second derivative condition of EPG. Non-deteriorating path A path is an non-deteriorating path if for all k 1 Weak improvement cycle A finite non-deteriorating path = (a0, a1,…,ak) where ak = a0 (Voorneveld) All OPG lack weak improvement cycles.
Properties not shared with an EPG The set of OPG is not a vector space. a2 a2 a2 b2 b2 b2 a1 a1 a1 1,2 2,1 1,2 1,0 0,0 1,1 b1 b1 b1 0,2 0,1 2,0 0,1 0,0 2,0 Improvement Cycle a2 a2 b2 b2 • = ((a1, a2), (b1, a2), (b1, b2), (a1, b2)) a1 a1 3 0 2 3 b1 b1 1 0 2 1 Still closed under scalar multiplication though.
