Communication Networks

Communication Networks A Second Course Jean Walrand Department of EECS University of California at Berkeley

Repeated, Bargaining, Dynamic • Motivation • Repeated Games • Bargaining • Dynamic Games

Motivation • So far: One-shot (static) games • Many games are repeated or dynamic: • Repeated interactions (market, network usage) • Multiple stages (chess, card games) • Negotiations (bargaining) • New effects: • Past actions affect information • Reputation of players • Threats • Learning

Repeated Games • Example 1: Prisoners’ Dilemma • Example 2: Battle of the Sexes • Folk Theorem

Repeated Prisoners’ Dilemma • Model: • One-Shot: • (B, R) is the unique NE. • Intuition: • If players know the game is repeated, they might want to collaborate and play (T, L). • Finitely Repeated Game: • Assume game is repeated N times (N > 1). • Reward of each player is the sum of the step rewards. • Players see and remember the past actions. • Both players play simultaneously at each step. • Both players know the matrix of rewards.

Repeated Prisoners’ Dilemma • Definitions: • Strategy: Specifies what to do at each step, given available information. • Subgame Perfect Equilibrium: Pair of strategies from which no player has an incentive to deviate unilaterally. • Theorem: • The unique SPE for the N-time repeated PD is as follows: • Player 1 plays B at every step; • Player 2 plays R at every step. • Proof: Backward induction (obvious at last step, ...). 

Repeated Prisoners’ Dilemma • Infinitely Repeated Game: • Game is repeated forever • Reward of each player is the sum of the discounted step rewards: Ri = (1 – b)S0 bnRi(n) where 0 < b < 1. • Players see and remember the past actions. • Both players play simultaneously at each step. • Both players know the matrix of rewards.

Repeated Prisoners’ Dilemma • Definition: • r is in C  r = convex comb. of reward pairs and r dominates a NE • Theorem: • Consider the infinitely repeated PD. For any r in C, there is some b0 < 1 such that there is an SPE that enforces r ifb > b0.

Repeated Prisoners’ Dilemma Proof:Pick r is in C. Then r  rewards of playing the pair (ak,bk) of actions a fraction pk  Nk/N of the times, k = 1, …, 4. Strategy of player 1 [2, resp.]: We play (a1, b1) for N1 steps, then … (a4, b4) for N4 steps, and we repeat forever. Ifyou deviate at any time, I play B [R, resp.] forever thereafter.

Repeated Prisoners’ Dilemma Proof (continued):Reward of P2 at steps n, n +1 , … if he deviates at time n: < (1 – b)bn5 + bn+1 2 + e =: v if b is large enough.Reward of P2 at steps n, n +1 , … if he does not deviate: > (1 – b)bn1 + bn+1r2 – e =: w if b is large enough.Note that w > v for b large enough since r2 > 2.The strategy is an SPE: Say P2 deviates at time n. ThenP1 plays B forever and, knowing this, P2 must play R forever and P1 must accordingly play B forever since (B, R) is NE.

Repeated Prisoners’ Dilemma Comments: The SPE that enforces the rewards r is a “threat strategy.” The key step in the argument is to show that the threatstrategy is an SPE. In other words, the threat is “credible.” This is the case because the threat is torevert to playing a NE forever.

Repeated Battle of the Sexes • Theorem: • Consider the infinitely repeated PD. For any r in C, there is some b0 < 1 such that there is an SPE that enforces r ifb > b0.

Folk Theorem

Bargaining • Question:What is a reasonable agreement between two parties if negotiations are expensive? • Model:Alice and Bob bargain on how to divide a pie of value 1. After one step, the value of the pie gets multiplied by d1 < 1 for Alice and by d2 for Bob. Alice makes the initial offer which Bob accepts or refuses and makes an alternate offer, and so on. • Theorem: (Rubinstein-Stahl, 1972, 1982)SPE: Alice always demands the fractionx := (1 – d2)/(1 – d1d2)of the pie and Bob accepts an offer iff it is at leastz :=d2(1 – d1)/(1 – d1d2).

Bargaining • Theorem:(Rubinstein-Stahl, 1972, 1982)SPE: Alice always demands the fractionx1 := (1 – d2)/(1 – d1d2)of the pie and Bob accepts an offer iff it is at leastz2 :=d2(1 – d1)/(1 – d1d2). • Comments:z2 = 1 – x1is the smallest offer Bob accepts.Alice cannot gain by proposing a larger 1 – x1 to Bob.If Alice offers less than 1 – x1, Bob refuses and offersx2 = (1 – d1)/(1 – d1d2)so that Alice would getonlyd1 (1 – x2) = d12x1 < x1.

Bargaining • Theorem:SPE: Alice always demands the fractionx1 := (1 – d2)/(1 – d1d2)of the pie and Bob accepts an offer iff it is at leastz2 :=d2(1 – d1)/(1 – d1d2). • Proof:

Bargaining • Theorem:SPE: Alice always demands the fractionx1 := (1 – d2)/(1 – d1d2)of the pie and Bob accepts an offer iff it is at leastz2 :=d2(1 – d1)/(1 – d1d2). • Proof (continued):

Dynamic Games • Example 1 • Example 2 • Example 3

Dynamic Game: Example 1 Claim: 2 NEs = (L, L) and (R, R) If P1 does not choose L: Fact: Only one SPE: (R, R)

Dynamic Game: Example 2 SPE: P1: L P2: R P1: R if P2 = L L otherwise

Dynamic Game: Example 3 Equivalent matrix game Cannot solve.

Communication Networks