Introduction to Game Theory

1. Introduction to Game Theory By: Mohammad Amin Latifi Tarbiat modares university-January 2008

2. Outline Definition of Game Theory Types of the Game Static Game of Complete Information Dynamic Game of Complete Information Static and Dynamic Games of Incomplete Information Application of Game Theory in Power System

3. What is game theory? Game theory is a formal way to analyze strategic interaction among a group of rational players (or agents) who behave strategically Game theory has applications Economics Politics etc.

4. Game Theory by Wikipedia Game theory is a branch of applied mathematics that is often used in the context of economics. It studies strategic interactions between agents. In strategic games, agents choose strategies which will maximize their return, given the strategies the other agents choose. The essential feature is that it provides a formal modeling approach to social situations in which decision makers interact with other agents. Game theory extends the simpler optimisation approach developed in neoclassical economics.

5. Rationality Assumptions: humans are rational beings humans always seek the best alternative in a set of possible choices Why assume rationality? narrow down the range of possibilities predictability

6. Why is game theory important? All intelligent beings make decisions all the time. AlI needs to perform these tasks as a result. Helps us to analyze situations more rationally and formulate an acceptable alternative with respect to circumstance. Useful in modeling strategic decision-making Provides structured insight into the value of information

7. History of Game Theory Timeline

8. Game Theory Elements All games have three elements players strategies payoffs

9. Classic Example: Prisoners’ Dilemma Two suspects held in separate cells are charged with a major crime. However, there is not enough evidence. Both suspects are told the following policy: If neither confesses then both will be convicted of a minor offense and sentenced to one month in jail. If both confess then both will be sentenced to jail for six months. If one confesses but the other does not, then the confessor will be released but the other will be sentenced to jail for nine months.

10. Example: The battle of the sexes At the separate workplaces, Chris and Pat must choose to attend either an opera or a prize fight in the evening. Both Chris and Pat know the following: Both would like to spend the evening together. But Chris prefers the opera. Pat prefers the prize fight.

11. Types of Games Sequential vs. Simultaneous moves Single Play vs. Iterated Zero vs. non-zero sum Perfect vs. Imperfect information Cooperative vs. conflict

12. Game types in this Presentation Static games of complete information Nash equilibrium Dynamic games of complete information Subgame-perfect Nash equilibrium Static games of incomplete information Bayesian Nash equilibrium Dynamic games of incomplete information Perfect Bayesian equilibrium

13. Static Games of Complete Information

14. Static (or simultaneous-move) games of complete information A set of players (at least two players) For each player, a set of strategies/actions Payoffs received by each player for the combinations of the strategies, or for each player, preferences over the combinations of the strategies {Player 1, Player 2, ... Player n} S1 S2 ... Sn ui(s1, s2, ...sn), for all s1?S1, s2?S2, ... sn?Sn.

15. Static (or simultaneous-move) games of complete information Simultaneous-move Each player chooses his/her strategy without knowledge of others’ choices. Complete information Each player’s strategies and payoff function are common knowledge among all the players. Assumptions on the players Rationality Players aim to maximize their payoffs Players are perfect calculators Each player knows that other players are rational

16. Normal-form representation: 2-player game Bi-matrix representation 2 players: Player 1 and Player 2 Each player has a finite number of strategies Example:S1={s11, s12, s13} S2={s21, s22}

17. Classic example: Prisoners’ Dilemma:normal-form representation Set of players: {Prisoner 1, Prisoner 2} Sets of strategies: S1 = S2 = {Mum, Confess} Payoff functions: u1(M, M)=-1, u1(M, C)=-9, u1(C, M)=0, u1(C, C)=-6;u2(M, M)=-1, u2(M, C)=0, u2(C, M)=-9, u2(C, C)=-6

18. Solving Prisoners’ Dilemma Confess always does better whatever the other player chooses Dominated strategy There exists another strategy which always does better regardless of other players’ choices

19. Definition: strictly dominated strategy

20. Definition: weakly dominated strategy

21. Strictly and weakly dominated strategy A rational player never chooses a strictly dominated strategy. Hence, any strictly dominated strategy can be eliminated. A rational player may choose a weakly dominated strategy.

22. Iterated elimination of strictly dominated strategies If a strategy is strictly dominated, eliminate it The size and complexity of the game is reduced Eliminate any strictly dominated strategies from the reduced game Continue doing so successively

23. Iterated elimination of strictly dominated strategies: an example

24. Nash Equilibrium: idea Nash equilibrium A set of strategies, one for each player, such that each player’s strategy is best for her, given that all other players are playing their equilibrium strategies

25. Definition: Nash Equilibrium

26. Cournot model of duopoly A product is produced by only two firms: firm 1 and firm 2. The quantities are denoted by q1 and q2, respectively. Each firm chooses the quantity without knowing the other firm has chosen. The market priced is P(Q)=a-Q, where a is a constant number and Q=q1+q2. The cost to firm i of producing quantity qi is Ci(qi)=cqi.

27. Cournot model of duopoly The normal-form representation: Set of players: { Firm 1, Firm 2} Sets of strategies: S1=[0, +8), S2=[0, +8) Payoff functions: u1(q1, q2)=q1(a-(q1+q2)-c)u2(q1, q2)=q2(a-(q1+q2)-c)

28. Cournot model of duopoly How to find a Nash equilibrium Find the quantity pair (q1*, q2*) such that q1* is firm 1’s best response to Firm 2’s quantity q2* and q2* is firm 2’s best response to Firm 1’s quantity q1* That is, q1* solves Max u1(q1, q2*)=q1(a-(q1+q2*)-c)subject to 0 ? q1 ? +8and q2* solvesMax u2(q1*, q2)=q2(a-(q1*+q2)-c)subject to 0 ? q2 ? +8

29. May 22, 2003 73-347 Game Theory--Lecture 4 29 Cournot model of duopoly How to find a Nash equilibrium The quantity pair (q1*, q2*) is a Nash equilibrium ifq1* = (a – q2* – c)/2 q2* = (a – q1* – c)/2 Solving these two equations gives us q1* = q2* = (a – c)/3

30. Cournot model of oligopoly How to find a Nash equilibrium Find the quantities (q1*, ... qn*) such that qi* is firm i’s best response to other firms’ quantities That is, q1* solves Max u1(q1, q2*, ..., qn*)=q1(a-(q1+q2* +...+qn*)-c)subject to 0 ? q1 ? +8and q2* solvesMax u2(q1*, q2 , q3*, ..., qn*)=q2(a-(q1*+q2+q3*+ ...+ qn*)-c)subject to 0 ? q2 ? +8.......

31. Bertrand model of duopoly (differentiated products) Two firms: firm 1 and firm 2. Each firm chooses the price for its product without knowing the other firm has chosen. The prices are denoted by p1 and p2, respectively. The quantity that consumers demand from firm 1: q1(p1, p2) = a – p1 + bp2. The quantity that consumers demand from firm 2: q2(p1, p2) = a – p2 + bp1. The cost to firm i of producing quantity qi is Ci(qi)=cqi.

32. Bertrand model of duopoly (differentiated products) The normal-form representation: Set of players: { Firm 1, Firm 2} Sets of strategies: S1=[0, +8), S2=[0, +8) Payoff functions: u1(p1, p2)=(a – p1 + bp2 )(p1 – c)u2(p1, p2)=(a – p2 + bp1 )(p2 – c)

33. Bertrand model of duopoly (differentiated products) How to find a Nash equilibrium Find the price pair (p1*, p2*) such that p1* is firm 1’s best response to Firm 2’s price p2* and p2* is firm 2’s best response to Firm 1’s price p1* That is, p1* solves Max u1(p1, p2*) = (a – p1 + bp2* )(p1 – c)subject to 0 ? p1 ? +8and p2* solvesMax u2(p1*, p2) = (a – p2 + bp1* )(p2 – c)subject to 0 ? p2 ? +8

34. Bertrand model of duopoly (differentiated products) How to find a Nash equilibrium The price pair (p1*, p2*) is a Nash equilibrium if p1* = (a + c + bp2*)/2  p2* = (a + c + bp1*)/2 Solving these two equations gives us p1* = p2* = (a + c)/(2 –b)

35. Bertrand model of duopoly (homogeneous products) Two firms: firm 1 and firm 2. Each firm chooses the price for its product without knowing the other firm has chosen. The prices are denoted by p1 and p2, respectively. The quantity that consumers demand from firm 1: q1(p1, p2) = a – p1 if p1 < p2 ; = (a – p1)/2 if p1 = p2 ; =0, ow. The quantity that consumers demand from firm 2: q2(p1, p2) = a – p2 if p2 < p1 ; = (a – p2)/2 if p1 = p2 ; =0, ow. The cost to firm i of producing quantity qi is Ci(qi)=cqi.

36. Mixed strategy A mixed strategy of a player is a probability distribution over player’s (pure) strategies. A mixed strategy for Chris is a probability distribution (p, 1-p), where p is the probability of playing Opera, and 1-p is that probability of playing Prize Fight. If p=1 then Chris actually plays Opera. If p=0 then Chris actually plays Prize Fight.

37. Mixed strategy equilibrium Mixed Strategy: A mixed strategy of a player is a probability distribution over the player’s strategies. Mixed strategy equilibrium A probability distribution for each player The distributions are mutual best responses to one another in the sense of expected payoffs

38. Chris’ expected payoff of playing Opera: 2q Chris’ expected payoff of playing Prize Fight: 1-q Chris’ best response B1(q): Prize Fight (r=0) if q<1/3 Opera (r=1) if q>1/3 Any mixed strategy (0?r?1) if q=1/3 Battle of sexes

39. Pat’s expected payoff of playing Opera: r Pat’s expected payoff of playing Prize Fight: 2(1-r) Pat’s best response B2(r): Prize Fight (q=0) if r<2/3 Opera (q=1) if r>2/3 Any mixed strategy (0?q?1) if r=2/3, Battle of sexes

40. Chris’ best response B1(q): Prize Fight (r=0) if q<1/3 Opera (r=1) if q>1/3 Any mixed strategy (0?r?1) if q=1/3 Pat’s best response B2(r): Prize Fight (q=0) if r<2/3 Opera (q=1) if r>2/3 Any mixed strategy (0?q?1) if r=2/3 Battle of sexes

41. Dynamic Games of Complete Information

42. Entry game An incumbent monopolist faces the possibility of entry by a challenger. The challenger may choose to enter or stay out. If the challenger enters, the incumbent can choose either to accommodate or to fight. The payoffs are common knowledge.

43. Dynamic (or sequential-move) games of complete information A set of players Who moves when and what action choices are available? What do players know when they move? Players’ payoffs are determined by their choices. All these are common knowledge among the players.

44. Definition: extensive-form representation The extensive-form representation of a game specifies: the players in the game when each player has the move what each player can do at each of his or her opportunities to move what each player knows at each of his or her opportunities to move the payoff received by each player for each combination of moves that could be chosen by the players

45. Dynamic games of complete and perfect information Perfect information All previous moves are observed before the next move is chosen. A player knows Who has moved What before she makes a decision

46. Game tree A path from the root to a terminal node represents a complete sequence of moves which determines the payoff at the terminal node

47. Nash equilibrium The set of Nash equilibria in a dynamic game of complete information is the set of Nash equilibria of its normal-form. How to find the Nash equilibria in a dynamic game of complete information Construct the normal-form of the dynamic game of complete information Find the Nash equilibria in the normal-form

48. Nash equilibria in entry game Two Nash equilibria ( In, Accommodate ) ( Out, Fight ) Does the second Nash equilibrium make sense? Non-creditable threats

49. Remove nonreasonable Nash equilibrium Subgame perfect Nash equilibrium is a refinement of Nash equilibrium It can rule out nonreasonable Nash equilibria or non-creditable threats

50. Subgame-perfect Nash equilibrium A Nash equilibrium of a dynamic game is subgame-perfect if the strategies of the Nash equilibrium constitute a Nash equilibrium in every subgame of the game. Subgame-perfect Nash equilibrium is a Nash equilibrium.

51. Subgame A subgame of a game tree begins at a nonterminal node and includes all the nodes and edges following the nonterminal node A subgame beginning at a nonterminal node x can be obtained as follows: remove the edge connecting x and its predecessor the connected part containing x is the subgame

52. Entry game Two Nash equilibria ( In, Accommodate ) is subgame-perfect. ( Out, Fight ) is not subgame-perfect because it does not induce a Nash equilibrium in the subgame beginning at Incumbent.

53. Find subgame perfect Nash equilibria: backward induction Starting with those smallest subgames Then move backward until the root is reached

54. Existence of subgame-perfect Nash equilibrium Every finite dynamic game of complete and perfect information has a subgame-perfect Nash equilibrium that can be found by backward induction.

55. Stackelberg model of duopoly A homogeneous product is produced by only two firms: firm 1 and firm 2. The quantities are denoted by q1 and q2, respectively. The timing of this game is as follows: Firm 1 chooses a quantity q1 ?0. Firm 2 observes q1 and then chooses a quantity q2 ?0. The market priced is P(Q)=a –Q, where a is a constant number and Q=q1+q2. The cost to firm i of producing quantity qi is Ci(qi)=cqi. Payoff functions: u1(q1, q2)=q1(a–(q1+q2)–c) u2(q1, q2)=q2(a–(q1+q2)–c)

56. Stackelberg model of duopoly Find the subgame-perfect Nash equilibrium by backward induction We first solve firm 2’s problem for any q1 ?0 to get firm 2’s best response to q1 . That is, we first solve all the subgames beginning at firm 2. Then we solve firm 1’s problem. That is, solve the subgame beginning at firm 1

57. Stackelberg model of duopoly Solve firm 2’s problem for any q1 ?0 to get firm 2’s best response to q1. Max u2(q1, q2)=q2(a–(q1+q2)–c)subject to 0 ? q2 ? +8FOC: a – 2q2 – q1 – c = 0 Firm 2’s best response, R2(q1) = (a – q1 – c)/2 if q1 ? a– c = 0 if q1 > a– c

58. Stackelberg model of duopoly Solve firm 1’s problem. Note firm 1 can also solve firm 2’s problem. That is, firm 1 knows firm 2’s best response to any q1. Hence, firm 1’s problem is Max u1(q1, R2(q1))=q1(a–(q1+R2(q1))–c)subject to 0 ? q1 ? +8That is,Max u1(q1, R2(q1))=q1(a–q1–c)/2subject to 0 ? q1 ? +8FOC: (a – 2q1 – c)/2 = 0 q1 = (a – c)/2

59. Stackelberg model of duopoly Firm 1 produces q1=(a – c)/2 and its profit q1(a–(q1+ q2)–c)=(a–c)2/8 Firm 2 produces q2=(a – c)/4 and its profit q2(a–(q1+ q2)–c)=(a–c)2/16 The aggregate quantity is 3(a – c)/4.

60. Dynamic games of complete and imperfect information Imperfect information A player may not know exactly Who has made What choices when she has an opportunity to make a choice. Example: player 2 makes her choice after player 1 does. Player 2 needs to make her decision without knowing what player 1 has made.

61. Imperfect information: illustration Each of the two players has a penny. Player 1 first chooses whether to show the Head or the Tail. Then player 2 chooses to show Head or Tail without knowing player 1’s choice, Both players know the following rules: If two pennies match (both heads or both tails) then player 2 wins player 1’s penny. Otherwise, player 1 wins player 2’s penny.

62. Static (or Simultaneous-Move) Games of Incomplete Information Introduction to Static Bayesian Games

63. Static (or simultaneous-move) games of INCOMPLETE information Payoffs are no longer common knowledge Incomplete information means that At least one player is uncertain about some other player’s payoff function. Static games of incomplete information are also called static Bayesian games

64. Cournot duopoly model of incomplete information A homogeneous product is produced by only two firms: firm 1 and firm 2. The quantities are denoted by q1 and q2, respectively. They choose their quantities simultaneously. The market price: P(Q)=a-Q, where a is a constant number and Q=q1+q2. Firm 1’s cost function: C1(q1)=cq1. All the above are common knowledge

65. Cournot duopoly model of incomplete information cont’d Firm 2’s marginal cost depends on some factor (e.g. technology) that only firm 2 knows. Its marginal cost can be HIGH: cost function: C2(q2)=cHq2. LOW: cost function: C2(q2)=cLq2. Before production, firm 2 can observe the factor and know exactly which level of marginal cost is in. However, firm 1 cannot know exactly firm 2’s cost. Equivalently, it is uncertain about firm 2’s payoff. Firm 1 believes that firm 2’s cost function is C2(q2)=cHq2 with probability ?, and C2(q2)=cLq2 with probability 1–?. All the above are common knowledge

66. Cournot duopoly model of incomplete information cont’d

67. Cournot duopoly model of incomplete information cont’d

68. Cournot duopoly model of incomplete information cont’d

69. Cournot duopoly model of incomplete information cont’d

70. Cournot duopoly model of incomplete information cont’d

71. Normal-form representation of static Bayesian games

72. Normal-form representation of static Bayesian games: payoffs

73. Normal-form representation of static Bayesian games: beliefs (probabilities)

74. Game Theory Applications in Power Systems Some Examples

75. A Continuous Strategy Game for Power Transactions Analysis in Competitive Electricity Markets (IEEE 2001) A game theory application for analyzing power transactions in a deregulated energy marketplace such as PoolCo, where participants, especially, generating entities,maximize their net profits through optimal bidding strategies (i.e.,bidding prices and bidding generations) based on Nash equilibrium idea. Game Assumptions: Enough resources to meet the demand Each generator capacity is lower than the demand Inelastic demand Neglecting transmission constraints and losses GenCo`s cost function is a common knowledge

76. A Continuous Strategy Game for Power Transactions Analysis in Competitive Electricity Markets (Con`d) Players-GenCo`s Payoff-Profit Strategy-Price & Generation Quantity Three cases are analyzed. One case for example to analysis: Considering a two player game, one player’s bidding generation is greater than a half of the total demand, and the other’s is greater than or equal to the total demand

77. A Continuous Strategy Game for Power Transactions Analysis in Competitive Electricity Markets (Con`d) Consider a price cap There are three situation for player I

78. A Cournot Game Dynamic Modeling of Power Market and its Dynamic Characteristics Analysis (2002) In order to more accurately model the behaviors of market participants, the Cournot game model i s introduced, and further the dynamic Cournot game model is proposed. With the proposed dynamic model, the dynamic behaviors of power market are analyzed. Game Assumption: The function relationships between electricity price and demand of consumer are a common knowledge. Suppose there are n generators (i.e., oligopolists) in oIigopolistic market, which lie in two regions connected with a transmission line. Cost function of each generator is not a common knowledge.

79. A Cournot Game Dynamic Modeling of Power Market and its Dynamic Characteristics Analysis (Con`d)

80. A Cournot Game Dynamic Modeling of Power Market and its Dynamic Characteristics Analysis (Con`d) In actual power market, all generators have not a complete knowledge of the market, that is, each generator only knows its own production cost. Hence, they behave adaptively, following a bounded rationality adjustment process based on a local estimate of the marginal profit (in oligopolistic competitive market, evaluating its own marginal profit is more accurate than forecasting the output quantities of other generators). The dynamic adjustment of this repeated Cournot game can be modeled as below that can be used to simulate the power market as market find its Nash equilibrium (equilibria).

81. A Game-Theoretic Model for Generation Expansion Planning (2001) Cournot model of oligopoly behavior is applied to formulate a GEP model that may characterize expansion planning in a competitive regime, particularly in pool-dominated generation supply industries. In the other words game is designated to make the decisions on which technology should be used to expand the generation system. Game assumptions Energy market clearing is based on pool model. Reserve capacity market is priced by ISO. All the GenCo`s make bids equal to their marginal cost.

82. A Game-Theoretic Model for Generation Expansion Planning (Con`d) Profit is the difference between revenues earned and costs incurred from providing electric service. Revenues consist of energy and real-time auction market payments and contracted reserve payments. Expenses include operating costs, capital investment costs, and outage costs in the planning horizon.

83. A New Game-Theoretic Framework for Maintenance Strategy Analysis (2003) The maintenance scheduling of generating units (MSU) problem is formulated as a dynamic noncooperative game with complete information. The player corresponds to the profit maximizing individual Genco, and the payoff of each player is defined as the profit from the energy auction market. The optimal strategy profile is defined by the Nash equilibrium of the game. Game assumption: Game strategy include the maintenance period of time. A state is defined by a system condition determined by whether each generator in the system is on maintenance or not. The scheduling horizon is decomposed into stages in such a way that the stage is a minimal duration of generator maintenance.

84. A New Game-Theoretic Framework for Maintenance Strategy Analysis (Con`d) The Hourly Energy Market (HEA) market is assumed to operate as a series of auction with complete information. This dynamic game has sub-game Nash equilibria

85. A New Game-Theoretic Framework for Maintenance Strategy Analysis (Con`d)

86. An Effective Transmission Network Expansion Cost Allocation Based on Game Theory (2007) The expansion of transmission systems impacts many Entities. A cooperative game theory-based scheme is proposed for the allocation of transmission expansion costs among market entities. The allocation takes into account both the physical and economic impacts of the new transmission assets and the influence of each firm on the expansion decisions in the market environment. Game assumptions: Pool based model and LMP pricing is considered for energy market. The social welfare of the system is calculated as the common social welfare minus investment cost in the planning horizon. Based on the marginal cost bidding, the best candidate lines are determined. Game is applied to allocate the investment cost

87. An Effective Transmission Network Expansion Cost Allocation Based on Game Theory (Con`d) Net surplus after the planning horizon: Since the market participants usually can influence the expansion of the network, it is assumed that the network expansion is decided by the result of a poll. The poll takes place before the -hour period. Each consumer and each producer is assigned a weight. The goal is to construct a vector of payments so that all the firms want to have the transmission network expanded. In this problem setting, the firms can be induced to cooperate.

88. Any Questions?