Zhu Han, Dusit Niyato, Walid Saad, and Tamer Basar

1. Game Theory in Wireless and Communication Networks: Theory, Models, and ApplicationsLecture 11Game with Bounded Rationality Zhu Han, Dusit Niyato, Walid Saad, and Tamer Basar

2. Overview of Lecture Notes • Introduction to Game Theory: Lecture 1, book 1 • Non-cooperative Games: Lecture 1, Chapter 3, book 1 • Bayesian Games: Lecture 2, Chapter 4, book 1 • Differential Games: Lecture 3, Chapter 5, book 1 • Evolutionary Games: Lecture 4, Chapter 6, book 1 • Cooperative Games: Lecture 5, Chapter 7, book 1 • Auction Theory: Lecture 6, Chapter 8, book 1 • Matching Game: Lecture 7, Chapter 2, book 2 • Contract Theory, Lecture 8, Chapter 3, book 2 • Learning in Game, Lecture 9, Chapter 6, book 2 • Stochastic Game, Lecture 10, Chapter 4, book 2 • Game with Bounded Rationality, Lecture 11, Chapter 5, book 2 • Equilibrium Programming with Equilibrium Constraint, Lecture 12, Chapter 7, book 2 • Zero Determinant Strategy, Lecture 13, Chapter 8, book 2 • Mean Field Game, Lecture 14, book 2 • Network Economy, Lecture 15, book 2

3. Overview • All players are assumed to be fully rational, which might not true with human and low capable devices. • Outline • Introduce the concept of bounded rationality and its implication on game theory. • develop into the fundamental details of one of the most important frameworks that can capture bounded rationality, prospect theory, which essentially deals with subjective perceptions. • conclude by shedding some light on other related notions of bounded rationality. • An example

4. games with bounded rationality • In a behavioral game, it is customary to study how certain real-world limits, such as the bounds on the cognitive abilities of a human, can impact the rational tenets of game theory and alter its predicted equilibrium outcomes. • For Wireless CPS • the natural, cognitive bounds on the rationality of human players (users, administrators, hackers, etc.) that interact with the wireless or cyber-physical system, • the limited resources and computational capabilities of certain devices (e.g., IoT sensors) that prevent them from following the conventional, rational path of game theory.

5. Example: Allais’s Paradox Fear of disappointment. Risk averse. High Probability • 95% chance to win $10,000 • 100% chance to win $9,499 Hope to avoid loss. Risk seeking 0.95*10000>9499 Low Probability Gain High Probability • 5% chance to win $10,000 • 100% chance to win $501 Lose • 95% chance to lose $10,000 • 100% chance to lose $9,499 0.05*10000<501 0.95*10000>9499 Hope of large gain. Risk seeking Low Probability • 5% chance to lose $10,000 • 100% chance to lose $501 Fear of large loss. Risk averse 0.05*10000<501

6. Example: evolutionary game • limit on the informational gathering capabilities of the players. • due to informational asymmetry and uncertainty, • instead of acting as typical optimizers, the players will simply copy strategies from others, based on observable payoffs. • In essence, evolutionary games take an “evolutionary biology" approach to optimization, through mutations and natural selection

7. Prospect Theory • Nobel-prize winning • PT provides one with mathematical tools to understand how decision making, in real life, can deviate from the tenets of expected utility theory (EUT), a conventional game-theoretic notion which is guided strictly by objective notions of gains and losses, player rationality, conformity to pre-determined decision making rules that are unaffected by real-life perceptions of benefits and risk. • people cannot be counted on to always choose optimally among alternatives if merely stating the alternatives differently influences their choices • Solution: use prospect-theoretic notions to refine existing game-theoretic mechanisms

8. Prospect Theory • Expected utility theory (EUT) cannot explain the deviations due to end-user subjectivity • Prospect theory (PT) [Kahneman,Tversky’79] as a Nobel prize winning theory explains the deviations in monetary transactions: • People usually over-weigh low probability outcomes and underweigh their outcomes with high probabilities • Loss looms larger than gains • Prospect theory has recently been applied in many contexts: • Social sciences [Gao’10] [Harrison’09] [Tanaka’16] • Communication networks [Li’14] [Yu’14] [Yang’14] [Lee’15] • Smart energy management [Wang’14] [Xiao’14]

9. Fundamentals Tenets of Prospect Theory: Weighting and Framing

10. weighting function: Prelec function

11. －α=1 －α=0.5 Probability Weighting Functions • Probability weighting function models the subjectivity of a player • Subjective probability for a player to weigh the outcome with a probability p • S-shaped and asymmetrical, ranging in [0,1] • Objective weight decreases with the player’s subjective evaluation distortion • Prelec function [Prelec’98]:

12. Subjective Perceptions of Utility Functions The Framing Effect • utility framing or reference points: • each player frames its gains or losses with respect to a possibly different reference point • Example: save $10 per month for bill for rich and poor people • subjective value functions - concave in gains, convex in losses - over the possible outcomes. Rk is a raw utility reference point and vk is a so-called value function that transforms the raw utility of the player into a PT utility that is framed with respect to a reference point

13. Framing Effect Illustration of the prospect-theoretic framing effect: how objective utilities are viewed subjectively by human participants. The utility function value changes depending on a certain reference point that highlights the individual perceptions of gains and losses.

14. Impact of PT on Game Analysis • properties may no longer hold under PT • requiring new analysis • existence of an equilibrium may be jeopardized by PT effects • overall operation of optimization mechanisms can be affected • traditional equilibrium analysis might not work • no general rules for analyzing games under PT exist • some application-specific approaches exit

15. Related Work • PT-based channel access game between two subjective end-users in wireless network [Li’12] • Wireless operator invests spectrum to users under uncertain spectrum supply using PT [Yu’14] • PT-based random access game between two users choosing the transmission probabilities on a radio channel [Li’14] • Stackelberg game between the SP offering the bandwidth and subjective end-users to choose services [Yang’14] • PT-based microgrids energy trading game in smart grids [Xiao’15]

16. Other Notions of Bounded Rationality • Satisficing: describe a procedure for constructing expectations of how good a potential game solution might reasonably be achieved. Once such a reasonably good solution is achieved, players may stop their search for improvements • Satisficing equilibrium: • defined as a state at which each player has already reached a “target" utility level or cannot achieve this target by unilaterally changing its action, given the other players actions across all games. • a refinement of the idea of a Nash equilibrium, that takes into account the satisficing behavior of the players: instead of seeking a best response, the players will seek a target outcome that satisfies their requirements.

17. quantal best response • players become more likely to make errors as those errors become less costly, a notion that is often known as cost-proportional errors. • instead of using a conventional best response, players will react quantally, rather than via strict maximization

18. quantal response equilibrium (QRE) • generalized the Nash equilibrium to the case in which no player can unilaterally improve its utility by using a quantal (rather than a fully rational) best response • A QRE is guaranteed to exist for any normal form game and non-negative precision parameter • QRE points are not guaranteed to be unique • human players will play games differently depending on the magnitudes of the payoffs involved

19. Summary • PT provides a rigorous framework for incorporating bounded rationality in game theory. • PT focuses on how game-theoretic decision making is affected by risk and uncertainty, through the use of the weighting and framing effects. • The incorporation of PT effects will, however, lead to non-linearities in the utility functions, hence requiring new analysis for the equilibrium or other outcomes of the game. • Other types: satisfaction equilibrium and quantal • Daniel Kahneman Nobel 2002, "for having integrated insights from psychological research into economic science, especially concerning human judgment and decision-making under uncertainty"