Interactive Cognitive Radios: Applying Game Theory for Analysis

1. Game Theory & Cognitive Radiopart A Hamid Mala

2. Presentation Objectives • Basic concepts of game theory • Modeling interactive Cognitive Radios as a game • Describe how/when game theory applies to cognitive radio. • Highlight some valuable game models.

3. Interactive Cognitive Radios • Adaptations of one radio can impact adaptations of others • Interactive Decisions • Difficult to Predict Performance

4. Scenario: Distributed SINR maximizing power control in a single cluster. Final state : All nodes transmit at maximum power. (1) the resulting SINRs are unfairly distributed (the closest node will have a far superior SINR to the furthest node) (2) battery life would be greatly shortened. Interactive Cognitive Radios Power SINR

5. traditional analysis techniques • Dynamical systems theory • optimization theory • contraction mappings • Markov chain theory

6. Applying game theory and game models (potential and supermodular) to the analysis of cognitive radio interactions Provides a natural method for modeling cognitive radio interactions Significantly speeds up and simplifies the analysis process Permits analysis without well defined decision processes Research in a nutshell

7. Game Theory Definition, Key Concepts

8. Exaple Same color winner opposite color winner $ = card number of winner

9. Exaple Same color winner opposite color winner $ = card number of winner

10. Girl’s strategies Boy’s strategies Pay-off function Exaple Matrix representation (2,-2) (-8,8) (-1,1) (7,-7)

11. Games • A game is a model (mathematical representation) of an interactive decision situation. • Its purpose is to create a formal framework that captures the relevant information in such a way that is suitable for analysis. • Different situations indicate the use of different game models. Normal Form Game Model • A set of 2 or more players, N • A set of actions for each player, Ai • A set of utility functions, {ui}, that describe the players’ preferences over the outcome space

12. Nash Equilibrium An action vector from which no player can profitably unilaterally deviate. Definition An action tuple a is a NE if for every i  N for all biAi.

13. Friend or Foe Example Foe Friend Friend 500,500 0,1000 Foe 0,0 1000,0 Yes (Friend, Friend)?? No (Friend, Foe)?? (Foe, Friend)?? Yes (Foe, Foe)?? Yes

14. Modeling and Analysis Review

15. void update_power( void ) { /*Adjusting power level*/ int k; } Modeling a Network as a Game Network Game Nodes Players Power Levels Actions Algorithms Utility Functions Structure of game is taken from the algorithm and the environment [Laboratoire de Radiocommunications et de Traitement du Signal]

16. Modeling Review • The interactions in a cognitive radio network can be represented by the tuple: <N, A, {ui}, {di},T> • Timings: • Synchronous • Round-robin • Random • Asynchronous Dynamical System

17. NE3 NE2 a2 NE3 NE1 NE2 a2 NE3 NE3 NE1 a1 NE2 a2 NE2 a1 a2 NE1 NE1 a3 a1 a1 Key Issues in Analysis • Steady state characterization • Steady state optimality • Convergence • Stability • Scalability Optimality Are these outcomes desirable? Do these outcomes maximize the system target parameters? Convergence How do initial conditions impact the system steady state? What processes will lead to steady state conditions? How long does it take to reach the steady state? Scalability As the number of devices increases, How is the system impacted? Do previously optimal steady states remain optimal? Stability How does system variations impact the system? Do the steady states change? Is convergence affected? Steady State Characterization Is it possible to predict behavior in the system? How many different outcomes are possible?

18. How Game Theory Addresses These Issues • Steady-state characterization • Nash Equilibrium existence • Identification requires side information • Steady-state optimality • In some special games • Convergence • in some cases • Stability, scalability • No general techniques • Requires side information

19. Nash Equilibrium Identification • Time to find all NE can be significant • Let tu be the time to evaluate a utility function. • Search Time: • Example: • 4 player game, each player has 5 actions. • NE characterization requires 4x625 = 2,500 tu • Desirable to introduce side information.

20. Example(1) : The Cognitive Radios’ Dilemma Example : The Cognitive Radios’ Dilemma • Two cognitive radios • Each radio can implement two different waveforms • low-power narrowband • higher power wideband Frequency domain representation of waveforms The Cognitive Radios’ Dilemma in Matrix NE=?

21. Repeated Game Model Consists of a sequence of stage games which are repeated a finite or infinite number of times. Most common stage game: normal form game. Finite Improvement Path (FIP) From any initial starting action vector, every sequence of round robin better responses converges. Weak FIP From any initial starting action vector, there exists a sequence of round robin better responses that converge. Repeated Games and Convergence

22. Better Response Dynamic • During each stage game, player(s) choose an action that increases their payoff, presuming other players’ actions are fixed. • Converges if stage game has FIP. B A a 1,-1 0,2 b -1,1 2,2

23. Best Response Dynamic • During each stage game, player(s) choose the action that maximizes their payoff, presuming other players’ actions are fixed. • converge if stage game has weak FIP. B A C a -1,1 1,-1 0,2 1,-1 b -1,1 1,2 2,1 c 2,0 2,2

24. Supermodular Games • Key Properties • Best Response (Myopic) Dynamic Converges • Nash Equilibrium Generally Exists • Why We Care • Low level of network complexity • How to Identify

25. Supermodulaar Games • NE Existence: have at least one NE. • NE Identification: all NE for a game form a lattice. While this does not particularly aid in the process of initially identifying NE, from every pair of identified • Convergence: have weak FIP, so a sequence of best responses will converge to a NE. • Stability: if the radios make a limited number of errors or if the radios are instead playing a best response to a weighted average of observations from the recent past, play will converge.

26. Example : outer loop power control • Parameters • Single Cluster • Pi = Pj = [0, Pmax]  i,j N • Utility target SINR • Supermodular – best response convergence

27. Summary • When we use game theory to model and analyse interactive CRs, it should address : • steady state existense and identification • convergence • stability • desirability of steady states • Supermodular games : to some extent

28. Questions?

29. Game Theory & Cognitive Radiopart B Mahdi Sadjadieh

30. Overview • Potential Game Model • Type of Potential Game • Example of Exact Potential Game • FIP and Potential Games • How Potential Games handle the shortcomings • Physical Layer Model Parameters and Potential Game

31. Identification NE Properties (assuming compact spaces) NE Existence: All potential games have a NE NE Characterization: Maximizers of V are NE Convergence Better response algorithms converge. Stability Maximizers of V are stable Design note: If V is designed so that its maximizers are coincident with your design objective function, then NE are also optimal. Potential Game Model • Existence of a potential function V such that

32. Existence of a function (called the potential function, V), that reflects the change in utility seen by a unilaterally deviating player. Potential Games E1 E2 E3 E4 GPGGOPG (Gilles) OPG  GPG (finite A)

33. Potential Games

34. Lack of weak improvement cycles [Voorneveld_97] FIP and no action tuples such that Better response equivalence to an exact potential game [Neel_04] Ordinal Potential Game Identification Not an OPG An OPG

35. Lack of weak improvement cycles [Voorneveld_97] FIP and no action tuples such that Better response equivalence to an exact potential game [Neel_04] Ordinal Potential Game Identification Not an OPG An OPG

36. Other Exact Potential Game Identification Techniques • Linear Combination of Exact Potential Game Forms [Fachini_97] • If <N,A,{ui}> and <N,A,{vi}> are EPG, then <N,A,{ui + vi}> is an EPG • Evaluation of second order derivative [Monderer_96]

37. Exact Potential Game Forms • Many exact potential games can be recognized by the form of the utility function

38. Self-motivated game BSI game Dummy game Example Identification • Single cluster target SINR • Better Response Equivalent

39. FIP and Potential Games • GOPG implies FIP ([Monderer_96]) • FIP implies GOPG for finite games ([Milchtaich_96]) • Thus we have a non-exhaustive search method for identifying when a CRN game model has FIP. • Thus we can apply FIP convergence (and noise) results to finite potential games.

40. Steady-states • As noted previously, FIP implies existence of NE

41. If uiare designed so that maximizers of V are coincident with your design objective function, then NE are also optimal. (*) Can also introduce cost function to utilities to move NE. In theory, can make any action tuple the NE May introduce additional NE For complicated NC,might as well completely redesign ui Optimality V a

42. Convergence in Infinite Potential Games • -improvement path • Given  >0, an -improvement path is a path such that for all k1, ui(ak)>ui(ak-1)+  where i is the unique deviator at step k. • Approximate Finite Improvement Property (AFIP) • A normal form game, , is said to have the approximate finite improvement property if for every >0 there exists an such that the length of all -improvement paths in  are less than or equal to L. • [Monderer_96] shows that exact potential games have AFIP, we showed that AFIP implies a generalized -potential game.

43. Convergence Implications

44. How potential games handle the shortcomings • Steady-states • Finite game NE can be found from maximizers of V. • Optimality • Can adjust exact potential games with additive cost function (that is also an exact potential game) • Sometimes little better than redesigning utility functions • Game convergence • Potential game assures us of FIP (and weak FIP) • DV satisfy Zangwill’s (if closed) • Noise/Stability • Isolated maximizers of V have a Lyapunov function for decision rules in DV • Remaining issue: • Can we design a CRN such that it is a potential game for the convergence, stability, and steady-state identification properties • AND ensure steady-states are desirable?

45. More Examples

46. Physical Layer Model Parameters

47. SINR Power Control Games Assume that there is a radio network wherein each radio can alter their power. Assume each radio reacts to some separable function of SINR, e.g. log ratio Each radio would also like to minimize power consumption Decentralized Power Control Using a dB Metric Thus game is a potential game and convergence is assured and we can quickly find steady states.

48. Example Power Control Game • Parameters • Single Cluster • DS-SS multiple access • Pi = Pj = [0, Pmax]  i,j N • Utility target BER Also a potential game.

49. Snapshot inner + outer loop power control • Parameters • Single Cluster • DS-SS multiple access • Pi = Pj = [0, Pmax]  i,j N • Utility target SINR • Supermodular – best response convergence

50. Game Models, Convergence, and Complexity • Determining the kind of game required to accurately model a RRM algorithm yields information about what updating processes are appropriate and thus indicates expected network complexity. • In [Neel04] the following relation between power control algorithms, game models, and network complexity was observed.