analysis and design of cognitive radio networks and distributed ...

1. Analysis and Design of Cognitive Radio Networksand Distributed Radio Resource Management Algorithms

3. Research in a nutshell Hypothesis: Applying game theory and game models (potential and supermodular) to the analysis of cognitive radio interactions Provides a natural method for modeling cognitive radio interactions Significantly speeds up and simplifies the analysis process (can be performed at the undergraduate level – Senior EE) Permits analysis without well defined decision processes (only the goals are needed) Can be supplemented with traditional analysis techniques Can provides valuable insights into how to design cognitive radio decision processes Has wide applicability Focus areas: Formalizing connection between game theory and cognitive radio Collecting relevant game model analytic results Filling in the gaps in the models Model identification (potential games) Convergence Stability Formalizing application methodology Developing applications

4. Modeling Cognitive Radio Networks James Neel August 23, 2006

5. Presentation Overview Cognitive Radio Concepts Implementation approaches Cognitive radio related standards Cognitive Radio Modeling Dynamical systems model Model Variances between cognitive radios and dynamical systems Example Game models Model Variances between cognitive radios and game models Example

6. Cognitive Radio Concepts How does a radio come to be “cognitive”?

7. Cognitive Radio: Basic Idea Cognitive radios enhance the control process by adding Intelligent, autonomous control of the radio An ability to sense the environment Goal driven operation Processes for learning about environmental parameters Awareness of its environment Signals Channels Awareness of capabilities of the radio An ability to negotiate waveforms with other radios

8. Cognitive Radio Capability Matrix [FCC] ET Docket No. 03-108, March 11, 2005. [Haykin] S. Haykin, “Cognitive Radio: Brain-Empowered Wireless Communications,” IEEE Journal on Selected Areas in Communications, vol. 23, No 2, Feb. 2005. [IEEEUSA] “Improving Spectrum Usage through Cognitve Radio Technology,” IEEE USA Position, Nov 13, 2003, Available online: http://www.ieeeusa.org/policy/positions/cognitiveradio.asp [IEEE 1900.1] Draft Document, “Standard Terms, Definitions and Concepts for Spectrum Management, Policy Defined Radio, Adaptive Radio, and Software Defined Radio” Nov 9, 2005. [VT CRWG] “Cognitive Radio Definition,” Virginia Tech Cognitive Radio Work Group Wiki. Availabile Online: http://support.mprg.org/dokuwiki/doku.php?id=cognitive_radio:definition [FCC] ET Docket No. 03-108, March 11, 2005. [Haykin] S. Haykin, “Cognitive Radio: Brain-Empowered Wireless Communications,” IEEE Journal on Selected Areas in Communications, vol. 23, No 2, Feb. 2005. [IEEEUSA] “Improving Spectrum Usage through Cognitve Radio Technology,” IEEE USA Position, Nov 13, 2003, Available online: http://www.ieeeusa.org/policy/positions/cognitiveradio.asp [IEEE 1900.1] Draft Document, “Standard Terms, Definitions and Concepts for Spectrum Management, Policy Defined Radio, Adaptive Radio, and Software Defined Radio” Nov 9, 2005. [VT CRWG] “Cognitive Radio Definition,” Virginia Tech Cognitive Radio Work Group Wiki. Availabile Online: http://support.mprg.org/dokuwiki/doku.php?id=cognitive_radio:definition

9. Used cognitive radio definition A cognitive radio is a radio whose control processes permit the radio to leverage situational knowledge and intelligent processing to autonomously adapt towards some goal. Intelligence as defined by [American Heritage_00] as “The capacity to acquire and apply knowledge, especially toward a purposeful goal.” The definition for intelligence as applied to cognitive radio differs only in that the acquisition of knowledge has been subsumed into the observation process.

10. Level 0 SDR 1 Goal Driven 2 Context Aware 3 Radio Aware 4 Planning 5 Negotiating 6 Learns Environment 7 Adapts Plans 8 Adapts Protocols Cognition Cycle Level 0 – No Cognitive Operations Level 1 – Minimal Cognition Establishes Minimum Cognition Cycle Requires ability to observe environment Environment includes RF, Network, Location, and Time Level 2 - Knowledgeable of Application Provides context to interpret stimuli from environment May provide additional information to better decide which waveform to implement e.g. Higher throughput for Data, Lower latency for voice Level 3 – Knowledgeable of Radio, Network, Channel Utilizes specific models to improve value of observations Level 4 – Has several alternate strategies Now chooses best strategy and best waveform to implement strategy Level 5 – Possible to coordinate actions with other radios Negotiation can be “Do you know this waveform?” or “Are you willing to pay $ for this service?” Level 6 – Learning Begins Significant Increase in complexity, may require AI Learning is based on observations and decisions At this stage CR can autonomously learn new models of the environment This is used to improve observations, orientation and decisions Level 7 – New Plans are learned in addition to pre-programmed plans Level 8 – CR can invent new waveforms. Must now Generate Best Waveform in response to selected plan. Implies need to negotiate protocolsLevel 0 – No Cognitive Operations Level 1 – Minimal Cognition Establishes Minimum Cognition Cycle Requires ability to observe environment Environment includes RF, Network, Location, and Time Level 2 - Knowledgeable of Application Provides context to interpret stimuli from environment May provide additional information to better decide which waveform to implement e.g. Higher throughput for Data, Lower latency for voice Level 3 – Knowledgeable of Radio, Network, Channel Utilizes specific models to improve value of observations Level 4 – Has several alternate strategies Now chooses best strategy and best waveform to implement strategy Level 5 – Possible to coordinate actions with other radios Negotiation can be “Do you know this waveform?” or “Are you willing to pay $ for this service?” Level 6 – Learning Begins Significant Increase in complexity, may require AI Learning is based on observations and decisions At this stage CR can autonomously learn new models of the environment This is used to improve observations, orientation and decisions Level 7 – New Plans are learned in addition to pre-programmed plans Level 8 – CR can invent new waveforms. Must now Generate Best Waveform in response to selected plan. Implies need to negotiate protocols

11. OODA Loop: (continuously) Observe outside world Orient to infer meaning of observations Adjust waveform as needed to achieve goal Implement processes needed to change waveform Other processes: (as needed) Adjust goals (Plan) Learn about the outside world, needs of user,… Conceptual Operation

12. A radio whose operation/ adaptations are governed by a set of rules Almost necessarily coupled with cognitive radio Allows flexibility for setting spectral policy to satisfy regional considerations Policy-Based Radio

13. Cognitive Radio Applications

14. 802.11h (“Weak” CR on hardware radios – defined shortly) Idea: Upgrade control processes to permit use bands 802.11a devices to operate as secondary users to radar and satellites Dynamic Frequency Selection (DFS) Avoid radars Listens and discontinues use of a channel if a radar is present Uniform channel utilization Transmit Power Control (TPC) Interference reduction Range control Power consumption Savings Bounded by local regulatory conditions “Revision of Parts 2 and 15 of the Commission’s Rules to Permit Unlicensed National Information Infrastructure (U-NII) devices in the 5 GHz band,” ET Docket No. 03-122, June 30, 2006. “Revision of Parts 2 and 15 of the Commission’s Rules to Permit Unlicensed National Information Infrastructure (U-NII) devices in the 5 GHz band,” ET Docket No. 03-122, June 30, 2006.

15. Comments on 802.11h Status Mandated in Europe beginning 2005 WiFi Alliance lists 72 802.11h products from Toshiba, Sony, Samsung, Symbol, NEC, Intel, HP, GemTek, Fujitsu, Colubris, Cisco, bluesocket, and bandspeed Reports of limited deployment due to sensitivity problems and frequency hopping radars FCC issued testing guidelines June 30, 2006 “Revision of Parts 2 and 15 of the Commission’s Rules to Permit Unlicensed National Information Infrastructure (U-NII) devices in the 5 GHz band,” ET Docket No. 03-122, June 30, 2006. “Revision of Parts 2 and 15 of the Commission’s Rules to Permit Unlicensed National Information Infrastructure (U-NII) devices in the 5 GHz band,” ET Docket No. 03-122, June 30, 2006.

16. IEEE 802.22 – Planned Cognition Wireless Regional Area Networks (WRAN) Aimed at bringing broadband access in rural and remote areas Takes advantage of better propagation characteristics at VHF and low-UHF Takes advantage of unused TV channels that exist in these sparsely populated areas (Opportunistic spectrum usage) 802.22 specifications TDD OFDMA PHY DFS, sectorization, TPC Policies and procedures for operation in the VHF/UHF TV Bands between 54 MHz and 862 MHz Target spectral efficiency: 3 bps/Hz Point-to-multipoint system 100 km coverage radius

17. 802.22: Cognitive Aspects Observation Aided by distributed sensing (subscriber units return data to base) Digital TV: -116 dBm over a 6 MHz channel Analog TV: -94 dBm at the peak of the NTSC (National Television System Committee) picture carrier Wireless microphone: -107 dBm in a 200 kHz bandwidth. Possibly aided by spectrum usage tables Orientation Infer type of signals that are present Decision Frequencies, modulations, power levels, antenna choice (omni and directional) Policies 4 W Effective Isotropic Radiated Power (EIRP) Spectral masks, channel vacation times

18. 802.22 Status Integrated last two independent drafts (March) Still negotiating pilots, sensing requirements (Tiger Team) PHY considering relay stations (like 802.16j) Still discussing when to move draft to first work group ballot Starting up Task Group 2 (Recommended Practices) PAR not approved yet Next meeting: July 16-21st San Diego

19. The Analysis Problem Outside world is determined by the interaction of numerous cognitive radios

20. Locally optimal decisions that lead to globally undesirable networks Scenario: Distributed SINR maximizing power control in a single cluster For each link, it is desirable to increase transmit power in response to increased interference Steady state of network is all nodes transmitting at maximum power

21. General Comments on Analyzing Cognition Cycle Level 0 SDR 1 Goal Driven 2 Context Aware 3 Radio Aware 4 Planning 5 Negotiating 6 Learns Environment 7 Adapts Plans 8 Adapts Protocols 0. No - not a CR 1. OK 2. OK 3. OK 4. Probably 5. Ok 6. Ok (might even simplify) 7. No – unconstrained problem 8. No – unconstrained problem Adapted From Mitola, “Cognitive Radio for Flexible Mobile Multimedia Communications ”, IEEE Mobile Multimedia Conference, 1999, pp 3-10. Level 0 – No Cognitive Operations Level 1 – Minimal Cognition Establishes Minimum Cognition Cycle Requires ability to observe environment Environment includes RF, Network, Location, and Time Level 2 - Knowledgeable of Application Provides context to interpret stimuli from environment May provide additional information to better decide which waveform to implement e.g. Higher throughput for Data, Lower latency for voice Level 3 – Knowledgeable of Radio, Network, Channel Utilizes specific models to improve value of observations Level 4 – Has several alternate strategies Now chooses best strategy and best waveform to implement strategy Level 5 – Possible to coordinate actions with other radios Negotiation can be “Do you know this waveform?” or “Are you willing to pay $ for this service?” Level 6 – Learning Begins Significant Increase in complexity, may require AI Learning is based on observations and decisions At this stage CR can autonomously learn new models of the environment This is used to improve observations, orientation and decisions Level 7 – New Plans are learned in addition to pre-programmed plans Level 8 – CR can invent new waveforms. Must now Generate Best Waveform in response to selected plan. Implies need to negotiate protocolsAdapted From Mitola, “Cognitive Radio for Flexible Mobile Multimedia Communications ”, IEEE Mobile Multimedia Conference, 1999, pp 3-10. Level 0 – No Cognitive Operations Level 1 – Minimal Cognition Establishes Minimum Cognition Cycle Requires ability to observe environment Environment includes RF, Network, Location, and Time Level 2 - Knowledgeable of Application Provides context to interpret stimuli from environment May provide additional information to better decide which waveform to implement e.g. Higher throughput for Data, Lower latency for voice Level 3 – Knowledgeable of Radio, Network, Channel Utilizes specific models to improve value of observations Level 4 – Has several alternate strategies Now chooses best strategy and best waveform to implement strategy Level 5 – Possible to coordinate actions with other radios Negotiation can be “Do you know this waveform?” or “Are you willing to pay $ for this service?” Level 6 – Learning Begins Significant Increase in complexity, may require AI Learning is based on observations and decisions At this stage CR can autonomously learn new models of the environment This is used to improve observations, orientation and decisions Level 7 – New Plans are learned in addition to pre-programmed plans Level 8 – CR can invent new waveforms. Must now Generate Best Waveform in response to selected plan. Implies need to negotiate protocols

22. Why focus on OODA loop, i.e., why exclude other levels? OODA loop is implemented now (possibly just ODA loop as little work on context awareness) Changing plans Over short intervals plans don’t change Messy in the general case (work could easily accommodate better response equivalent goals) Negotiating Could be analyzed, but protocols fuzzy General case left for future work Learning environment Implies improving observations/orientation. Over short intervals can be assumed away Left for future work Creation of new actions, new goals, new decision rules makes analysis impossible Akin to solving a system of unknown functions of unknown variables Most of this learning is supposed to occur during “sleep” modes Won’t be observed during operation

23. General Model (Focus on OODA Loop Interactions) Cognitive Radios Set N Particular radios, i, j

24. General Model (Focus on OODA Loop Interactions) Actions Different radios may have different capabilities May be constrained by policy Should specify each radio’s available actions to account for variations Actions for radio i Ai

25. General Model (Focus on OODA Loop Interactions) Decision Rules Maps observations to actions ui:O?Ai Intelligence implies that these actions further the radio’s goal ui:O?? The many different ways of doing this merit further discussion

26. Strong Artificial Intelligence Concept: Make a machine aware of its environment and self aware

27. Weak Artificial Intelligence Concept: Develop powerful (but limited) algorithms that appear to intelligently respond to sensory stimuli Applications Machine Translation Voice Recognition Intrusion Detection Computer Vision Music Composition

28. Implementation classes Procedural cognitive radio Radio’s adaptations determined by hard coded algorithms and informed by observations Many may not consider this to be cognitive (see discussion related to Fig 6 in 1900.1 draft) Ontological cognitive radio Radio’s adaptations determined by some reasoning engine which is guided by its ontological knowledge base (which is informed by observations) A GA cognitive radio is a procedural radio in that there’s no actual reasoning being employed nor an ontological knowledge base, but its characteristics in a network are similar to that of an ontological radio.A GA cognitive radio is a procedural radio in that there’s no actual reasoning being employed nor an ontological knowledge base, but its characteristics in a network are similar to that of an ontological radio.

29. Weak/Procedural Cognitive Radios Radio’s adaptations determined by hard coded algorithms and informed by observations Many may not consider this to be cognitive (see discussion related to Fig 6 in 1900.1 draft) A function of the fuzzy definition Implementations: CWT Genetic Algorithm Radio MPRG Neural Net Radio Multi-dimensional hill climbing DoD LTS (Clancy) Grambling Genetic Algorithm (Grambling) Simulated Annealing/GA (Twente University) Existing RRM Algorithms? A Reconfigurable Platform for Cognitive Radio Zhang, Q.   Smit, G.J.M.   Smit, L.T.   Kokkeler, A.   Hoeksema, F.W.   Heskamp, M.   University of Twente, Department EEMCS, P.O. Box 217, 7500 AE Enschede, The Netherlands, E-mail: Q.Zhang@utwente.nl; This paper appears in: Mobile Technology, Applications and Systems, 2005 2nd International Conference onPublication Date: 15-17 Nov. 2005On page(s): 1- 5ISBN: 981-05-4573-8Posted online: 2006-07-24 08:57:19.0 A Reconfigurable Platform for Cognitive Radio Zhang, Q.   Smit, G.J.M.   Smit, L.T.   Kokkeler, A.   Hoeksema, F.W.   Heskamp, M.   University of Twente, Department EEMCS, P.O. Box 217, 7500 AE Enschede, The Netherlands, E-mail: Q.Zhang@utwente.nl; This paper appears in: Mobile Technology, Applications and Systems, 2005 2nd International Conference onPublication Date: 15-17 Nov. 2005On page(s): 1- 5ISBN: 981-05-4573-8Posted online: 2006-07-24 08:57:19.0

30. Strong/Ontological Radios Radio’s adaptations determined by some reasoning engine which is guided by its ontological knowledge base (which is informed by observations) Proposed Implementations: CR One Model based reasoning (Mitola) Prolog reasoning engine (Kokar) Policy reasoning (DARPA xG)

31. Modeling Interactions (1/3)

32. Modeling Interactions (2/3) Radios implement actions, but observe outcomes. Sometimes the mapping between outcomes and actions is one-to-one implying f is invertible. In this case, we can express goals and decision rules as functions of action space. Simplifies analysis One-to-one assumption invalid in presence of noise.

33. Modeling Interactions (3/3) When decisions are made also matters and different radios will likely make decisions at different time Tj – when radio j makes its adaptations Generally assumed to be an infinite set Assumed to occur at discrete time Consistent with DSP implementation T=T1?T2?????Tn t ? T Decision timing classes Synchronous All at once Round-robin One at a time in order Used in a lot of analysis Random One at a time in no order Asynchronous Random subset at a time Least overhead for a network

34. Cognitive Radio Network Modeling Summary Radios Actions for each radio Observed Outcome Space Goals Decision Rules Timing i,j ?N, |N| = n A=A1?A2?????An O uj:O?? (uj:A??) dj:O?Ai (dj:A? Ai) T=T1?T2?????Tn

35. DFS Example Two radios Two common channels Implies 4 element action space Both try to maximize Signal-to-Interference Ratio Alternate adaptations

36. Dynamical Systems Modeling

37. Basic Model Dynamical system A system whose change in state is a function of the current state and time Autonomous system Not a function of time OK for synchronous timing Characteristic function Evolution function First step in analysis of dynamical system Describes state as function of time & initial state.

38. Connection to Cognitive Radio Model g = ?d/ ? t Assumption of a known decision rule obviates need to solve for evolution function. Reflects innermost loop of the OODA loop Useful for deterministic procedural radios

39. Example: ([Yates_95]) Power control applications Defines a discrete time evolution function as a function of each radio’s observed SINR, ?j , each radio’s target SINR and the current transmit power Applications Fixed assignment - each mobile is assigned to a particular base station Minimum power assignment - each mobile is assigned to the base station in the network where its SINR is maximized Macro diversity - all base stations in the network combine the signals of the mobiles Limited diversity - a subset of the base stations combine the signals of the mobiles Multiple connection reception - the target SINR must be maintained at a number of base stations.

40. Applicable analysis models & techniques Markov models Absorbing & ergodic chains Standard Interference Function Can be applied beyond power control Contraction mappings Lyapunov Stability

41. Differences between assumptions of dynamical system and CRN model Goals of secondary importance Technically not needed Not appropriate for ontological radios May not be a closed form expression for decision rule and thus no evolution function Really only know that radio will “intelligently” – work towards its goal Unwieldy for random procedural radios Possible to model as Markov chain, but requires empirical work or very detailed analysis

42. Game Models Models of interactive decision processes

43. Game A (well-defined) set of 2 or more players A set of actions for each player. A set of preference relationships for each player for each possible action tuple.

44. Set of Players (decision makers) N – set of n players consisting of players “named” {1, 2, 3,…,i, j,…,n} Note the n does not mean that there are 14 players in every game. Other components of the game that “belong” to a particular player are normally indicated by a subscript. Generic players are most commonly written as i or j. Usage: N is the SET of players, n is the number of players. N \ i = {1,2,…,i-1, i+1 ,…, n} All players in N except for i

45. Actions

46. Preference Relations (1/2)

47. Preference Relationship (2/2) Games generally assume the relationship between actions and outcomes is invertible so preferences can be expressed over action vectors. Preferences are really an ordinal relationship Know that player prefers one outcome to another, but quantifying by how much introduces difficulties

48. Utility Functions (1/2)(Objective Fcns, Payoff Fcns)

49. Utility Functions (2/2)

50. Variety of game models Normal Form Game <N,A,{ui}> Synchronous play T is a singleton Perfect knowledge of action space, other players’ goals (called utility functions) Repeated Game <N,A,{ui},{di}> Repeated synchronous play of a normal form game T may be finite or infinite Perfect knowledge of action space, other players’ goals (called utility functions) Players may consider actions in future stages and current stages Strategies (modified di) Asynchronous myopic repeated game <N,A,{ui},{di},T> Repeated play of a normal form game under various timings Radios react to most recent stage, decision rule is “intelligent” Many others in the literature and in the dissertation

51. Cognitive radios are naturally modeled as players in a game Level 0 – No Cognitive Operations Level 1 – Minimal Cognition Establishes Minimum Cognition Cycle Requires ability to observe environment Environment includes RF, Network, Location, and Time Level 2 - Knowledgeable of Application Provides context to interpret stimuli from environment May provide additional information to better decide which waveform to implement e.g. Higher throughput for Data, Lower latency for voice Level 3 – Knowledgeable of Radio, Network, Channel Utilizes specific models to improve value of observations Level 4 – Has several alternate strategies Now chooses best strategy and best waveform to implement strategy Level 5 – Possible to coordinate actions with other radios Negotiation can be “Do you know this waveform?” or “Are you willing to pay $ for this service?” Level 6 – Learning Begins Significant Increase in complexity, may require AI Learning is based on observations and decisions At this stage CR can autonomously learn new models of the environment This is used to improve observations, orientation and decisions Level 7 – New Plans are learned in addition to pre-programmed plans Level 8 – CR can invent new waveforms. Must now Generate Best Waveform in response to selected plan. Implies need to negotiate protocolsLevel 0 – No Cognitive Operations Level 1 – Minimal Cognition Establishes Minimum Cognition Cycle Requires ability to observe environment Environment includes RF, Network, Location, and Time Level 2 - Knowledgeable of Application Provides context to interpret stimuli from environment May provide additional information to better decide which waveform to implement e.g. Higher throughput for Data, Lower latency for voice Level 3 – Knowledgeable of Radio, Network, Channel Utilizes specific models to improve value of observations Level 4 – Has several alternate strategies Now chooses best strategy and best waveform to implement strategy Level 5 – Possible to coordinate actions with other radios Negotiation can be “Do you know this waveform?” or “Are you willing to pay $ for this service?” Level 6 – Learning Begins Significant Increase in complexity, may require AI Learning is based on observations and decisions At this stage CR can autonomously learn new models of the environment This is used to improve observations, orientation and decisions Level 7 – New Plans are learned in addition to pre-programmed plans Level 8 – CR can invent new waveforms. Must now Generate Best Waveform in response to selected plan. Implies need to negotiate protocols

52. Interaction is naturally modeled as a game

53. Some differences between game models and cognitive radio network model Assuming numerous iterations, normal form game only has a single stage. Useful for compactly capturing modeling components at a single stage Normal form game properties will be exploited in the analysis of other games Repeated games are explicitly used as the basis for cognitive radio algorithm design (e.g., Srivastava, MacKenzie) Not however, focus of dissertation Not the most commonly encountered implementation

54. Summary The interactions in a cognitive radio network (levels 1-3) can be represented by the tuple <N, A, {ui}, {di},T> A dynamical system model adequately represents inner-loop procedural radios A myopic asynchronous repeated game adequately represents ontological radios and random procedural radios Suitable for outer-loop processes Not shown here, but can also handle inner-loop Some differences in models Most analysis carries over Some differences

55. Questions?