Cognitive Radio Technologies

Cognitive Radio Technologies Applications of Game Theory to CR

Material • A little about CRT • A little game theory • GT + CR Networks • GT, CRN and CJ

CognitiveRadioTechnologies Business Details • Founded in 2007 by Dr. James Neel and Professor Jeff Reed to commercialize cognitive radio research out of Virginia Tech • 6 employees / contractors • 07 Sales = 64k, 08 Sales = 127k • 09 Sales = 394k, 10 (contracts) = 890k Business Model • Partner with established companies to spin in cognitive radio research • Navy SBIR 08-099 => L3-Nova • Air Force SBIR 083-160 => GDC4S • Contract research and consulting related to cognitive radio and software radio • DARPA, DTI, CERDEC, Global Electronics • Position for entry in emerging wireless markets • Cognitive Zigbee

Selected Projects CR Projects SDR Projects • Prototype SDR for software controllable antenna • Fundamental limits to SDR performance • Rapid estimation of SDR resources • Distributed spectrum management for WNW • White Space Networking • Cognitive gateway with ad-hoc extensions

CRT’s Value Proposition • Designing and analyzing systems to work with interactions of complex intelligent agents in distributed processes • Frequent collaboration with DoD contractors and universities • GDC4S, L3, VT, USF, UNLV • Carry a concept from fundamental research to prototype • Matlab, OPNET, HW-in-the loop sims, prototypes on varying SDRs • Across and within layers 1-3

Outside World CRs don’t just react, they interact • Outside world is determined by the interaction of numerous cognitive radios • What makes sense for a link, may not work for a net

WhiteFi Channel Adaptation • Access nodes choose tuple (center frequency, bandwidth) • Uses 5 MHz bandwidth • 20 MHz in 5 UHF channels • Managed by access nodes with measurements from clients • Unspecified form of hysteresis • Chosen to maximize (N = num clients, c = channel) Airtime Utilization Number AP in c

WhiteFi Channel Adaptation Unstable (Not published) • Consistent with paper assumption that AP much more active than clients • Infinite Loop! • 4,5,1,3,2,6,4,… No interference, very high download Utility Characterization 0 1 2 3 4 5 6 7

Game Theory and CR • Collection of models / tools for modeling / analyzing interactive decision problems • Traditional focus on • Fixed points: Nash Equilibria • Performance: Pareto efficiency, fairness • Stability had to be grafted on • Convergence to a lesser extent

Potential Games • Existence of a function (called the potential function, V), that reflects the change in utility seen by a unilaterally deviating player. • Cognitive radio interpretation: • Every time a cognitive radio unilaterally adapts in a way that furthers its own goal, some real-valued function increases. V(a) time

Exact Potential Game Forms • Many exact potential games can be recognized by the form of the utility function Network-wide Objective Function Can’t Influence Own Outcome Only impacted by self Sum of mini- coordination games

This correlation between selfish and social benefit yields desirable behavior • Convergence • *ALL* sequences of unilateral selfish adaptations induce monotonically decreasing network interference levels • For finite waveform sets, completely unsynchronized adaptations form absorbing Markov chains • Optimality of steady-states • Assuming exhaustive adaptations, interference minimizers are the only steady-states • Stability • Sum network interference is a Lyapunov function in neighborhoods of isolated interference minimizers • In practice, many minimizers aren’t isolated, so some hysteresis is needed Figure from Fig 2.6 in I. Akbar, “Statistical Analysis of Wireless Systems Using Markov Models,” PhD Dissertation, Virginia Tech, January 2007

Implications of Monotonicity • Monotonicity implies • Existence of steady-states (maximizers of V) • Convergence to maximizers of V for numerous combinations of decision timings decision rules – all self-interested adaptations • Does not mean that that we get good performance • Only if V is a function we want to maximize

Other Potential Game Properties • All finite potential games have FIP • All finite games with FIP are potential games • Very important for ensuring convergence of distributed cognitive radio networks • -V is a is a Lyapunov function for isolated maximizers • Stable NE solvable by maximizers of V • Linear combination of exact potential games is an exact potential game • Maximizer of potential game need not maximize your objective function • Cognitive Radios’ Dilemma is a potential game

Interference Reducing Networks (Dissertation) • Concept • Cognitive radio network is a potential game with a potential function that is negation of observed network interference • Definition • A network of cognitive radios where each adaptation decreases the sum of each radio’s observed interference is an IRN • Implementation: • Design DFS algorithms such that network is a potential game with  -V () time

Two cognitive radios, j,kN, exhibit bilateral symmetric interference if k – waveform of radio k pk - the transmission power of radio k’s waveform gkj - link gain from the transmission source of radio k’s signal to the point where radio j measures its interference, - the fraction of radio k’s signal that radio j cannot exclude via processing (perhaps via filtering, despreading, or MUD techniques). Bilateral Symmetric Interference (Dissertation) What’s good for the goose, is good for the gander… Source: http://radio.weblogs.com/0120124/Graphics/geese2.jpg

An IRN 802.11 DFS Algorithm(Milcom06) • Suppose each access node measures the received signal power and frequency of the RTS/CTS (or BSSID) messages sent by observable access nodes in the network • Ignore client interference • Assumed out-of-channel interference is negligible and RTS/CTS transmitted at same power J. Neel, J. Reed, “Performance of Distributed Dynamic Frequency Selection Schemes for Interference Reducing Networks,”Milcom 2006, Washington DC, October 23-25, 2006

Statistics (Milcom / Dissertation) Reduction in Net Interference • 30 cognitive access nodes in European UNII bands • Choose channel with lowest interference • Random timing • n=3 • Random initial channels • Randomly distributed positions over 1 km2 Asynchronous Round-robin Legacy Devices Reduction in Net Interference

Asymmetry Extensions (SDRF07) • Symmetry not always there naturally • Power control • Prioritization • Beamforming • Ad-hoc nets • Symmetry can be induced by manipulating observation processes • Network optimization correlates with desired metric, but may not be desired metric • Some practical considerations ignored in pubs. ??

Sources considered Dynamic multipath environments (mobile fading) Hostile users Stability impact What if the environment is “unstable”? From Phase I Navy SBIR(“Published” at JSTeF 09) • Performance Impact Hopping Jamming (Mobile) Tone Jamming (Mobile) • Constraints: • Irregularly timed observations without collaboration or centralization • Preserve performance and responsiveness, minimal complexity • No “special” new measurements

Load-Sensitive Routing (Not published yet) • Traditional stability issues when load-sensitive • Interactions intractable for ARPANET • Generalized congestion game • Stable and load-sensitive • Ignores information distribution • Each edge is EPG • Action is contribution of traffic • Cost = 0 if not using edge • Path cost is sum of edge costs CRT Proprietary

Multi-Layer, Multiple CR Process Integration (Not published yet) • Enables stable, desirable operation of CRN with multiple different distributed processes • Spectrum, routing, DSA • Orthogonal • No interaction • Edgewise • Effectively MSI game + Generalized Congestion • Network wide (products) CRT Proprietary

Improving Coexistence(SDRF TVWS Workshop 09) • Modfifies BSI to induce “affinity” for classes of radios • Leverages database for locations & classes of radios • Assumed two step-coexistence process • Distributed sort of fractious networks into different channels (frequency deconfliction) • Can sort themselves out without direct coordination • Coordinated coexistence of compatible networks within channels (transmission time deconfliction ala .22 or .16h) • Limit frequency deconfliction to when it’s absolutely necessary • Limits trunking gains • Can account for tethered radios without revealing location / IDs • Weighted fairness needs mechanism for broadcasting weights if weights are situationally dependent Initial Final CRT Proprietary

Malicious != Mischievous (From dissertation) • Popular “solution” to mischievous nodes (selfish nodes that damage network) is to “punish” nodes • Also implies a way to “brainwash” learning nodes • Imperfect information can obfuscate punishment from mischievous behavior and produce catastrophic cascades • Even with perfect information, malicious node may be masochistic From Fig 6 in [MacKenzie_01] From [Srivastava_06]

Malicious CRs can blend in (Not published yet) Average interference levels for nodes 6-35 • Normal CR • Given available adaptations and knowledge about network state • Maximize system (own) performance • Malicious CR • Given available adaptations and knowledge about network state • Minimize system performance • Adapt at inopportune times • Simply minimize performance • Ensure marginally stable network goes unstable • Plus learning exploits • And spoofing • And information corruption 5 malicious, 30 normal 35 dB 35 normal CRT Proprietary

Detecting malicious behavior from Nash equilibria (Not published yet, but not that useful ) Predicted operating point • With non-deterministic decision processes, difficult to say whether instantaneous adaptations are “ok” • Assume we know radios are trying to maximize specific goals • We can identify the expected operating points • Assuming CRs adhere to specified goals • But: • Predictions depend on environment • Doesn’t help identify the CJ • Convergence to / existence of NE not generally guaranteed Allowable region CR2 Detected operating point CR1 CRT Proprietary

Malicious User Detection with Potential Games (Not published yet) • Implement as monitoring system that evaluates potential (emergent) function • Frequently sum of performance levels • Complexity is in the transmission / connectivity • No single node / cluster knows / can evaluate emergent function • But a malicious CR will lie • E.g., Claim massive gains to offset others’ losses • With BSI, a malicious node can’t tell a credible lie! • Other relationships exist • Need to be WPG / EPG for linear relationships V CRT Proprietary

Illustration of using emergent property to detect malicious CR Adaptations Adaptations Policy restricted channels Radio Utilities Radio Utilities Malicious User Detected Potential Function No Malicious User Potential Function CRT Proprietary

CR/CJ Interaction (in seedling proposal to Bruce) • Cognitive radios will be faced with cognitive attackers • Any fixed mitigation strategy to eventually be learned and defeated • Partial Solution: • Model as multi-armed bandit problem (classic machine learning model) • Use regret learning to achieve near-optimal performance (given presence of intelligent adversary) • Issues: • Bandit (cognitive attacker) strategy nor is solution space constant • Starting from untrained state may be too long to track assailant • Proposed Solution to Issues: • Seed the routines by learning and classifying what attacks are underway • Effectively adds case based reasoning and attack recognition / learning • State at end of seedling: • Proof of concept simulation and analysis • Anticipated Benefits: • Reduce the period of time from when new attacks emerge to when defenses are implemented • Intermediate tasks: • Learn to detect when attacked • Important to differentiate poor performance from attack • Learn to classify attacks • Generalize multi-armed bandit solutions to changing solution space • Define how to integrate together CBR and bandit solutions into CR • Characterize “reaction” times (for adversarial OODA loop) CJ is influencing outside world to intentionally confuse and harm CR Akin to trying to build a HMM, while an opponent is changing the states CRT Proprietary

CRT’s Value Proposition • Designing and analyzing systems to work with interactions of complex intelligent agents in distributed processes • Frequent collaboration with DoD contractors and universities • GDC4S, L3, VT, USF, UNLV • Carry a concept from fundamental research to prototype • Matlab, OPNET, HW-in-the loop sims, prototypes on varying SDRs • Across and within layers 1-3

Extras

All self-interested adaptations Based only on observations of own performance Decrease aggregate network interference For example, for a collection of 802.11 clusters independently choosing operating frequencies 25-30 dB Solved issues with game theory (JSTeF 09) Clusters’ Frequencies • Scalable resource utilization • No synchronization required 0 5 10 15 20 Observed Interference Levels • No information exchange overhead • More responsive network 0 5 10 15 20 25-30 dB Aggregate Network Interference • Self-stable • Converges to local-optima 0 5 10 15 20 seconds

Gain the performance without the overhead or complexity(SDRF07) • CRT’s distributed algorithms performance equivalent to “omniscient” centralized local search algorithm • Large capacity gains and interference reduction • Without the overhead, complexity, or the single-point of failure • With much better scaling • O(node density) • Using generally available measurements, e.g., RSS, node ID, time stamps (later) Reduce interference by 25-30 dB Steady-state interference Network Density Support 16 x more links Collision Probability Network Density

Cognitive Radio Technologies