Cooperation and Fairness of Wireless Networking using Game Theoretical Approaches

1. Cooperation and Fairness of Wireless Networking using Game Theoretical Approaches Zhu Han UNIK June 5th, 2008

2. Outline Motivation and game theoretical approaches OFDMA Resource Allocation Power control, bit loading and channel assignment problem Simple high efficient bargaining solution Cooperative transmission: new communication paradigm Distributed implementation with less signaling Broader impact other than that in physical layer Packet forwarding wireless networks with selfish nodes Curse of boundary nodes Cooperative game using cooperative transmission Other topics Summary UNIK talk 6/5/08

3. Resource Allocation over Wireless Networks • Resource Allocation over Wireless Networks • Limited radio resources, conflict interests among users • Different parameters and constraints in different layers • New Perspectives Compared to Traditional Communications • System optimality instead of individual link optimality • Interactions among users in addition to overcome nature • Cross layer approaches instead of layered design • Challenges • Traditional approach for resource allocation: centralized control • Excessive measurement, signaling, and feedback • Network and MAC layer • Distributive resource allocation: user autonomy • Pro: local information, less signaling/overhead, flexible • Con: low system efficiency and unfairness UNIK talk 6/5/08

4. Enforcing Cooperation • Enforcing Cooperation in Wireless Networks Greedy usage of system resources by the autonomous distributive users: reducing system efficiency • Current Approaches for Enforcing Cooperation • Pricing anarchy: using price/tax to control resource usage • Pro: no incentive to overuse the resources • Con: price itself hard to calculate; continuous parameters only; hard for cross-layer optimization, heterogeneous networks, multicell networks, ad hoc/sensor networks … • Tradeoff: system efficiency and individual fairness • Game Theoretical Approaches • Natural: conflict between parties; equilibrium of competition • Flexible: rich mathematical tools; different ways to enforce cooperation: incentive, threat, referee, negotiation… UNIK talk 6/5/08

5. Kill Bill Rich Game Theoretical Approaches • Non-cooperative static game: • Play once • Prison dilemma • Zero sum game: O+H=1 • Dynamic game: play multiple times • Threat of punishment by repeated game. MAD: Nobel prize 2005. • Tit-for-Tat: • Cooperative game • Startup company: everybody wants IPO, while competing for more stock shares. • Coalition game: M +(O+H)=1, where O and H belongs to the same party • Auction Theory and Mechanism Design (Nobel Prize 2007) An eye for eye makes the world blind. UNIK talk 6/5/08

6. User 3 User 4 User 5 Frequency User 2 User 1 User 6 Time Single Cell OFDMA Networks • Orthogonal Frequency Division Modulation (OFDM) • Frequency selective fading. No ISI. High speed • CSMA: RTS/CTS for multiuser system, TDMA system • Why OFDMA systems? • Frequency, time, and multiuser diversity. Challenges: difficult mixed resource allocation assignment problems; need to consider fairness UNIK talk 6/5/08

7. Single Cell System Descriptions (Example) • Single cell uplink case: • M subcarriers, K users • Optimization: overall rate • Subcarrier assignment: only one user per subcarrier. • Conflict: the same subcarrier may be good for many users. • Constraints • Minimal requirement Rmin • Maximal power from mobile unit Pmax UNIK talk 6/5/08

8. Basic Problem (An Example) • Problem Formulation (an example for single cell uplink case) • Optimization Goals U: maximal rate and max-min • Channel Assignment User i occupies subcarrier j [A]ij=Aij {0,1} • Bit Loading: Rate for user i at subcarrier j Adaptive modulation • Power Allocation: • Complicated Integer Non-convex Assignment Problem. UNIK talk 6/5/08

9. Motivations Using Game Theory for OFDMA • Existing Work • Relaxation and then back to integer • Finding the lowest point in the basin or valley does not mean finding the lower village (which is discrete in nature). NP hard • Hungarian method: complexity • Two Step Solution: Integer heuristic first, then programming • Local optima • Cooperative game for single cell OFDMA system • Competition: each subcarrier can be occupied by one user. • Exist a central node: base station, similar to the market in reality where negotiations and exchanges between mobiles can take place. • Distributed users can negotiate via base station to cooperate in making the decisions on the subcarrier usage, such that each will operate at its optimum and joint mutual benefits are made about their operating points. UNIK talk 6/5/08

10. New Optimization Goal Using Game Theory • New Optimization Goal: Nash Bargaining Solutions • Why product form? Why not max-min or maximal rates? • Minimal Rate Requirement • Nash Six Axioms: Unique optimal solution • NBS Fairness: Generalized proportional fairness • Efficiency: Little overall performance loss • Any Simple Algorithm? UNIK talk 6/5/08

11. user1 user2 Channel Gain 1 1 1 4 2 3 3 2 4 5 5 6 6 6 4 5 3 2 Sorted Channel Index Preference   Preference Two-User Algorithm • Two band partition algorithm: Two users exchange subcarriers. • Initialization: Merge subcarrier sets • Sort the combined subcarrier set by the ratio of channel gains • For j=1,…,M-1 User 1 occupies and water-fills subcarrier 1 to j User 2 occupies and water-fills subcarrier j+1 to M Calculate U=(R1-Rmin)(R2-Rmin) End • Choose the j that generates the largest U that satisfies all constraints. 5. Update: 6. Continue until convergence User 1 channel gain in jth subcarrier User 2 channel gain in jth subcarrier UNIK talk 6/5/08

12. Properties • low complexity O(MlogM) • Theorem 1: • When , NBS fairness is the proportional fairness. NBS fairness is a generalized proportional fairness. • Theorem 2: • There exists a unique and optimal solution for the formulated multi-user problem. • Theorem 3: • The algorithm can find the unique and optimal solution for two user case, when SNR is high. • Theorem 4: Convergence UNIK talk 6/5/08

13. N-Person OFDMA Resource Allocation • Proposed N-person cooperative games • Scheme 1. Initialization 2. Grouping users to pairs, which is called coalitions 3. Apply two-user algorithm to each pair 4. Go to 2, stop until no improvement can be achieved • Low complexity K: number of users • Key Difference • Traditional scheme in Subcarrier level: with dimension of M • Optimization in user domain. Complexity of with order of K • Iterative improvement: Soul of interior-point method How to group users into pairs (coalitions)? UNIK talk 6/5/08

14. Cooperative Game Approach:Multiple User Scheme: Grouping Users • Random Method: free market. • Negotiate between arbitrary two users to exchange subcarrier • Converge slowly and achieve local optima • Hungarian Method: • Select optimal coalition pairs to maximize payoff for each negotiation round. • Benefit Table b: negotiation effect bij: benefit via negotiation between user i and user j. • Assignment Table X: Xij=1: negotiation between i and j =0: no negotiation UNIK talk 6/5/08

15. Hungarian Algorithm • A~E Brides and H~L Grooms: • Brides rank grooms 1~5 • Maximize the overall happiness • Complexity • K user • Much lower than • Find most effective negotiation for each round. • Con: limited central control Homeless, Slave, or Ph.D. student Millionaire Professor Assignment table UNIK talk 6/5/08

16. Overall Rate (MHz) Two User Simulations • Setup :User1 locates at 100m from base station. User2 moves • Fairness and efficiency • Rates for different user 2 location D2 • Fairness, compared with maximal rate algorithm • Little rate loss to maximal rate algorithm, but great rate gain over max-min algorithm. • Open Issue: beyond cognitive, dynamic spectrum access, mesh, video, what else to extend the ideas to and could it be used in standards UNIK talk 6/5/08

17. Transition Motivation and game theoretical approaches OFDMA Resource Allocation Power control, bit loading and channel assignment problem Simple high efficient bargaining solution Cooperative transmission: new communication paradigm Distributed implementation with less signaling Broader impact other than that in physical layer Packet forwarding wireless networks with selfish nodes Curse of boundary nodes Cooperative game using cooperative transmission Other topics Summary UNIK talk 6/5/08

18. Cooperative Transmission New communication paradigm Exploring broadcast nature of wireless channel Relays can be served as virtual antenna of the source MIMO system Multi-user and multi-route diversity Most popular research in current wireless communication Industrial standard: IEEE WiMAX 802.16J Destination Destination Phase 1 Phase 2 Sender Sender Relay Relay UNIK talk 6/5/08

19. System Model (1) • System model: • One source-destination node pair; N relay nodes, amplify-and-forward cooperation protocol. • Phase 1 - received signals from source node s to destination node d and each relay node ri • Phase 2 - received signal at destination node d via relay node ri with . • Destination combines two phases to improve performance. UNIK talk 6/5/08

20. System Model (2) • Maximal achievable rate of direct transmission is • Maximal achievable rate at the destination output with relay node rihelping is with as a bandwidth factor and • Increase of capacity region and diversity gain for BER • Depending on the power control and relay locations • Challenge • Broader impact other than power control and relay selection • Needs all channel information; a lot of signalling • Motivation for game theory UNIK talk 6/5/08

21. Packet Forwarding Networks Characteristics of packet forwarding networks such as MANET Most likely involved multiple hops transmissions Require other nodes to forward packets. Individual node has its own autonomy Forwarding others’ packets consumes the node’s limited energy Reluctant to forward others’ packets If nodes do not cooperate Network can be disconnected Fatal effects on network as well as individual performances Nash equilibrium No user can achieve better if the others do not change strategy Likely nobody forwards the others’ information in our case To overcome this problem, we need to employ the repeated game UNIK talk 6/5/08

22. Repeated Game Basics Packet forwarding network modeled as a graph G(L,A) Each node has transmission destination To reach the destination j in , depending graph contains the nodes that transmitter i will depend on packet forwarding. Repeated game: average utility (power in our case) over time. Discounting factor  Folk theorem If the nodes are mutually dependent, ensure cooperation by threat of future punishment. Any feasible solution can be enforced by repeated game UNIK talk 6/5/08

23. Cartel Maintenance • Enforcing Cooperation by Punishment • Each user tries to maximize the benefit over time. • Short term greedy benefit will be weighted out by the future punishment from others. By maintaining this threat of punishment, cooperation is enforced among greedy users. • Cartel Maintenance Repeated Game Approach • Initialization: Cooperation • Detect the outcome of the game: If better than a threshold, play cooperation in the next time; Else, play non-cooperation for T period, and then cooperate. • Applications • Rate control for selfish users in multiple access networks • Packet forwarding for ad hoc network • Power control for co-channel interfered networks • Self learning algorithms UNIK talk 6/5/08

24. Curse of Boundary Nodes Boundary nodes depend on the backbone nodes for transmission. but backbone nodes do not depend boundary nodes. (dependence graph) Example: 1,2 backbone nodes; 0,3 boundary nodes Very famous problem in this research community UNIK talk 6/5/08

25. Cooperative Transmission Model No cooperation (direct transmission), backbone needs power Cooperative transmission Stage one: direct transmission. s, source; r, relay; d, destination Stage two: relay retransmission using orthogonal channels, amplified-and-forward Maximal ration combining at the receiver of backbone node To achieve same SNR, power saving for backbone nodes P0<Pd UNIK talk 6/5/08

26. Main Idea Boundary nodes help the backbone node reduce transmission power using cooperative transmission, for future rewards of packet forwarding by the backbone node. The idea can be formulated by a coalition game. My own understanding of the idea If bullied by a Mafia, take revenge, (repeated game) If revenge cannot be taken, join the Mafia, (coalition game) Poor guy’s daughter got bullied by sons of an influential man Punishment could not be taken by law or revenge, then he asked for help from Don. Don ordered to beat the sons, and asked for payback when his son was dead UNIK talk 6/5/08

27. Coalition Game Stability and Fairness Coalition S, (N,v), N is the set of nodes, v is the characteristic function: overall benefit by coalition. Payoff function Group rational Individual rational, better than work alone; mutual benefit Core: no node has incentive to leave grand coalition Fairness Min-Max Fairness Average Fairness Market Fairness Key to the success collaboration Mutual benefits and fairness UNIK talk 6/5/08

28. Joint Repeated Game and Coalition Game UNIK talk 6/5/08

29. Simulation Results Setups: source-destination 100m or 50m, source-relay distance varying 1/i: How many packets need to relay before a transmission reward Longer the distance, less effective the boundary nodes to help backbone node, the smaller i, and more packets the boundary nodes need to transmit to get rewards. UNIK talk 6/5/08

30. Simulation Results Connectivity: any node can reach any other node in the network More than 50% network connectivity improvement. Conclusion: using cooperative transmission and cooperative game, we solve a well known problem in wireless networking. UNIK talk 6/5/08

31. Transition Motivation and game theoretical approaches OFDMA Resource Allocation Power control, bit loading and channel assignment problem Simple high efficient bargaining solution Cooperative transmission: new communication paradigm Distributed implementation with less signaling Broader impact other than that in physical layer Packet forwarding wireless networks with selfish nodes Curse of boundary nodes Cooperative game using cooperative transmission Other topics Summary UNIK talk 6/5/08

32. Game Referee Non-cooperative Game Approach:Referee-Based Approach for Multicell OFDMA • Algorithm 1. Initialization 2. Non-cooperative game 3. Desired Nash Equilibrium? 4. Subcarrier removal/ rate reduction • Implementation • Where is referee • Small overhead • No more measurement • Complexity O(MlogM) • Synchronization R: required rate S: occupied subcarrier set Candidate? Candidate? UNIK talk 6/5/08

33. Auction Theory Example: painting auction Highest bidder gets the good and pays the bid Elements of auction: Good: resource Auctioneer (manager): representing seller of the good Bidders (users): buyers of the good Rules of auction: Bids: what the bidders submit to the auctioneer Allocation: how auctioneer allocates the good to the bidders Payments: how the bidders pay the auctioneer Suitable for communication resource allocation; video UNIK talk 6/5/08

34. Sensor Networks Energy and Lifetime Security Problem Key idea Use cooperative transmission to bypass the energy depleting nodes Reduce the transmission power for each link Beamforming to null the direction of malicious nodes Future works Cooperative routing Video surveillance Bio and medical sensor Car torrent Direct Transmission Cooperative Transmission 1 0 k Sink UNIK talk 6/5/08

35. $1000 Per Power $800 Per Power Two Level: Buy/Seller Game for Power Control and Relay Section for Cooperative Transmission • Buyer-Seller Game • Sender (buyer) buying the services from the relays to improve its performance, such as the transmission rate • Relays (sellers) selling service, such as power, by setting prices • Tradeoffs: price too high, sender buying others; price too low, profit low; sender deciding buy whose and how much to spend • Procedures: convergence to the optimal equilibrium UNIK talk 6/5/08

36. Others MUD + Network coding + Cooperative transmission Cooperative OFDMA Security in cooperative transmission Cooperative UWB Coverage extension using cooperative transmission Cognitive radios: Double auction and evolutional game Collaborative sensing Security in cognitive radio Random matrix theory for cooperative transmission Physical layer security UNIK talk 6/5/08

37. Other Work • Dynamic Adaptive Wireless Resource Allocation • Ad hoc/Sensor Network Design • Ultra Wide Band Communication • Cognitive Radios • Information Assurance and Network Security • Multimedia over Wireless Networks • Underwater Acoustic Communication • Unmanned Air Vehicle • Wireless Access in Vehicular Environment • Compressed Sensing for Image Processing • Physical Layer Security • Bio Signal Processing and Bio Information Processing UNIK talk 6/5/08

38. Conclusions Cooperation and fairness problems for wireless networking Advantages of game theory Examples OFDMA resource allocations Cooperative transmission for networking problem Many other examples Many future research directions Many collaboration opportunities UNIK talk 6/5/08

39. Questions? Thank you very much UNIK talk 6/5/08