Multiple-access Communication in Networks A Geometric View

1. Multiple-access Communication in Networks A Geometric View W. Chen & S. Meyn Dept ECE & CSL University of Illinois

2. Relaxation Techniques for Net Opt W. Chen & S. Meyn ACHIEVEMENT DESCRIPTION STATUS QUO END-OF-PHASE GOAL COMMUNITY CHALLENGE NEW INSIGHTS • Implementation – Consensus algorithms & Information distribution • Adaptation – Reinforcement learning techniques • Integration with Network Coding projects: Code around network hot-spots What is the state of the art and what are its limitations? Notes from Austin: MW routing inflexible, and does not easily incorporate multi-access capacity region in wireless. Workload relaxation techniques: Tremendous value for policy synthesis based on dynamic hot-spots in the network Can these techniques be extended to wireless models? MAIN RESULT: Numerical findings: With many flows, the rate region appears smooth even in a static interference model Impact: Network cut is no longer a useful concept Infinite complexity leads to simple solution: Dynamics of 720 queues Half space relaxation provides: • KEY NEW INSIGHTS: • Extend to wireless? YES Geometric picture is very different. Interpretation: The number of resources is infinite • Structure of optimal solution to relaxation is very simple, even for very complex networks • New application of relaxation: Q-learning and TD-learning for routing and power control • Lower bound on performance and • Tools for policy synthesis HOW IT WORKS: Step 1: Estimate capacity region near estimated allocation rate vector Step 2: Construct half-space relaxation Step 3: Optimal policy for relaxation: Buffer priorities, based on coefficients of normal vector • Un-consummated union challenge: Integrate coding and resource allocation • Generally, solutions to complex decision problems should offer insight Algorithms for dynamic routing: Visualization and Optimization

3. Where to focus attention for coding and routing? Message from 15 years of research: Achieving stability is possible using very simple routing schemes. Implementation in multiple access settings possible with a bit of genius Lacking: Methods to improve delay performance, and methods to make appropriate tradeoffs between throughput and delay. • Issues addressed: • Where should effort be directed in coding and control for complex networks? • Performance evaluation: Lower bounds, and approximate optimality • Special attention to issues surrounding MANETs: • Multiple access phenomena and fading Understanding MANETs – where do we direct genius for coding and control?

4. Decision & Control Perspective D&C Perspective: Obtain the simplest model that captures essential constraints and dynamics Design highly robust control solution for the simple model Translate design to the relatively complex system D&C Perspective for networks Simple, idealized model is the dynamic fluid model Further simplification to obtain the workload relaxation Lower bounds on performance, and control solutions from the relaxation Lyapunov based design constitutes translation to network (e.g. h-MaxWeight). Understanding MANETs – Relaxations capture essential constraints and dynamics

5. Example: Dynamic Power Control Dynamic speed scaling Model: Fluid and stochastic model for arrivals, and controlled service rate Further relaxation not required in single link model Solve DP equation for fluid model Solution to fluid model used to construct architecture for reinforcement learning Optimal control solution for fluid model gives perfect architecture for on-line learning/on-line optimization Input power as function of queue length. Policy for fluid model closely approximates the optimal average delay policy for the discrete/stochastic model Approximate Dynamic Programming using Fluid and Diffusion Approximations with Applications to Power Management Power control solved using fluid model + reinforcement learning techniques

6. Example: h-MaxWeight Policy h-MaxWeight policy - introduced in ITMANET project Model: Fluid and stochastic model for arrivals, and controlled service rate Workload relaxations – dynamic generalization of network cuts Optimal control for relaxation is simple Breakthrough: Translation using Lyapunov function Optimal control of complex routing models solved using workload relaxation

7. Extension to MANETs? Issues Complexity from fading – interpreted as infinite resources in a wireless multiple access setting TDMA – complexity is nearly infinite for multi-hop interference models. Resulting capacity region again appears smooth Relaxation in previous work relied on a dominant face in the capacity region. For MANET models this region is smooth How to cope with infinite complexity in Interference models?

8. Complexity Results in Simplicity D&C Approach Step 1: First identify or approximate rate region near desired operating point. This is the basis of the dynamic fluid model Step 2: Relaxation is again justified through separation of time scales Step 3: Policy synthesis and translation as in 2008 result Step 4: Expand capacity region at hot spots through network coding Conclusion: Half space relaxation is more easily justified in MANETs

9. Summaries and challenges KEY CONCLUSION Complexity in MANETs actually results in a simple model description Challenges CAN WE LEARN? Critical information for optimization is easy to identify. How can this information be shared? CAN WE CODE? With the identification of dynamic bottlenecks, it is then evident where the capacity region can be improved. • References • S. Meyn. Stability and asymptotic optimality of generalized MaxWeight policies. SIAM J. Con Optim., 47(6):3259–3294, 2009 • W. Chen et. al. Approximate Dynamic Programming using Fluid and Diffusion Approximations with Applications to Power Management. Submitted to the 48th IEEE Conference on Decision and Control, 2009. • W. Chen, S. P. Meyn and M. Medard. Optimal Control of Stochastic Networks. Plenary Lecture at Erlang Centennial, April 2009. Manuscript in preparation. • S. P. Meyn. Control Techniques for Complex Networks. Cambridge University Press, 2007. • F. S. Melo, S. Meyn, and M. I. Ribeiro. An analysis of reinforcement learning with function approximation. In Proceedings of ICML, pages 664–671, 2008.