Myopic and Optimal Distributed Routing

1. Myopic and Optimal Distributed Routing Wei Chen and Sean P. Meyn ECE & CSL University of Illinois

2. MaxWeight: Issues Raised before and after 2007 Routing requires information. In the MaxWeight policy, this information is obtained through local queue length values. This can lead to irrational behavior when information is scarce • Issues addressed by 12/07 • Why does MW work? Answer led to h-MW policies introduced in CTCN – Performance evaluation and approximate optimality • Improved robustness • Specialization to routing • Issues addressed in 2008 • Understanding dynamic multi-commodity flow problem, providing workload relaxation • Distributed implementation in wireless setting based on message passing • Simulation and analysis Example: (and Leith, 2007 Subramanian, submitted). MaxWeight or Backpressure routing will send packets upstream! Better performance through geometric insight and better information sharing

3. Context of the reported work Work shows how important global information can be used if available. Generally, amount of global information required for approximate optimization is low

4. Optimizing MaxWeight for Routing ACHIEVEMENT DESCRIPTION STATUS QUO END-OF-PHASE GOAL COMMUNITY CHALLENGE NEW INSIGHTS • What is the state of the art and what are its limitations? • MW routing inflexible with respect to performance improvement; requires global information (wireless); very little flexibility • New h-MW technique requires information, but is highly flexible, and approximately optimal in certain cases • MAIN RESULT: • Decentralized implementation, use of consensus algorithms • Simulation study for wireless model • Multiple cuts/bottlenecks addressed • Performance impact of consensus algorithms, and other learning schemes used in this project • Simulation study in realistic wireless setting • Full performance analysis of multiple bottlenecks • Integration with Network Coding projects • KEY NEW INSIGHTS: • Goal: develop insights of 07 • Distributed implementation via message-passing • New learning technique based on Oja’s algorithm • Learning cuts/workload through stochastic approximation techniques • Construction of h for models with multiple bottlenecks based on DP insights for w. relaxation HOW IT WORKS: Step 1: Estimation of workload – generalization of network cuts Step 2: Estimation of congestion Step 3: Choice of h0 - piecewise quadratic Special case: Single dominant destination gives h0 quadratic function of workload, cost, and effective cost w.r.t. workload relaxation • Un-consummated union challenge: Integrate coding and resource allocation • Generally, solutions to complex decision problems should offer insight Algorithms for dynamic routing: Visualization and Optimization

5. Simulations for Multiple Traffic Streams Network approximately 100 nodes. Single destination, multiple sources MaxWeight compared to policy based on logarithmic perturbation of Simulation for high load: 50% improvement over greedy, 25% over MW Greedy Approximation of DP solution Source of performance loss in MW : Cycling back and forth across bottleneck network cut leads to higher workload values Performance improves for functions h that more closely approximate DP solution

6. Summaries and challenges CHALLENGES: Implementation requires learning on several levels – from global architecture, to local policy parameters. CONCLUSIONS: In wireless model with multiple bottlenecks we again see 25% delay improvement in simulation. Logarithmic perturbation gives universally stabilizing policies, with no evident performance loss in experiments. Learning insights: 1. Message passing can lead to performance loss - analysis underway. 2. Stochastic approximation Oja’s algorithm completed. Application to network decomposition. 3. Local capacity learning investigated. SCIENTIFIC FOUNDATIONS Stochastic Lyapunov theory + workload relaxation to approximate DP solution PERFORMANCE Without attention to bottlenecks, a decentralized routing algorithm will create inefficiency through cycling. Stability has been established for logarithmic perturbation - results from simulation studies give optimism. Analysis is required. LEARNING Distributed learning of workload in a dynamic setting Distributed learning of control parameters Learning of channel characteristics Research bottlenecks: Learning dynamic bottleneck location and workload

7. References • S. P. Meyn. Stability and asymptotic optimality of generalized MaxWeight policies. To appear, SIAM J. Control Opt. (Preliminary version to appear at the 46th IEEE Conference on Decision and Control, December 2007). • W. Chen and S. P. Meyn Optimizing MaxWeight For Routing. In preparation. • S. P. Meyn. Control Techniques for Complex Networks. Cambridge University Press, 2007. Chinese edition to appear. • S. P. Meyn and R. L. Tweedie, Markov Chains and Stochastic Stability. CUP Mathematical Library edition, to appear 2008. • C. Lin, V. V. Veeravalli, and S. P. Meyn. Distributed beamforming with feedback: Convergence analysis. Submitted to the IEEE Transactions on Information Theory. http://arxiv.org/abs/0806.3023, 2008 (adaptation at nodal level) • V. Borkar and S. Meyn. Oja’s algorithm for graph clustering and Markov spectral decomposition. In Proceedings of VALUETOOLS, Athens, Greece, 2008 (adaptation at network level) • References in on-going research: CTCN and … • Iterative Scheduling Algorithms, M. Bayati, B. Prabhakar, D. Shah and M. Sharma, Proceedings of IEEE Infocom 2007 • Distributed Subgradient Methods for Multi-agent Optimization Angelia Nedic and Asuman Ozdaglar. To appear in IEEE Transactions on Automatic Control. • Polynomial Complexity Algorithms for Full Utilization of Multi-hop Wireless Networks Atilla Eryilmaz, Asuman Ozdaglar and Eytan Modiano. INFOCOM 2007.

8. Optimizing MaxWeight: From 12/07 Meeting ACHIEVEMENT DESCRIPTION STATUS QUO END-OF-PHASE GOAL COMMUNITY CHALLENGE NEW INSIGHTS What is the state of the art and what are its limitations? MW routing inflexible with respect to performance improvement MW corresponds to h-myopic, with h quadratic. Key geometric property of quadratic identified by Meyn prior to July meeting. MAIN RESULT: h-myopic policy is universally stabilizing Application to policy synthesis for approximately optimal performance (delay or backlog) in heavy traffic, with log regret • Decentralized implementation, use of consensus algorithms • Wireless models: Apply D. Shah’s insights on maxproduct convergence • Full analysis of multiple bottlenecks • Integration with Network Coding projects: Can we code around network hot-spots? Numerical study underway Investigate performance and feasibility: 100 nodes, multiple arrivals. Only wireline models investigated to-date. Excellent performace as predicted by theory • KEY NEW INSIGHTS: • New perturbation technique: • Application to routing & refinements for decentralization • Heavy traffic optimality • Taylor series approximation gives interpretation as adaptive MaxWeight - Diagonal matrix adapts to varying congestion Decentralized implementation appears feasible. HOW IT WORKS: Step 1: Estimation of network cuts Step 2: Estimation of congestion on either side Step 3: Choice of h0 - piecewise quadratic Special case: Single dominant destination gives h0 quadratic function of workload, cost, and effective cost w.r.t. workload relaxation • Un-consummated union challenge: Integrate coding and resource allocation • Generally, solutions to complex decision problems should offer insight Algorithms for dynamic routing: Visualization and Optimization