Dept. of ECE, University of British Columbia

1. Joint Physical and Network Layer Optimization of Communication Systems: Current Challenges and Perspectives Dejan V. Djonin NSERC PostDoctoral Fellow Dept. of Electrical and Computer Engineering University of British Columbia E-mail: ddjonin@ece.ubc.ca www.ece.ubc.ca/~ddjonin Dept. of ECE, University of British Columbia

2. My Brief Background… (Sep 2003 - ) Postdoctoral Teaching Fellow, University of British Columbia, Department of Electrical and Computer Engineering (May 2000- Jun 2003) PhD Studies, University of Victoria, Department of Electrical and Computer Engineering Ph.D. Thesis Title: "On Some Limiting Performance Issues of Multiuser Receivers in Fading Channels" (1996 - 1999), Faculty of Electrical Engineering in Belgrade, M.Sc. studies, M.Sc. Thesis Title: "Application of Non-linear One-dimensional Maps in Generation of Error-Correction Block Codes" Dept. of ECE, University of British Columbia

3. UBC and Vancouver Dept. of ECE, University of British Columbia

4. Presentation Outline • An Overview of My Previous Professional Results • Example: Cross-layer optimization for V-BLAST transmission • under delay constraints • Problem Formulation and Introduction • Real-Time Traffic Model + Flow Control • Channel Model: Finite State Markov Model • Mathematical Framework: Stochastic Control and MDP’s • Solution Techniques • Resource allocation for imperfectly known channel models • Perspectives: - Sensor Scheduling for Network Lifetime Maximization • - Opportunistic Spectrum Access Dept. of ECE, University of British Columbia

5. An Overview of My Previous Results (1) • Non-linear mappings in the design of error-correction codes: (M.Sc. Thesis) • D.V. Djonin, D.Gacesa, "Performances of Error-Correction Codes Generated by Iterative Nonlinear Mappings", in Advances in Systems, Signals, Control & Computers, vol. 3, Durban, SAR, ISBN 0-620-23136-10, pp. 114-118, 1998. • D.V.Djonin and D.Gacesa,  "Performances of error-correction codes generated by non-linear iterative mappings", in Proc. of URSI International Symposium on Signals, Systems, and Electronics, ISSSE 98, pp. 356  360, 1998. • D.V.Djonin and L.Manojlovich, "Application of deterministic chaos in generation of error correction block codes" , in Proc. of Second IEEE International Caracas Conference on Devices, Circuits and Systems, pp. 343-347,1998. • D.V.Djonin, "Efficient Construction of Error-Correction Codes Generated by Iterative Non-linear Maps", in proc. of Telecommunications Conf. TELFOR, Belgrade Yugoslavia, 1998. • D.V.Djonin, "On the application of the theory of deterministic chaos in the generation of error-correction codes", pp. 82-85, in Proc. of the ETRAN conference, Zlatibor, Yugoslavia, 1997. • D.V.Djonin, D.Gacesa, "Performances of Error-Correction Codes Generated by Iterative Nonlinear Mappings", in Advances in Systems, Signals, Control & Computers, vol. 3, ISBN 0-620- 23136-10, pp. 114-118, Durban, 1998. Dept. of ECE, University of British Columbia

6. An Overview of My Previous Results (2) • Performance analysis and optimization of CDMA systems (Ph.D. Thesis) • D.V.Djonin and V.K.Bhargava, "Asymptotic Analysis of the Conventional Decision Feedback Receiver in Fading channels'', IEEE Trans. on Wireless Communications, pp. 1066-1078, September 2003. • D.V.Djonin and V.K.Bhargava, "On the Optimal Sequence Allocation in Flat Fading Channels'', IEEE Trans. on Wireless Communication, vol. 24, no. 5, pp. 680-689, July 2003. • D.V.Djonin and V.K.Bhargava, "Comments on 'Symmetric Capacity and Signal Design for L-out-of-K Symbol-Synchronous CDMA Gaussian Channels'", IEEE Trans. on Inf. Theory, pp. 2921-2923, vol. 50, November 2004. • D.V.Djonin and V.K.Bhargava, "Spectral Efficiency of the Feedback Receiver for Two Sets of Orthogonal Sequences" , IEEE Communication Letters, pp. 497-499, Nov. 2002. • D.V.Djonin and V.K.Bhargava,  "Spectral Efficiency of the Feedback Receiver for Multiple Orthogonal Sequence Sets", in Proc. ISWC 02 Conf., Victoria, Canada, pp.107-108, Sep. 2002. • D.V.Djonin and V. K.Bhargava, "Asymptotic Analysis of the Conventional Decision Feedback Receiver in Flat Fading Channels'', in Proc. of ICC 2002, pp. 1368  1372, April-May 2002. • D.V.Djonin and V.K.Bhargava,  "Asymptotic Analysis of the Optimal Spreading Sequence Allocation in Flat Fading Channels" , in Proc. of VTC 2002, pp.582-585, September 2002. • D.V.Djonin, and V. K. Bhargava, "Low Complexity Receivers for Over-Saturated CDMA System'', in Proc. of the Globecom 2001, vol. 2, pp. 846 -850, Nov. 2001. Dept. of ECE, University of British Columbia

7. An Overview of My Previous Results (3) • Performance analysis of communications systems using the theory of stochastic majorization • D.V.Djonin, P. Tarasak and V.K.Bhargava,  "On the Influence of the Power Delay Profile on the Performance of Diversity Combining Systems" , Proc. of Globecom 2003 Conference, San Francisco, CA, vol. 3, pp. 1659 - 1663, December 2003. • D.V.Djonin and V.K.Bhargava,  "On the Influence of the Power Delay Profile on the Performance of Diversity Combining Systems" , IEEE Transactions on Wireless Communications, pp. 1854-1861, vol. 3, Sep. 2004. Dept. of ECE, University of British Columbia

8. An Overview of My Previous Results (4) • Results on Space-Time Code • K. C. B. Wavegedara, D.Djonin, and V.K.Bhargava, "Space-Time Coded Uplink Transmission with Decision Feedback Sequence Estimation", in Proc. of Globecom 2004 Conference, Dallas, TX, vol. 6, pp. 3448 - 3453, Nov.-Dec. 2004. • K. C. B. Wavegedara, D.V.Djonin, and V.K.Bhargava, "Space-Time Coded Uplink Transmission with Decision Feedback Sequence Estimation", in IEEE Trans on Wireless Comm., Nov 8th, 2005. (full paper). Dept. of ECE, University of British Columbia

9. An Overview of My Previous Results (5) • Rate and Power Control algorithms for time-varying wireless channels using Markov Decision Processes • A.Karmokar, D.V.Djonin and V.K.Bhargava, "Cross-layer Rate and Power Adaptation Strategies for IR-HARQ Systems over Fading Channels with Memory: A SMDP-based Approach", submitted to Trans on Communications, February 21st, 2006. • D.V.Djonin and V.Krishnamurthy, " V-BLAST Power and Rate Control under Delay Constraints in Markovian Fading Channels -Structured Policy Learning", submitted to IEEE Trans. on Signal Processing, Jan. 25, 2006. • D.V.Djonin and V.Krishnamurthy, " V-BLAST Power and Rate Control under Delay Constraints in Markovian Fading Channels -Optimality of Monotonic Policies", submitted to IEEE Trans. on Signal Processing, Jan. 05, 2006. • A.Karmokar, D.V.Djonin and V.K.Bhargava, "Delay Aware Power Adaptation for Incremental Redundancy Hybrid ARQ over Fading Channels with Memory", to be presented at the ICC 2006 Conference, Istanbul, Turkey. • D.V.Djonin and V.Krishnamurthy, "Structural Results on the Optimal Transmission Scheduling Policies and Costs for Correlated Sources and Channels", in CDC 2005, (invited paper). • Md.J.Hossain, D.V.Djonin and V.K.Bhargava, "Delay Limited Optimal and Suboptimal Power and Bit Loading Algorithms for OFDM Systems over Correlated Fading", presented at the Globecom 2005, St. Louis, Dec. 2005. • Md.J.Hossain, D.Djonin and V.K.Bhargava, "Power and Rate Adaptation for OFDM System over Correlated Fading Channels",  presented at the IST 2005 Symposium, Dresden, Germany, June 2005. • D.V.Djonin and V.K.Bhargava, "An Upper Bound on the Throughput of Opportunistic Transmission in a Multiple-Access Fading Channel", IEEE Trans. on Comm., pp. 1618-1621, vol. 52, Oct. 2004. • A.Karmokar, D.V.Djonin and V.K.Bhargava, "Optimal and Suboptimal Packet Scheduling over Time-Varying Flat Fading Channels",  to be published in IEEE Trans on Wireless Comm., (full paper), Jan., 2006. • A.Karmokar, D.Djonin and V.K.Bhargava, "Delay Constrained Rate and Power Adaptation over Correlated Fading Channels", in Proc. of Globecom 2004 Conference, Dallas, TX, vol. 5, pp. 2941 - 2945, Nov.-Dec. 2004. • D.V.Djonin, A.Karmokar and V.K.Bhargava, "Rate and Power Adaptation over Correlated Fading Channels under Different Buffer Cost Constraints", submitted to Trans on Vehicular Technology, March. 9th, 2004. • D.V.Djonin, A. Karmokar and V.K.Bhargava,  "Optimal and Suboptimal Packet Scheduling over Time-Varying Flat Fading Channels" in Proc. of ICC 2004, pp. 906-910, Paris, France, June 2004. • A.Karmokar, D.V.Djonin and V.K.Bhargava, "POMDP Based Coding Rate Adaptation for Hybrid ARQ Systems over Fading Channels with Memory", submitted to Trans on Wireless Communications, August 18th, 2004. Dept. of ECE, University of British Columbia

10. Common Themes • Performance Analysis of Communication Systems • Performance Improvement of Communication Systems Through • On-line and Off-line Optimization • Main tool: Stochastic Control + Markov Decision Processes Dept. of ECE, University of British Columbia

11. Presentation Outline • An Overview of My Previous Professional Results • Example: Cross-layer optimization for V-BLAST transmission • under delay constraints • Problem Formulation and Introduction • Real-Time Traffic Model + Flow Control • Channel Model: Finite State Markov Model • Mathematical Framework: Stochastic Control and MDP’s • Solution Techniques • Resource allocation for imperfectly known channel models • Perspectives: - Sensor Scheduling for Network Lifetime Maximization • - Opportunistic Spectrum Access Dept. of ECE, University of British Columbia

12. Problem Formulation and Introduction • Modern and future wireless networks will support different services • with a wide range of quality of service requirements such as delay, rate, BER • Consideration of Transmission Latency is of crucial interest • for some applications (real-time high quality audio, video transmission) • However, time-varying nature of a wireless channel poses a challenging • task to delivering a wide variety of services • the effect is similar to congestion in wireline networks • the need for transmission buffer • transmitted signals are delayed • Do these methods only apply to wireless channels? • The solution is through adaptation of transmission parameters based • on the current state and the statistical model of the channel and • supported traffic • Essentially a Cross-layer optimization approach Dept. of ECE, University of British Columbia

13. Power versus Delay Tradeoff: A Simple Illustration A B Dept. of ECE, University of British Columbia

14. OSI Model Data Link (Layer 2) At this layer, data packets are encoded and decoded into bits. It furnishes transmission protocol knowledge and management and handles errors in the physical layer, flow control and frame synchronization. The data link layer is divided into two sublayers: The Media Access Control (MAC) layer and the Logical Link Control (LLC) layer. The MAC sublayer controls how a computer on the network gains access to the data and permission to transmit it. The LLC layer controls frame synchronization, flow control and error checking. Physical (Layer 1)This layer conveys the bit stream - electrical impulse, light or radio signal -- through the network at the electrical and mechanical level. It provides the hardware and software means of sending and receiving data on a carrier. Dept. of ECE, University of British Columbia

15. V-BLAST transmission control model • Let fn denote the number of packets arriving at the buffer in time n. • Transmission adaptation parameters can include power, • error-correction or source coding rate (flow control) • At the beginning of the n-th time slot, the scheduler controls the packet retrievals from the buffer and bit-loading across carriers. Dept. of ECE, University of British Columbia

16. Channel Model: FSMC • For example, a slowly varying flat Fading Rayleigh channel can be • represented as a Finite State Markov Chain (FSMC) as shown in figure: • Channel can also be modeled as an Auto Regressive (AR) model Dept. of ECE, University of British Columbia

17. Presentation Outline • An Overview of My Previous Professional Results • Example: Cross-layer optimization for V-BLAST transmission • under delay constraints • Problem Formulation and Introduction • Real-Time Traffic Model + Flow Control • Channel Model: Finite State Markov Model • Mathematical Framework: Stochastic Control and MDP’s • Solution Techniques • Resource allocation for imperfectly known channel models • Perspectives: - Sensor Scheduling for Network Lifetime Maximization • - Opportunistic Spectrum Access Dept. of ECE, University of British Columbia

18. Markov Decision Processes (MDP) Markov Chain: Example p(S2|S1) p(S2|S2) S1 p(S1|S1) S2 p(S1|S2) Markov Decision Processes: Example for state S1 Action U1, c(S1,U1) p(S2|S1,U1) Action U2,c(S1,U2) p(S2|S1,U2) p(S1|S1,U1) S1 S2 p(S1|S1,U2) Dept. of ECE, University of British Columbia

19. Constrained MDPs • What happens if in addition to the immediate costs, c(s,u), there is an another cost d(s,u)that corresponds to a constraint? I.e. optimization problem is: • The answer can be found in the theory of Constrained Markov Decision Processes (CMDP). CMDP can be expressed as equivalent unconstrained MDP using Lagrangian Approach: • Note that policies do not have to be deterministic in CMDPs. In general optimal policies for CMDPs are randomized. Dept. of ECE, University of British Columbia

20. V-BLAST transmission control model • Let fn denote the number of packets arriving at the buffer in time n. • Transmission adaptation parameters can include power, • error-correction or source coding rate (flow control) • At the beginning of the n-th time slot, the scheduler controls the packet retrievals from the buffer and bit-loading across carriers. Dept. of ECE, University of British Columbia

21. Presentation Outline • An Overview of My Previous Professional Results • Example: Cross-layer optimization for V-BLAST transmission • under delay constraints • Problem Formulation and Introduction • Real-Time Traffic Model + Flow Control • Channel Model: Finite State Markov Model • Mathematical Framework: Stochastic Control and MDP’s • Solution Techniques • Resource allocation for imperfectly known channel models • Perspectives: - Sensor Scheduling for Network Lifetime Maximization • - Opportunistic Spectrum Access Dept. of ECE, University of British Columbia

22. Sample Results (1) • As fading rate , the rate of decrease of average power . • As the number of antennas , average power  Dept. of ECE, University of British Columbia

23. Sample Results (2) Dept. of ECE, University of British Columbia

24. Structural Results Extracted from the paper: MIMO Power and Rate Control under Delay Constraints in Markovian Fading Channels – Optimality of Monotonic Policies, Dejan V. Djonin, Vikram Krishnamurthy, submitted to Trans. on Signal Processing, Jan 2006, revised May 2006. also to be presented at the ISIT Conference, Seattle 2006. Dept. of ECE, University of British Columbia

25. Resource allocation for imperfectly known channel models (1) • This a challenging problem as the policy has to be “learned” on-line as the actions are being applied and observations on the incurred cost are collected. • The appropriate framework for the solution of this problem is to consider Q-learning, which is a version of stochastic approximation algorithm. • For details on Q-algorithm and related topics have a look at: • D. Bertsekas and J.Tsitsiklis, “Neuro-Dynamic Programming” Dept. of ECE, University of British Columbia

26. Resource allocation for imperfectly known channel models (2) Extracted from the Paper: Dejan Djonin, Vikram Krishnamurthy, “V-BLAST Power and Rate Control under Delay Constraints in Markovian Fading Channels- Structured Policy Learning Algorithm”, submitted to Trans on Signal Processing, Jan 2006. Dept. of ECE, University of British Columbia

27. Resource allocation for imperfectly known channel models (3) • Advantages of Learning based algorithms for Optimal Control • It can be proved that Q-learning algorithm converges to the optimal solution with probability 1 (both structured and non-structured Q-learning) • These algorithms are suitable for unknown channel environments whose statistics changes slowly over time • It is possible to incorporate more complicated delay costs in the model: average delay cost, maximum delay guarantees, delay profile shaping Dept. of ECE, University of British Columbia

28. Presentation Outline • An Overview of My Previous Professional Results • Example: Cross-layer optimization for V-BLAST transmission • under delay constraints • Problem Formulation and Introduction • Real-Time Traffic Model + Flow Control • Channel Model: Finite State Markov Model • Mathematical Framework: Stochastic Control and MDP’s • Solution Techniques • Resource allocation for imperfectly known channel models • Perspectives: - Sensor Scheduling for Network Lifetime Maximization • - Opportunistic Spectrum Access Dept. of ECE, University of British Columbia

29. Sensor Scheduling for Network Lifetime Maximization h1 hN h2 eN e1 e2 Sensor 1 Sensor N Sensor 2 Collaborators: Qing Zhao, Yunxia Chen (UC Davis), V.Krishnamurthy(UBC) Dept. of ECE, University of British Columbia

30. Sensor Scheduling for Network Lifetime Maximization • The problem is how to design an optimal sensor scheduling policy to • maximize the lifetime of a network as a whole • The sensor network is considered to be functioning while a predefined portion of sensors have enough energy to transmit • Transmission energy is dependent on the channel conditions: Wi= f(hi) • Two approaches to model and solve the problem: • centralized scheduling, global state MDP • decentralized scheduling, multi-armed bandit formulation • Some results on this topic are given in: • 1) Y. Chen, Q. Zhao, V. Krishnamurthy and D.V.Djonin, "Transmission Scheduling for Optimizing Sensor Network Lifetime: A Stochastic Shortest Path Approach", submitted to IEEE Trans. on Signal Processing, Jan. 2006, revised May 2006. • 2) Y. Chen, Q. Zhao, V. Krishnamurthy and D.V.Djonin, "Transmission Scheduling for Sensor Network Lifetime Maximization: A Shortest Path Bandit Formulation", presented at the ICASSP 2006 Conference, Toulouse, France, May 2006. Dept. of ECE, University of British Columbia

31. Sensor Scheduling for Network Lifetime Maximization: Open Problems • Further simplification of the computation of the optimal sensor scheduling policy for centralized scheduling • Incorporation of the content based scheduling (the information sent by different schedulers can be prioritized) • Adaptive Source Coding Control prior to transmission • Multiple Access transmission resolution Dept. of ECE, University of British Columbia

32. Opportunistic Spectrum Access Channel 1 Channel 2 Channel 3 Channel 4 Channel 5 Channel 6 Channel 7 B1 B2 B3 B4 B5 B6 B7 p1 p2 p4 p5 p6 p7 p3 Scheduler = f(p1,…, p7) pi = Prob [channel i is available] Bi = Bandwidth of the Channel i Collaborator: Qing Zhao (UC Davis) Dept. of ECE, University of British Columbia

33. Opportunistic Spectrum Access: Open Problems • Design of a computationally efficient Spectrum Access control policy • Exploration of the decentralized formulation of the problem: a restless multi-armed bandit formulation • Protocol design for coordination between primary and secondary users Dept. of ECE, University of British Columbia

34. Thank You for Your Attention ! Dept. of ECE, University of British Columbia