Graphical Models and Message Passing Receivers for Interference Limited Communication Systems

1. Graphical Models and Message Passing Receivers for Interference Limited Communication Systems Marcel Nassar PhD Defense Committee Members: Prof. Gustavo de Veciana Prof. Brian L. Evans (supervisor) Prof. Robert W. Heath Jr. Prof. Jonathan Pillow Prof. Haris Vikalo April 17, 2013

2. Outline • Uncoordinated interference in communication systems • Effect of interference on OFDM systems • Prior work on receivers in uncoordinated interference • Message-passing receiver design • Learning interference model parameters: robust receivers • Summary and future work Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

3. Modern Communication Systems Single Carrier System: frequency selective fading Noise/Interference Channel + 0110010 Implementation complexity equalizer 0 1 1 Orthogonal Frequency Division Multiplexing (OFDM): 0 0 1 0 Simpler Equalization Noise/Interference Channel Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary + + 0110010 N sub-bands

4. Interference in Communication Systems Wireless LAN in ISM band Powerline Communication Non-interoperable standards Co-Channelinterference Platform Coexisting Protocols Non-CommunicationSources Electromagnetic emissions Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

5. Interference Management • An active area of research … • Orthogonal Access • MAC Layer Access: Co-existence [Rao2002] [Andrews2009] • Precoding Techniques: Inter-cell interference cancellation [Boudreau2009], network MIMO (CoMP)[Gesbert2007], Interference Alignment [Heath2013] • Successive Interference Cancellation[Andrews2005] • What about residual interference? • What about non-communicating sources? • What about non-cooperative sources? UncoordinatedInterference Complementary approach: statistically model and mitigate Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

6. Thesis Contributions • Contribution I: Uncoordinated Inference Modeling • Contribution II: Receiver Design • Contribution III: Robust Receiver Design Thesis Statement: Receivers can leverage interference models to enhance decoding and increase spectral efficiency in interference limited systems. Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

7. Uncoordinated Interference Modeling

8. Statistical-Physical Modeling Wireless Systems PowerlineSystems WiFi, Ad-hoc [Middleton77, Gulati10] Rural, Industrial, Apartments [Middleton77,Nassar11] Middleton Class-A A Impulsive Index Mean Intensity WiFi Hotspots [Gulati10] Dense Urban, Commercial [Nassar11] Gaussian Mixture (GM) : #of comp. comp. probability comp. variance Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

9. Empirical Modeling – WiFi Platform Gaussian HMM Model: • [Data provided by Intel] 1 2 Gaussian Mixture Model: Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

10. Empirical Modeling – Powerline Systems • Markov Chain Model [Zimmermann2002] 1 … 2 1 5 1 2 Impulsive Background [Katayama06] Measurement Data Proposed Model Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

11. Receiver Design

12. OFDM Basics System Diagram noise +interference SymbolMapping LDPC Coded DFT Inverse DFT Source + 1+i 1-i -1-i 1+i … 0111 … channel Noise Model Receiver Model • After discarding the cyclic prefix: • After applying DFT: • Subchannels: total noise where background noise interference and GM or GHMM Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

13. Effect of GM Interference on OFDM Single Carrier (SC) OFDM Single Carrier vs. OFDM (symbol by symbol decoding) DFT • Impulse energy concentrated • Symbol lost with high probability • Symbol errors independent • Min. distance decoding is MAP-optimal • Minimum distance decoding under GM: • Impulse energy spread out • Symbol lost ?? • Symbol errors dependent • Disjoint minimum distance is sub-optimal • Disjoint minimum distance decoding: Impulse energy low → OFDM sym. recovered Impulse energy high → OFDM sym. lost • In theory, with joint MAP decodingOFDM Single Carrier tens of dBs • [Haring2002] Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

14. OFDM Symbol Structure Coding Data Tones • Added redundancy protects against errors • Symbols carry information • Finite symbol constellation • Adapt to channel conditions Pilot Tones Null Tones • Edge tones (spectral masking) • Guard and low SNR tones • Ignored in decoding • Known symbol (p) • Used to estimate channel pilots → linear channel estimation → symbol detection → decoding But, there is unexploited information and dependencies Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

15. Prior Work All don’t consider the non-linear channel estimation, and don’t use code structure Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

16. Joint MAP-Decoding • The MAP decoding rule of LDPC coded OFDM is: • Can be computed as follows: depends on linearly-mixed N noise samples and L channel taps LDPC code non iid & non-Gaussian Very high dimensional integrals and summations !! Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

17. Belief Propagation on Factor Graphs • Graphical representation of pdf-factorization • Two types of nodes: • variable nodes denoted by circles • factor nodes (squares): represent variable “dependence “ • Consider the following pdf: • Corresponding factor graph: Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

18. Belief Propagation on Factor Graphs • Approximates MAP inference by exchanging messages on graph • Factor message = factor’s belief about a variable’s p.d.f. • Variable message = variable’s belief about its own p.d.f. • Variable operation = multiply messages to update p.d.f. • Factor operation = merges beliefs about variable and forwards • Complexity = number of messages = node degrees Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

19. Coded OFDM Factor Graph unknown channel taps Unknown interference samples Symbols Information bits Coding & Interleaving Bit loading & modulation Received Symbols Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

20. BP over OFDM Factor Graph MC Decoding LDPC Decoding via BP [MacKay2003] Node degree=N+L!!! Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

21. Generalized Approximate Message Passing[Donoho2007,Rangan2010] Decoupling via Graphs Estimation with Linear Mixing variables observations coupling • Generally a hard problem due to coupling • Regression, compressed sensing, … • OFDM systems: • If graph is sparse use standard BP • If dense and ”large”→Central Limit Theorem • At factors nodes treat as Normal • Depend only on means and variances of incoming messages • Non-Gaussian output → quad approx. • Similarly for variable nodes • Series of scalar MMSE estimation problems: messages Interference subgraph channel subgraph given given and and 3 types of output channels for each Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

22. Proposed Message-Passing Receiver Schedule coded bits to symbols symbols to Run channel GAMP Run noise “equalizer” to symbols Symbols to coded bits Run LDPC decoding Turbo Iteration: GAMP LDPC Dec. GAMP Equalizer Iteration: Run noise GAMP MC Decoding Repeat Initially uniform Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

23. Receiver Design & Complexity Design Freedom • Not all samples required for sparse interference estimation • Receiver can pick the subchannels: • Information provided • Complexity of MMSE estimation • Selectively run subgraphs • Monitor convergence (GAMP variances) • Complexity and resources • GAMP can be parallelized effectively Notation : # tones : # coded bits : # check nodes : set of used tones Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

24. Simulation Settings Interference Model • Two impulsive components: • 7% of time/20dB above background • 3% of time/30dB above background • Two types of temporal dynamics: • i.i.d. samples • Hidden Markov Model Receiver Parameters Definitions 15 GAMP iterations 5 turbo iterations FFT Size 256 (PLC) FFT Size1024 (Wireless) 50 LDPC iterations SER: Symbol Error Rate BER: Bit Error Rate SNR: Signal to “noise + interference” power ratio Introduction |Message Passing Receivers | Simulations| Robust Receivers| Summary

25. Simulation - Uncoded Performance use LMMSE channel estimate performs well when interference dominates time-domain signal Settings use only known tones, requires matrix inverse 5 Taps GM noise 4-QAM N=256 15 pilots 80 nulls 2.5dB better than SBL within 1dB of MF Bound 15db better than DFT Matched Filter Bound: Send only one symbol at tone k Introduction |Message Passing Receivers | Simulations| Robust Receivers| Summary

26. Simulation – Modeling Gain correct marginal: 4dB gain temporal dependency: extra 4dB amplitude accuracy is not important using all tones gives 8dB gain amplitude accuracy gives 7db gain Settings flat ch. N=256 60 nulls Introduction |Message Passing Receivers | Simulations| Robust Receivers| Summary

27. Simulation – Tone Map Design Typical configuration performs significantly worse Tone Map Design • How to allocate tones? • Limited resources → select tones? • Optimal solution not known……. • Dictionary coherence • For same node type Introduction |Message Passing Receivers | Simulations| Robust Receivers| Summary

28. Simulation - Coded Performance one turbo iteration gives 9db over DFT Settings 5 turbo iterations gives 13dB over DFT 10 Taps GM noise 16-QAM N=1024 150 pilots Rate ½ L=60k Integrating LDPC-BP into JCNED by passing back bit LLRs gives 1 dB improvement Introduction |Message Passing Receivers | Simulations| Robust Receivers| Summary

29. Robust Receiver Design

30. Recall - Coded OFDM Factor Graph contains parameters that might be unknown Introduction |Message Passing Receivers | Simulations| Robust Receivers| Summary

31. Learning the Interference Model Training-Based Robust Receiver parameterestimate quiet period model training joint estimation & detection corruptedtransmission Detection 01010011 corruptedtransmission Detection 01010011 • Computationally simpler detection • For slowly varying environments • Suffer from model mismatch in rapidly varying environments • More computation for parameter estimation (but not always) • Adapts to rapidly varying environment Introduction |Message Passing Receivers | Simulations| Robust Receivers| Summary

32. Parameter Estimation via EM Algorithm EM Algorithm • Simplifies ML Estimation by: • Marginalize over latent • Maximize w.r.t parameters • Marginalization easy for directly observed GM and GHMM samples EM for Robust Receivers Approximate marginalization using GAMP messages Introduction |Message Passing Receivers | Simulations| Robust Receivers| Summary

33. Sparse Bayesian Learning via GAMP Sparse Bayesian Learning • Bayesian approach to compressive sensing • Use data to fit via EM • If e is sparse lot of end up zero • Requires big matrix inverse • Use only null tones • Linear channel estimation Prior: SBL via GAMP • Integrate into GAMP estimation functions • Linear input estimator • Can include all tones and code Introduction |Message Passing Receivers | Simulations| Robust Receivers| Summary

34. Simulation Result – SBL via GAMP SBL via EM using only null tones SBL via GAMP using all tones Settings 5 Taps GM noise 4-QAM N=256 15 pilots 80 nulls EM Parameter estimation Blind EM Parameter Estimation Introduction |Message Passing Receivers | Simulations| Robust Receivers| Summary

35. FPGA Test System for G3 PLC Receiver • Simplified message-passing receiver using only null tones • In collaboration with Karl Nieman NI PXIe-7965R (Virtex 5) NI PXIe-1082 Real-time host Introduction |Message Passing Receivers | Simulations| Robust Receivers| Summary

36. Summary • Significant performance gains if receiver accounts for uncoordinated interference • Proposed solution combines all available information to perform approximate-MAP inference • Asymptotic complexity similar to conventional OFDM receiver • Can be parallelized • Highly flexible framework: performance vs. complexity tradeoff • Robust for fast-varying interference environments

37. Future Work • Temporal Modeling of Uncoordinated Interference • Wireless Networks • Powerline Networks • Tractable Inference • Pilot and null tone allocation in impulsive noise • Coherence not optimal • Trade-off between channel and noise estimation • Extension to different interference and noise models • Cyclostationary noise • ARMA models for spectrally shaped noise • Mitigation of narrowband interferers • Sparse in frequency domain

38. Related Publications M. Nassar, K. Gulati, M. R. DeYoung, B. L. Evans and K. R. Tinsley, "Mitigating Near-Field Interference in Laptop Embedded Wireless Transceivers", Journal of Signal Processing Systems, Mar. 2009, invited paper. M. Nassar, X. E. Lin and B. L. Evans, "Stochastic Modeling of Microwave Oven Interference in WLANs", Proc. IEEE Int. Comm. Conf., Jun. 5-9, 2011, Kyoto, Japan. M. Nassarand B. L. Evans, "Low Complexity EM-based Decoding for OFDM Systems with Impulsive Noise", Proc. Asilomar Conf. on Signals, Systems and Computers, Nov. 6-9, 2011, Pacific Grove, CA USA. M. Nassar, K. Gulati, Y. Mortazavi, and B. L. Evans, "Statistical Modeling of Asynchronous Impulsive Noise in Powerline Communication Networks", Proc. IEEE Int. Global Comm. Conf.. Dec. 5-9, 2011, Houston, TX USA. J. Lin, M. Nassarand B. L. Evans, "Non-Parametric Impulsive Noise Mitigation in OFDM Systems Using Sparse Bayesian Learning", Proc. IEEE Int. Global Comm. Conf., Dec. 5-9, 2011, Houston, TX USA. M. Nassar, A. Dabak, I. H. Kim, T. Pande and B. L. Evans, "Cyclostationary Noise Modeling In Narrowband Powerline Communication For Smart Grid Applications“, Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing, March 25-30, 2012, Kyoto, Japan. M. Nassar, J. Lin, Y. Mortazavi, A. Dabak, I. H. Kim and B. L. Evans, "Local Utility Powerline Communications in the 3-500 kHz Band: Channel Impairments, Noise, and Standards", IEEE Signal Processing Magazine, Sep. 2013 J. Lin, M. Nassar, and B. L. Evans, ``Impulsive Noise Mitigation in Powerline Communications using Sparse Bayesian Learning'', IEEE Journal on Selected Areas in Communications, vol. 31, no. 7, Jul. 2013, to appear. K. F. Nieman, J. Lin, M. Nassar, B. L. Evans, and K. Waheed, ``Cyclic Spectral Analysis of Power Line Noise in the 3-200 kHz Band'', Proc. IEEE Int. Symp. on Power Line Communications and Its Applications, Mar. 24-27, 2013, Johannesburg, South Africa. Won Best Paper Award. M. Nassar, P. Schniter and B. L. Evans, ``Message-Passing OFDM Receivers for Impulsive Noise Channels'', IEEE Transactions on Signal Processing, to be submitted.

39. Thank you

40. BACK UP SLIDES