Belief-Propagation with Information Correction: Near Maximum-Likelihood Decoding of LDPC Codes

1. Belief-Propagation with Information Correction: Near Maximum-Likelihood Decoding of LDPC Codes Ned Varnica+, Marc Fossorier#, Alek Kavčić+ +Division of Engineering and Applied Sciences Harvard University #Department of Electrical Engineering University of Hawaii

2. Outline • Motivation – BP vs ML decoding • Improved iterative decoder of LDPC codes • Types of BP decoding errors • Simulation results

3. . . . . . . LDPC Code Graph • Parity check matrix • H • Bipartite Tanner code graph G = (V,E,C) • Variable (symbol) nodes vi  V, i = 0, 1, …, N-1 • Parity check nodes cj C, j = 0 , 1, … , Nc-1 Nc x N A non-zero entry in H an edge in G • Code rate • R = k/N, k  N-Nc • Belief Propagation • Iterative propagation of conditional probabilities

4. Standard Belief-Propagation on LDPC Codes • Locally operating • optimal for cycle-free graphs • Optimized LDPC codes (Luby et al 98, Richardson, Shokrollahi & Urbanke 99, Hou, Siegel & Milstein 01, Varnica & Kavcic 02) • sub-optimal for graphs with cycles • Good finite LDPC have an exponential number of cycles in their Tanner graphs (Etzion, Trachtenberg and Vardy 99) • Encoder constructions • BP to ML performance gap due to convergence to pseudo-codewords (Wiberg 95, Forney et al 01, Koetter & Vontobel 03)

5. Examples • Short Codes - e.g. Tanner code with N = 155, k = 64, diam = 6, girth = 8,dmin = 20 • Long Codes • - e.g. Margulis Code with N = 2640k = 1320

6. Goals • Construct decoder • Improved BP decoding performance • More flexibility in performance versus complexity • Can nearly achieve ML performance with much lower computational burden • Reduce or eliminate LDPC error floors • Applications • Can use with any “off-the-shelf” LDPC encoder • Can apply to any communication/data storage channel

7. Subgraph Definitions ^ x(L) {0,1}N Definition 1:SUC graphGS(L) = (VS(L), ES(L), CS(L))is graph induced by SUCCS(L) rRN x {0,1}N BCJR detector BP decoder channel transmitted binary vector received vector decoded vector after L iterations ^ • Syndrome s = H x(L) • CS(L) - Set of unsatisfied check nodes(SUC) CS(L) = {ci :(Hx(L))i 0} • VS(L) - Set of variable nodes incident to c  CS(L) • ES(L) - Set of edges connecting VS(L)and CS(L) ^ • dGs(v) - Degree in SUC graphGS(L)for v  V • dGs(v)  dG(v)

8. Properties of SUC graph Observation 1: The higher the degree dGs(v) of a node v  Vs(L) the more likely is v to be in error e.g. Statistics for Tanner (155,64) code blocksfor which BP failed on AWGN channel at SNR = 2.5 dB • Select v node • Perform information correction

9. Node Selection Strategy 1 Strategy 1: Determine SUC graph and select the node with maximal degree dGs in SUC graphGS(L) Select node v0 or v2 or v12

10. Properties of SUC graph, cntd Definition 2: Nodes v1 and v2 are neighbors with respect to SUC if there exist cCS(L) incident to both v1 and v2 CS(L) • nv(m) - number of neighbors of v with degree dGs = m . . . nv(2) = 1 and nv(1) = 4 . . . Observation 2: The smaller the number of neighbors (wrt to SUC graph) with high degree, the more likely v is to be in error

11. Node Selection Strategy 2 Strategy 2: Among nodes with maximal degree dGs select a node with minimal number of highest degree neighbors nv0(2)= nv12(2)= 1; nv2(2)= 2 nv0(1)= 4; nv12(1)= 6 Select node v0

12. Alternatives to Strategy 2 max • dGs = max dGs(v) • Set of suspicious nodes Sv= {v : dGs(v) = dGs } • Edge penalty function r(v,c) = (Nc- set of v nodes incident to c) • Penalty function R(v) =  r(v,c) –  r(v,c) • Select vp Svas vp = argmin R(v) • Numerous related approaches possible v  V max max max dGs(vn); if Nc \ {v} vn Nc\{v} 0 ; if Nc \ {v} =  c  Cs c  Cs max max v  Sv

13. Node Selection Strategy 3 • Decoder input on node vi • Memoryless AWGN channel: Observation 3: A variable node v is more likely to be incorrect if its decoder input is less reliable, i.e., if |O(v)| is lower Strategy 3: Among nodes with maximal degree dGs select node with minimal input reliability|O(v)|

14. Message Passing - Notation • Set of log-likelihood ratios messages on v nodes: M = (C,O) • Decoder input: O = [O (v0 ), …, O (vN-1)] • Channel detector (BCJR) input B = [B (v0 ), …, B (vN-1)] . . . . . . C . . . V . . . . . . O . . . . . . T T T

15. j = 1 j = 2 j = 3 7 3 8 1 9 4 10 start 11 5 12 2 13 6 14 Symbol Correction Procedures • Replace decoder and detector input LLRs corresponding to selected vp • O (vp) = +Sand B (vp) = +S • O (vp) = –Sand B (vp) = –S • Perform correction in stages • Test 2j combinations at stage j • For each test perform additional Kjiterations • Max number of attempts (stages) jmax

16. Symbol Correction Procedures • “codeword listing” approach • Test all 2jmax possibilities • W – collection of valid codeword candidates • Pick the most likely candidate • e.g. for AWGN channel set x = argmin d(r,w) j = 1 j = 2 j = 3 7 3 8 1 9 ^ 4 wW start 10 • “first codeword” approach • Stop at a first valid codeword • Faster convergence, slightly worse performance for large jmax 11 5 12 2 13 6 14

17. j = 1 j = 2 j = 3 j = 1 j = 2 j = 3 7 3 3 2 8 4 1 1 9 6 4 5 10 7 start 11 10 5 9 12 11 2 8 13 13 12 6 14 14 Parallel and Serial Implementation ( jmax= 3 ) start

18. j = 1 j = 2 j = 3 7 3 8 1 9 4 10 start 11 5 12 2 13 6 14 Complexity - Parallel Implementation • Decoding continued • M need to be stored • storage  (2jmax) • lower Kj required • “first codeword” procedure - fastest convergence • Decoding restarted • M need not be stored • higher Kjrequired

19. ML decoder 0 10 “codeword listing” procedure original BP (max 100 iter) -1 10 -2 10 WER -3 10 -4 10 -5 10 0 0.5 1 1.5 2 2.5 3 3.5 4 E / N [dB] b 0 Can we achieve ML? Fact 1: As jmax N, “codeword listing” algorithm with Kj = 0, for j < jmax, and Kjmax = 1 becomes ML decoder • For low values of jmax (jmax << N) performs very close to ML decoder • Tanner (N = 155, k = 64) code • jmax = 11, Kj = 10 • Decoding continued • faster decoding • M need to be stored • ML almost achieved

20. Pseudo-codewords Elimination • Pseudo-codewords compete with codewords in locally-operating BP decoding (Koetter & Vontobel 2003) • c - a codeword in an m-cover of G • i - fraction of time vi V assumes incorrect value in c •  = (0,1, …,N-1) - pseudo-codeword • pseudo-distance (for AWGN) • Eliminate a large number of pseudo-codewords by forcing symbol ‘0’ or symbol ‘1’ on nodes vp • Pseudo-distance spectra improved • Can increase min pseudo-distance if jmax is large enough

21. Types of BP decoding errors • Very high SNRs (error floor region) Stable errors on saturated subgraphs: • decoder reaches a steady state and fails • messages passed in SUC graph saturated Definition 3: Decoder D has reached a steady state in the interval [L1,L2] if Cs(L) = Cs(L1) for all L [L1,L2] • 2. Medium SNRs (waterfall region) • Unstable Errors: • decoder does not reach a steady state

22. SUC Properties in Error Floor Region Theorem 1: In the error floor region Corollary: For regular LDPC codes with • Information correction for high SNRs (error floor region) • Pros: • Small size SUC • Faster convergence • Cons: • dGsplays no role in node selection

23. ML decoder 0 “codeword listing” procedure 10 “first codeword” procedure original BP (max 100 iter) -1 10 -2 10 WER -3 10 -4 10 -5 10 0 0.5 1 1.5 2 2.5 3 3.5 4 E / N [dB] b 0 Simulation Results • Tanner (155,64) code • Regular (3,5) code • Channel: AWGN • Strategy 3 • jmax = 11, Kj = 10 • More than 1dB gain • ML almost achieved

24. Simulation Results • Tanner (155,64) code • Regular (3,5) code • Channel: AWGN • Strategy 3 • “First codeword” procedure • jmax = 4,6,8 and 11 • Kj = 10

25. Simulation Results – Error Floors • Margulis (2640,1320) code • Regular (3,6) code • Channel: AWGN • Strategy 3 • “First codeword” procedure • jmax = 5, Kj = 20 • More than 2 orders of magnitudes WER improvement

26. Simulation Results – ISI Channels • Tanner (155,64) code • Channels: • Dicode (1-D) • EPR4 (1-D)(1+D)2 • Strategy 2 • jmax = 11, Kj = 20 • 1dB gain • 20 % of detected errors are ML

27. Conclusion • Information correction in BP decoding of LDPC codes • More flexibility in performance vs complexity • Can nearly achieve ML performance with much lower computational burden • Eliminates a large number of pseudo-codewords • Reduces or eliminates LDPC error floors • Applications • Can use for any “off-the-shelf” LDPC encoder • Can apply to any communication/data storage channel