1 / 27

Diagnosis of Weaknesses in Modern Error-Correction Codes: Physics Approach

Diagnosis of Weaknesses in Modern Error-Correction Codes: Physics Approach. Misha Chertkov (Theory Division, LANL) Misha Stepanov (Theory Division, LANL) Vladimir Chernyak (Department of Chemistry, Wayne State) Bane Vasic (Department of ECE, University of Arizona).

lindamhall
Download Presentation

Diagnosis of Weaknesses in Modern Error-Correction Codes: Physics Approach

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Diagnosis of Weaknesses in Modern Error-Correction Codes: Physics Approach Misha Chertkov (Theory Division, LANL) Misha Stepanov (Theory Division, LANL) Vladimir Chernyak (Department of Chemistry, Wayne State) Bane Vasic (Department of ECE, University of Arizona) Phys.Rev.Lett. 93, 198702 (2004) IT workshop, San Antonio 10/2004 CNLS workshop, Santa Fe 01/2005 Phys.Rev.Lett. -- Nov. 25 (2005) arxiv.org/abs/cond-mat/0506037 arxiv.org/abs/it/0507031 IT workshop, Allerton 09/2005

  2. Analogous vs Digital & • Analogous Error-Correction & • Digital Error-Correction & • LDPC, Tanner graph, Parity Check & • Inference, Maximum-Likelihood, MAP & • MAP vs Belief Propagation (sum-product) & • BP is exact on the tree & • Error-correction Optimization & • Shannon-Transition & • Error-floor & Menu: Introduction • Instanton method – the idea & • Instanton-amoeba (efficient numerical method) & • Instantons on the tree (loops are neglected) & • Test code: (155,64,20) LDPC & • Different channels (Gaussian and Laplacian) & • Instantons for the Gaussian channel (Results) & • BER: Monte-Carlo vs Instanton (Gaussian channel) & • Rational structure of the instatnon (computational tree analysis) & • Instantons as medians between pseudo-codewords & • Instantons for the Laplacian channel (Results) & • BER: Monte-Carlo vs Instanton (Laplacian channel) & Instanton: proof of principles test • Conclusions & • Path Forward &

  3. Analogous vs digital Analogous Digital discrete easy to copy continuous hard to copy 0111100101 camera picture music on tape typed text computer file real number better/worse integer number yes/no menu

  4. Error-correction for analogous 4 iteration 16 iteration clean One iteration menu

  5. Digital Error-Correction N L L N N > L R=L/N - code rate Coding Decoding noise white channel example Gaussian symmetric menu

  6. mod 2 (linear coding) Low Density Parity Check Codes N=10 variable nodes Parity check matrix Tanner graph M=N-L=5 checking nodes “spin” variables - - set of constraints menu

  7. Given the detected (real) signal --- To find the most probable (integer) pre-image --- Inference Maximum-Likelihood (ML) Decoding menu

  8. constraints “partition function” “free energy” (symbol to symbol) Maximum-A-Posteriori (MAP) decoding (close to optimal) Stat Mech interpretation was suggested by N. Sourlas (Nature ‘89) Efficient but Expensive: requires operations To notice – spin glass (replica) approach for random codes: e.g. Rujan ’93, Kanter, Saad ’99; Montanari, Sourlas ’00; Montanari ’01; Franz, Leone, Montanari, Ricci-Tersenghi ‘02 Decoding (optimal) “magnetic” field log-likelihood “magnetization”=a-posteriori log-likelihood menu

  9. Sub-optimal but efficient decoding Belief Propagation (BP=sum-product) Gallager’63;Pearl ’88;MacKay ‘99 =solving Eqs. on the graph Iterative solution of BP = Message Passing (MP) Q*m*N steps instead of Q - number of MP iterations m - number of checking nodes contributing a variable node What about efficiency?Why BP is a good replacement for MAP? menu * (no loops!)

  10. Tree -- no loops -- approximation Analogy: Bethe lattice(1937) MAP BP Belief Propagation is optimal (i.e. equivalent to Maximum-A-Posteriori decoding) on a tree (no loops) Gallager ’63; Pearl ’88; MacKay ’99 Vicente, Saad, Kabashima ’00; Yedidia, Freeman, Weiss ‘01 menu

  11. Bit Error Rate (BER) {+1} is chosen for the initial code-word Probability of making an error in the bit “i” probability density for given magnetic field/noise realization (channel) measure of unsuccessful decoding Digital error-correction scheme/optimization • describe the channel/noise --- External • suggest coding scheme • suggest decoding scheme • measure BER/FER • If BER/FER is not satisfactory (small enough) goto 2 menu

  12. Shannon transition/limit BER, B SNR, s menu From R. Urbanke, “Iterative coding systems”

  13. Error floor(finite size & BP-approximate) Error floor prediction for some regular (3,6) LDPC Codes using a 5-bit decoder. From T. Richardson “Error floor for LDPC codes”, 2003 Allerton conference Proccedings. No-go zonefor brute-force Monte-Carlo numerics. Estimating very low BER is the major bottleneck in coding theory/practice menu

  14. Our (current) objective: For given (a) channel (b) coder (c) decoder to estimate BER by means of analytical and/or semi-analytical methods. Hint: BER is small and it is mainly formed at some very special “bad” configurations of the noise/”magnetic field” Instanton approach is the right way to identify the “bad” configurations and thus to estimate BER! menu

  15. Instanton Method Point at the ES closest to zero errors no errors Error-surface (ES) BER = d(noise) Weight(noise) instanton config. of the noise BER Weight instanon config of the noise Point at the ES closest to zero Laplace method Saddle-point method Steepest descent menu

  16. Instanton-amoeba(efficient-numerical scheme) unite vector in the noise space error-surface To minimize BER with respect to the unit vector !! Minimization method of our choice is simplex-minimization (amoeba) menu

  17. Instantons on the tree (semi-analytical) m=2, l=3, n=3 m=3, l=5, n=2 ITW 2004, San Antonio PRL 93, 198702 (2004) menu

  18. Parity check matrix of Tanner’s (155,64,20) code menu

  19. Different noise models for different channels Linear simplifications White Symmetric Gaussian Laplacian menu

  20. Instantons for (155,64,20) code: Gaussian channel Phys. Rev. Lett -- Nov 25, 2005 menu

  21. Bit-Error-Rate: Gaussian channel menu

  22. PRL -- Nov 25, 2005 Rational structure of instanton (computational tree analysis/explanation) min-sum 4 iterations Minimize effective action keeping the condition menu based on Wiberg ‘96

  23. Instantons as medians of pseudo-codewords PRL -- Nov 25, 2005 menu

  24. Instantons for (155,64,20) code: Laplacian channel menu IT workshop, Allerton 09/2005

  25. Bit-Error-Rate: Laplacian channel menu

  26. Conclusions We suggested amoeba-instanton method for efficient numerical evaluation of BER in the regime of high SNR (error floor). The main idea: error-floor is controlled by only a few most damaging configurations of the noise (instantons). The efficiency of the method was tested on the example of (155,64,20) LDPC code decoded with 4-iterations min-sum algorithm in Gaussian and Laplacian channels Results of the amoeba-instanton are succesfully validated against brut-force Monte-Carlo (in the regime of moderate SNR) Instantons found numerically allow theoretical interpretation in terms of the generalized computational tree approach (e.g. degeneracy) and pseudo-codewords Instantons (error-floor) is channel sensitive: configuration that is most damaging for channel A can be innocent for channel B even when performed over the same coding/decoding menu

  27. Extend the amoeba-instanton test • to study the error-floor • to develop universal computational tool-box for the error-floor analysis Other codes Other decoding schemes(e.g. number of iterations) Other channels(e.g. magnetic recording and fiber-optics specific) New decoding ?! Major challenge !!!! – to improve BP qualitatively New coding ?! Efficient (channel specific) LDPC optimization Path Forward Inter-symbol interference (1d and 2d + error-correction) Network coding Combinatorial optimization menu

More Related