1 / 33

Extremal Problems of Information Combining

Extremal Problems of Information Combining. Information Combining: formulation of the problem Mutual Information Function for the Single Parity Check Codes More Extremal Problems of Information Combining Solutions (with the help of Tchebysheff Systems)

Sophia
Download Presentation

Extremal Problems of Information Combining

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Extremal Problems of Information Combining • Information Combining: formulation of the problem • Mutual Information Function for the Single Parity Check Codes • More Extremal Problems of Information Combining • Solutions (with the help of Tchebysheff Systems) for the Single Parity Check Codes Alexei Ashikhmin Joint work with Yibo Jiang, Ralf Koetter, Andrew Singer

  2. Channel Encoder Channel Channel Information Transmission APP Decoder Density function of the channel is not known We only know

  3. Optimization Problem We assume that and that the channel is symmetric Problem 1 Among all probability distributions such that determine the probability distribution that maximizes (minimizes) the mutual information at the output of the optimal decoder

  4. To variable nodes Check nodes processing Variable nodes processing From variable nodes Decoder of single parity check code Input from channel Interleaver

  5. Problem is Solved Already 1. I.Land, P. Hoeher, S.Huettinger, J. Huber, 2003 2. I.Sutskover, S. Shamai, J. Ziv, 2003

  6. erasure Repetition code: The Binary Erasure Channel (BEC) is the best The Binary Symmetric Channel (BSC) is the worst Single Parity Check Code: is Dual ofRepetition Code The Binary Erasure Channel (BEC) is the worst The Binary Symmetric Channel (BSC) is the best

  7. Our Goals • We would like to solve the optimization problem for the Single Parity Check Codes directly (without using duality) • Get some improvements

  8. Channel Soft Bits We callsoft bit, it has support on

  9. erasure

  10. Binary symmetric channel, Gaussian Channel:

  11. Decoder Single Parity Check Code Encoder Single Parity Check Code Channel Channel Channel E.Sharon, A. Ashikhmin, S. Litsyn Results:

  12. Properties of the moments Lemma • is nonnegative and nonincreasing 2. The ratio sequence is nonincreasing Lemma In the Binary Erasure Channel all moments are the same

  13. Problem 2 Among all T-consistent probability distributions on [0,1] such that determine the probability distribution that maximizes (minimizes) the second moment

  14. Solution of Problem 2 Theorem Among all binary-input symmetric-output channel distributions with a fixed mutual information Binary Symmetric Channel maximizes and Binary Erasure Channel minimizes the second moment Proof: We use the theory of Tchebysheff Systems

  15. Lemma Binary Symmetric, Binary Erasure and an arbitrary channel with the same mutual information have the following layout of

  16. Lemma Let and 1) 2) if for and for then

  17. satisfy conditions of the previous lemma This is exactly our case

  18. Problem 1 on extremum of mutual information and Problem 2 on extremum of the second moment are equivalent

  19. Channel Extrema of MMSE It is known that the channel soft bit is the MMSE estimator fo thechannel input Theorem Among all binary-input symmetric-output channels with fixed the Binary Symmetric Channel has the minimum MMSE: and the Binary Erasure Channel has the maximum MMSE:

  20. How good the bounds are

  21. Decoder Single Parity Check Code Encoder Single Parity Check Code Channel Channel Channel Problem 3 1) 2) Among all T-consistent channels find that maximizes (minimizes)

  22. Problem 4 Among all T-consistent probability distributions on [0,1] such that 1) 2) determine the probability distribution that maximizes (minimizes) the fourth moment

  23. Theorem The distribution with mass at , mass at and mass at 0 maximizes The distribution with mass at , mass at and mass at 1 minimizes

  24. Extremum densities Maximizing Minimizing:

  25. Lemma Channel with minimum and maximum and an arbitrary channel with the same mutual information have the followin layout of

  26. Problem 3 on extremum of mutual information and Problem 4 on extremum of the fourth moment are equivalent

  27. Assume that and is the same as in AWGN channel with this

  28. Tchebysheff Systems Definition A set of real continues functions is called Tchebysheff system (T-system) if for any real the linear combination has at most distinct roots at Definition A distribution is a nondecreasing, right-continues function The moment space, defined by ( is the set of valid distributions), is a closed convex cone. For define

  29. Problem For a given find that maximizes (minimizes)

  30. Theorem If and are T-systems, and then the extrema are attained uniquely with distrtibutions and with finitely many mass points Lower principal representation Upper principal representation

  31. Channel Soft Bits We callsoft bit, it has support on Lemma (Sharon, Ashikhmin, Litsyn) If then Random variables with this property are called T-consistent

  32. Find extrema of Under constrains

  33. Theorem Systems and are T-systems on [0,1]. --------------------------------------------------------------------------------- the distribution that maximizes has only one mass point at : has probability mass at and at This is exactly the Binary Symmetric Channel

More Related