1 / 32

Shannon’s Ideal Channel

The Cutoff Rate and Other Limits: Passing the Impassable Richard E. Blahut University of Illinois UIUC. Shannon’s Ideal Channel. Stationary Discrete Memoryless. Example: Binary Memoryless Channel. A Large Code. … 0 0 0 1 1 1. … 0 0 0 … 0 0 1 … 0 1 0 … 0 1 1 … 1 1 1.

adolfo
Download Presentation

Shannon’s Ideal Channel

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. TheCutoff Rate and Other Limits:Passing the ImpassableRichard E. BlahutUniversity of IllinoisUIUC

  2. Shannon’s Ideal Channel • Stationary • Discrete • Memoryless Example: Binary Memoryless Channel

  3. A Large Code … 0 0 0 1 1 1 … 0 0 0 … 0 0 1 … 0 1 0 … 0 1 1 … 1 1 1 0 1 0 0 1 1 0 1 0 1 0 … 1 1 1 0 0 1 0 1 1 0 1 … 0 1 1 1 0 0 1 0 1 1 1 … 1 0 1 …. . . . 1 1 1 1 1 0 0 0 1 0 0 … 0 0 … 0 0 … 0 0 … 0 0 … . . . 1 1 …

  4. + + A convolutional encoder +

  5. There are binary codes Information theory asserts existence of good codes Coding theory wants practical codes and decoders

  6. Brief History of Codes • Algebraic Block Codes 1948 • Reed-Solomon codes (1960) • Convolutional Codes 1954 • Sequential decoding (1951) • Viterbi algorithm (1967) • Euclidean Trellis Codes 1982 • Turbo Codes 1993 • Gallager (LDPC) codes (1960)

  7. Decoders Maximum Likelihood Maximum Block Posterior Maximum Symbol Posterior Typical Sequence Iterative Posterior Minimum Distance Bounded Distance

  8. My View • Channel Capacity • Cutoff Rate • Critical Rate Distance -based codes Likelihood -based codes Posterior -based codes Polar codes

  9. Channel Error Exponent Fact #1 Codes exist such that Fact #2 Every code satisfies For any fixed there is a sequence of codes for which exponentially in blocklength. This sequence does not approach

  10. E(R)

  11. A sequence of codes drawn from a set of ensembles

  12. Channel Capacity Channel Critical Rate Channel Cutoff Rate

  13. Binary Hypotheses Testing Type 1 Error Type 2 Error

  14. Binary Hypotheses Testing Change Notation

  15. and Bounds on Upper Bounds on Sphere Packing Bound Lower Bounds on Minimum Distance Bound Random Coding Bound Expurgated Bound

  16. Bhattacharyya Distance

  17. A Code Sequence Approaching Capacity is quadratic near Let be a sequence with Then with so with if

  18. Capacity: C • Shannon (1948) • Cutoff Rate: • Jacobs & Berlekamp (1968) • Massey (1981) • Arikan (1985/1988) • Error Exponent: • Gallager (1965) • Forney (1968) • Blahut (1972)

  19. Gallager (1965) Forney (1968) Blahut (1972) where is the Kullback divergence

  20. Forney’s List Decoding Likelihood Function Likelihood Ratio

  21. Sequential Decoding • Exponential waiting time • Work exponential in time • Pareto Distribution with • Work unbounded if Sequential decoding fails if Is maximum likelihood decoding sequential decoding?

  22. Pareto Distribution Two Pareto parameters and

  23. The Origin of a Pareto Distribution Start with an exponential distribution If is exponential, then is a Pareto distribution

  24. The Origins of Graph-Based Codes Brillouin deBrogle Shannon Battail (1987) Hagenauer (1989) Berrou et al (1993)

  25. Coding Beyond the Cutoff Rate Parallel – Pinsker Hybrid – Jelinek Turbo – Berrou/Glavieux LDPC – Gallager/Tanner/Wiberg Polar - Arikan

  26. The Massey Distraction (1981) QEC BEC QEC 2 BEC

  27. Performance MeasuresBit Error Ratevs.Message Error Rate

  28. The Arikan Retraction

  29. The Arkan Redistraction* *Rhetorical

More Related