Distributed Source Coding Using Syndromes (DISCUS): Design and Construction

Distributed Source Coding Using Syndromes (DISCUS): Design and Construction S.Sandeep Pradhan, Kannan Ramchandran IEEE Transactions on Information Theory, vol. 49, no.3, pp.626-643, Mar 2003

Outline • Introduction • Preliminaries • Encoding with a Fidelity Criterion • Problem Formulation • Design Algorithm • Constructions based on Trellis Codes • Simulation Results • Conclusion

Introduction • Slepian-Wolf theorem: By knowing joint distribution of X and Y, without explicitly knowing Y, encoder of X can perform as well as encoder who knows Y. • Both encoder and decoder have access to side information Y • Only decoder has access to side information Y

Introduction • Wyner-Ziv Problem: If decoder knows Y, then the information-theoretic rate-distortion performance for coding X is identical, no matter encoder knows Y or not.(X &Y are Gaussian.) • Prior work on source quantizer design. • Contributions: • Construction of a framework resting on algebraic channel coding principles • Performance analysis on Gaussian signals. Source: discrete-alphabet  continuous-valued Compression: lossless  lossy

Preliminaries • Example: X, Y: equiprobable 3-bit binary words Hamming distance is no more than 1. Y is available to decoder. Solution? Cosets: {000,111},{100,011},{010,101},{001, 110} Only transmit coset index/syndrome.

+3 -0.5 1 Preliminaries • Quantization: Digitizes an analog signal. Two parameters: a partition and a codebook. Codebook: [-2, 0.4, 2.3, 6]

yi-2 ai yi ai-1 yi-1 Preliminaries • Lloyd Max Quantization: partition: ai are midpoints. codebook: yiare centroids. Optimal scalar quantization.

Preliminaries • Trellis Coded Quantization (TCQ):[24] • Dual of TCM • Example: • Uniformly distributed source in [-A, A] • Implemented by Viterbi algorithm [24] M.W. Marcellin and T. R. Fischer, “Trellis coded quantization of memoryless and Gauss-Markov sources,” IEEE Trans. Commun., vol. 38, pp.82–93, Jan. 1990.

Encoding with a Fidelity Criterion • Problem Formulation • X, Y: correlated, memoryless, i.i.d distributed sequences • Yi = Xi + Ni • Xi, Yi, Ni: continuous-valued • Ni: i.i.d distributed, independent from X • Xi, Ni: zero-mean Gaussian random variables with known variance • Decoder alone has access to Y. • Goal: Form best approximation to X given R bits per sample • Encoding in blocks of length L • Distortion measure: • Min R, s.t. reconstruction fidelity is less than given value D.

Encoding with a Fidelity Criterion System Model: encoder and decoder. Interplay of source coding, channel coding and estimation

Encoding with a Fidelity Criterion • Design Algorithm • Source Coding (M1, M2): • Partition source space: • Defining source codebook (S) • Characterizing active codeword by W (r.v.) • Estimation (M3): Get best estimate of X (minimizing distortion) conditioned on outcome of Y and the element in . • Channel Coding (M4, M5): • Transmit over an error-free channel with rate R (less than Rs) • Doable: I(W;Y) > 0, so H(W|Y) = H(W) – I(W;Y) • Build channel code with rate Rc on channel P(Y|W) • R = Rs – Rc.

Encoding with a Fidelity Criterion • Summary of Design Algorithm: • M1 and M3: • minimize Rs, s.t. reconstruction distortion within given criterion. • M2: maximize I(W;Y). • M4: • maximize Rc, s.t. error probability meets a desired tolerance level. • M5: minimize decoding computational complexity.

Encoding with a Fidelity Criterion • Scalar Quantization and Memoryless Coset Construction (C1): • Lloyd-Max (memoryless) quantizer • Memoryless coset partition (M4) • Example: L=1, (sample by sample) Quantization codebook: {r0, r1, …, r7}, (Rs = 3) Channel coding codebook: {r0, r2, r4, r6}, {r1, r3, r5, r7}. (Rc = 2) R = Rs – Rc = 1 bit/sample.

Encoding with a Fidelity Criterion • Scalar Quantization and Trellis-Based Coset Construction (C2): • Scalar quantizer for {Xi}i=1L • Coset partition (M4) by trellis code. Codebook (size of 8L), Rs = 3 bits/sample, two cosets

Encoding with a Fidelity Criterion • Example: Computing syndrome (Rs = 3, Rc = 2) outcome of quantization be 7, 3, 2, 1, 4. L = 5, Syndrome is given by 10110 for 5 samples.

Encoding with a Fidelity Criterion • Trellis-Based Quantization and Memoryless Coset Construction (C3): • Trellis coded quantizer • Memoryless coset partition • Example: Quantization codebook: Rs = 2 D0={r0, r4}, D1={r1, r5}, D2={r2, r6}, D3={r3, r7}. Memoryless channel code: Rc = 1 1 coded bit with another 1 uncoded bit (from Y) to recover Di.

Encoding with a Fidelity Criterion • Trellis-Based Quantization and Trellis-Based Coset Coset Construction (C4): • Trellis coded quantizer • Trellis coded coset partition Comparison between C3 and C4.

Encoding with a Fidelity Criterion • Distance Property • Given a uniform partition, four cases of coset constructions have same distance property. • Non-uniform quantizer, analyze performance by simulations.

Outline • Introduction • Preliminaries • Encoding with a Fidelity Criterion • Problem Formulation • Design Algorithm • Four Constructions • Simulation Results • Conclusion

Simulation Results Quantization levels decrease distortion. (C1) Correlation -SNR: ratio of X’s variance and N’s variance.

Simulation Results Correlation -SNR: ratio of X’s variance and N’s variance. Quantization levels increase prob. Of error. (C1)

Simulation Results Correlation -SNR: ratio of X’s variance and N’s variance. Error probability comparison of C1 and C2 (3-4dB gain)

Simulation Results Correlation -SNR: ratio of X’s variance and N’s variance. Error probability of C4 codes.

Conclusions • Constructive practical framework based on algebraic trellis codes. • Promising performance.

Distributed Source Coding Using Syndromes (DISCUS): Design and Construction