510 likes | 719 Views
Distributed Video Coding. VLBV , Sardinia, September 16, 2005 Bernd Girod Information Systems Laboratory Stanford University. Outline. Foundations of distributed coding Slepian-Wolf Theorem and practical Slepian-Wolf coding Wyner-Ziv results and practical Wyner-Ziv coding
E N D
Distributed Video Coding VLBV, Sardinia, September 16, 2005 Bernd Girod Information Systems Laboratory Stanford University
Outline • Foundations of distributed coding • Slepian-Wolf Theorem and practical Slepian-Wolf coding • Wyner-Ziv results and practical Wyner-Ziv coding • Low-complexity video encoding • Pixel-domain and transform-domain coding • Hash-based receiver motion estimation • Wyner-Ziv residual coding • Error-resilient video transmission • Systematic lossy joint source-channel coding • Improving the error-resiliency of MPEG (or anything else) by Wyner-Ziv coding
Outline • Foundations of distributed coding • Slepian-Wolf Theorem and practical Slepian-Wolf coding • Wyner-Ziv results and practical Wyner-Ziv coding • Low-complexity video encoding • Pixel-domain and transform-domain coding • Hash-based receiver motion estimation • Error-resilient video transmission • Systematic lossy joint source-channel coding • Improving the error-resiliency of MPEG by Wyner-Ziv coding
Compression of Dependent Sources Source X X Joint Encoder Joint Decoder Y Source Y
Distributed Compression of Dependent Sources Encoder X Source X X Joint Decoder Y Source Y Encoder Y
Slepian Wolf Theorem Achievable rate region for i.i.d sequences Independent decoding
Slepian Wolf Theorem Achievable rate region for i.i.d sequences Independent decoding: No errors Joint decoding: Vanishing error probabilityfor long sequences [Slepian, Wolf, 1973]
Lossless Compression with Receiver Side Information Source Decoder Encoder
Idea Interpret Y as a “noisy” version of X with “channel errors” D Encoder generates “parity bits” P to protect against errors D Decoder concatenates Y and P and performs error-correcting decoding Distributed Compression and Channel Coding Source X|Y Decoder Encoder P
Towards Practical Slepian-Wolf Coding • Convolution coding for data compression [Blizard, 1969, unpublished] • Convolutional source coding [Hellman, 1975] • Coset codes [Pradhan and Ramchandran, 1999] • Trellis codes [Wang and Orchard, 2001] • Turbo codes [Garcia-Frias and Zhao, 2001] [Bajcsy and Mitran, 2001] [Aaron and Girod, 2002] • LDPC codes [Liveris, Xiong, and Georghiades, 2002] • . . . • . . .
Systematic Convolutional Code Systematic Convolutional Code Systematicbits X . . . . . . . Y SISO Decoder SISO Decoder Interleaver Decision “Correlation channel” Slepian-Wolf Coding Using Turbo Codes Parity bits Parity bits [Aaron and Girod, 2002]
Lossy Compression with Side Information Source Decoder Encoder [Wyner, Ziv, 1976] For mse distortion and Gaussian statistics, rate-distortion functions of the two systems are the same. Source Decoder Encoder
Practical Wyner-Ziv Encoder and Decoder Wyner-Ziv Decoder Wyner-Ziv Encoder Slepian- Wolf Decoder Minimum Distortion Reconstruction Slepian-Wolf Encoder Quantizer
Non-Connected Quantization Regions • Example: Non-connected intervals for scalar quantization • Decoder: Minimum mean-squared error reconstruction with side information x x
Outline • Foundations of distributed coding • Slepian-Wolf Theorem and practical Slepian-Wolf coding • Wyner-Ziv results and practical Wyner-Ziv coding • Low-complexity video encoding • Pixel-domain and transform-domain coding • Hash-based receiver motion estimation • Error-resilient video transmission • Systematic lossy joint source-channel coding • Improving the error-resiliency of MPEG by Wyner-Ziv coding
Interframe Video Coding PredictiveInterframe Encoder PredictiveInterframe Decoder X X’ Side Information
Video Coding with Low Complexity Wyner-ZivIntraframe Encoder Wyner-ZivInterframe Decoder X X’ Side Information [Witsenhausen, Wyner, 1978] [Puri, Ramchandran, Allerton 2002] [Aaron, Zhang, Girod, Asilomar 2002]
Video frame Reconstruction X’ X Y Pixel Domain Wyner-Ziv Coder Interframe Decoder Intraframe Encoder Slepian-Wolf Codec Turbo Decoder Turbo Encoder Scalar Quantizer Buffer Request bits previous Interpolation Key frames next [Aaron, Zhang, Girod, Asilomar 2002] [Aaron, Rane, Zhang, Girod, DCC 2003]
Pixel Domain Wyner-Ziv Coder After Wyner-Ziv Decoding Decoder side informationgenerated by motion-compensated interpolationPSNR 30.3 dB 16-level quantization – 1.375 bpp11 pixels in errorPSNR 36.7 dB
Pixel Domain Wyner-Ziv Coder After Wyner-Ziv Decoding Decoder side informationgenerated by motion-compensated interpolationPSNR 24.8 dB 16-level quantization – 2.0 bpp0 pixels in errorPSNR 36.5 dB
Stanford Camera Array Courtesy Marc Levoy, Stanford Computer Graphics Lab
Distributed Compression Wyner-Ziv Cameras Conventional Cameras … … Distributed Encoding WZ-ENC WZ-ENC WZ-DEC WZ-DEC Geometry Reconstruction Joint Decoding Rendering [Zhu, Aaron, Girod, 2003]
Light Field Compression Wyner-Ziv, Pixel-Domain JPEG-2000 Rate: 0.11 bpp PSNR 39.9 dB Rate: 0.11 bpp PSNR 37.4 dB
Low freq coeffs bit-plane 2 bit-plane 1 High freq coeffs DCT-Domain Wyner-Ziv Video Encoder Turbo Encoder Extract bit-planes level Quantizer Wyner-Ziv parity bits DCT Input video frame … bit-plane Mk For each low frequency coefficient band k Previous-frame quantized high freq coefficients Comparison High frequency bits Entropy Coder Quantizer
Wyner-Ziv Video Decoder with Motion Compensation Decoded frame For each low frequency band Turbo Decoder Reconstruction Wyner-Ziv parity bits IDCT Refinedside information Side information DCT DCT Motion-compensated Extrapolation ExtrapolationRefinement Previous frame High frequency bits Entropy Decoder and Inverse Quantizer Reconstructed high frequency coefficients
3 dB 6 dB Rate-Distortion Performance - Salesman • Every 8th frame is a key frame • Salesman QCIF sequence at 10fps 100 frames
3 dB 8 dB Rate-Distortion Performance – Hall Monitor • Every 8th frame is a key frame • Hall Monitor QCIF sequence at 10fps 100 frames
2 dB 1.5 dB Rate-Distortion Performance – Foreman • Every 8th frame is a key frame • Foreman QCIF sequence at 10fps 100 frames
Salesman at 10 fps DCT-based Intracoding 149 kbps PSNRY=30.0 dB Wyner-Ziv DCT codec 152 kbps PSNRY=35.6 dB GOP=8
Hall Monitor at 10 fps DCT-based Intracoding 156 kbps PSNRY=30.2 dB Wyner-Ziv DCT codec 155 kbps PSNRY=37.1 dB GOP=8
Foreman at 10 fps DCT-based Intracoding 290 kbps PSNRY=34.4 dB Wyner-Ziv DCT codec 293 kbps PSNRY=35.5 dB GOP=8
X’n-1 X’n-1 Xn - Side Information Frame difference Xn Wyner-Ziv Residual Coding Wyner-ZivDecoder Wyner-Ziv Encoder X’n Side Information
Rate-Distortion Performance – Foreman • Every 8th frame is a key frame • Foreman QCIF sequence at 30fps 100 frames
Rate-Distortion Performance – Foreman • Every 8th frame is a key frame • Foreman QCIF sequence at 30fps 100 frames
Outline • Foundations of distributed coding • Slepian-Wolf Theorem and practical Slepian-Wolf coding • Wyner-Ziv results and practical Wyner-Ziv coding • Low-complexity video encoding • Pixel-domain and transform-domain coding • Hash-based receiver motion estimation • Error-resilient video transmission • Systematic lossy joint source-channel coding • Improving the error-resiliency of MPEG by Wyner-Ziv coding
Wyner-Ziv Decoder A Wyner-Ziv Encoder A S* Wyner-Ziv Decoder B Wyner-Ziv Encoder B S** Systematic Lossy Error Protection (SLEP)of Compressed Video Analog channel (uncoded) Any OldVideo Encoder Video Decoder with Error Concealment Graceful degradation without a layered signal representation S S’ Error-Prone channel [Aaron, Rane, Girod, ICIP 2003]
main S* S MPEG Encoder ED + Q-1 T-1 MC Reconstructed Frame at Encoder Channel S’ q-1 ED T-1 + MC Parityonly R-S Decoder R-S Encoder MPEG Encoder coarse MPEG Encoder Side information Slepian-Wolf Encoder coarse Wyner-Ziv Encoder MPEG with Systematic Lossy Error Protection [Rane, Aaron, Girod, VCIP 2004]
Transmit parity slices only 1 byte in slice filler byte parity byte Reed-Solomon Coding Across Slices RS code across slices
Results: CIF Foreman FECSLEP Main Stream @ 1.092 Mbps FEC (n,k) = (40,36) FEC bitrate = 120 Kbps Total = 1.2 Mbps WZ Stream @ 270 Kbps SLEP (n,k) = (52,36) WZ bitrate = 120 Kbps Total = 1.2 Mbps SLEP FEC
S* ED + Q-1 T-1 MC Reconstructed Frame at Encoder S’ q-1 ED T-1 + MC R-S Decoder MPEG Encoder coarse MPEG Encoder Side information Slepian-Wolf Encoder coarse Wyner-Ziv Encoder MPEG with Systematic Lossy Error Protection main S MPEG Encoder Channel Parityonly R-S Encoder [Rane, Aaron, Girod, VCIP 2004]
LEGACY BROADCASTING SYSTEM MPEG2 Decoder Input Video MPEG2 Encoder Entropy Decoding Conventionally encoded stream Q-1 + T-1 Quantization parameter (Q) MC Q1 Error-prone Channel Entropy Coding (motion vecs, mode decisions) Fallback to coarser version Quantized transformed Prediction error Side Information (motion vectors, mode decisions) Side Info Entropy Coding RS Encoder Parity only RS Decoder Entropy Decoding Q1 Coarse Quantizer Decoded motion vecs WYNER-ZIV DECODER WYNER-ZIV ENCODER SLEP MPEG Codec with Simple Decoder
Performance at Symbol Error Rate 10-4 MPEG-2 video: 2 Mbps + FEC 222 Kbps (PSNR 29.78 dB) MPEG-2 video: 2 Mbps + Wyner-Ziv 222 Kbps (PSNR 35.45 dB)
Distributed Coding of Video:Why Should We Care? • Chance to reinvent compression from scratch • Entropy coding • Quantization • Signal transforms • Adaptive coding • Rate control • . . . • Enables new compression applications • Very low complexity encoders • Compression for networks of cameras • Error-resilient transmission of signal waveforms • Digitally enhanced analog transmission • Unequal error protection without layered coding • . . .
The End Further interest: B. Girod, A. Aaron, S. Rane, D. Rebollo-Monedero, "Distributed Video Coding," Proceedings of the IEEE, Special Issue on Video Coding and Delivery. January 2005. http://www.stanford.edu/~bgirod/pdfs/DistributedVideoCoding-IEEEProc.pdf