Distributed Video Coding with Unsupervised Learning of Motion Estimation

1. Distributed Video Coding with Unsupervised Learning of Motion Estimation Young Min Kim Stephanie Kwan Karen Zhu EE 398B Project

2. Outline • Distributed Source Coding • Wyner-Ziv Video Coder • Distributed Stereo Image Coder • Lossless Pixel Domain Distributed Video Coding • Lossy Pixel Domain Distributed Video Coding • Simulation Results • Conclusion

3. Distributed Video Coding • Conventional Video Coding • High complexity encoder • Low complexity decoder • Distributed Video Coding • Low complexity encoder • High complexity decoder

4. Slepian-Wolf Theorem on Lossless Distributed Coding Separate Encoding Joint Decoding (X,Y) Decoder Encoder 1 X (X,Y) Y Encoder 2 Slepian-Wolf Theorem :

5. Wyner-Ziv Lossy Coding Wyner-Ziv Coding Separate Encoder Joint Decoder X X’ Y Side Information available at Encoder Joint Encoder Joint Decoder X X’ Y Y Wyner-Ziv Coding Performance = 0

6. Wyner-Ziv Video Coder Interframe Decoded Intraframe Encoded Decoded WZ Frames WZ Frames Slepian-Wolf Coder Quantization q Reconstruction Turbo Encoder Turbo Decoder S’ S Side Information Ŝ Request bits Interpolation or Extrapolation Conventional Intraframe Encoder Conventional Intraframe Decoder K K’ Decoded Key Frames Key Frames

7. Distributed Compression of Stereo Images with Unsupervised Learning Request bits X LDPC Encoder S LDPC Decoder (M-step) θ Termination Threshold X ψ Disparity Estimator (E-step) Y

8. Lossless Distributed Video Coder with Unsupervised Learning of Motion Request bits LDPC Encoder LDPC Decoder (M-step) θ Termination Threshold X X Decoded Frames ψ Motion Estimator (E-step) Side Information Y Previous Reconstructed Frame

9. LDPC Coding

10. Lossless Distributed Video Coder with Unsupervised Learning of Motion Request bits LDPC Encoder LDPC Decoder (M-step) θ Termination Threshold X X Decoded Frames ψ Motion Estimator (E-step) Side Information Y Previous Reconstructed Frame

11. 2D Motion Estimation

12. Motion Vector Prediction (MVP) • Change initial probability to the motion vector found from previous two frames B

13. Lossy Distributed Video Coder Intraframe Encoded Interframe Decoded Request bits Reconstructed Frames LDPC Encoder LDPC Decoder (M-step) Termination Threshold Q Q-1 S’ S Non-Key Frames θ ψ Side Information Motion Estimator (E-step) Previous Reconstructed Frame Q Conventional Intraframe Encoder Conventional Intraframe Decoder K’ K Decoded Key Frames Key Frames

14. Comparison Schemes for Lossless Coding • Proposed Schemes • 2D motion estimation (2DME) • 2D motion estimation + motion vector prediction (MVP) • Reference Schemes • H(X|Y) – Slepian-Wolf bound • Motion estimation with motion oracle • No motion estimation • Intra-coding

15. Comparison Schemes for Lossy Coding • Proposed Schemes (2DME & MVP) • 7 bits coder • 6 bits coder • 5 bits coder • Reference Schemes • H(X|Y) for given quantization level • Motion estimation with motion oracle • No motion estimation • Intra-coding

16. Simulation Setting • Foreman 65-95, Carphone 180-210 • 8 bitplanes for lossless, 5, 6, 7 bitplanes for lossy • Frame size 72x88, Block size 8x8 • Motion vector between -5 and 5 • Initial probability • 2DME - 0.75 at (0,0) • MVP - 0.75 at previous motion vector

17. Average Rate for Lossless Distributed Video Coder

18. Sequence Rate Trace for Lossless Distributed Video Coder

19. Rate-PSNR Curve for Lossy Distributed Video Coder

20. Conclusion • Our coders achieve rates close to oracle • Better than no estimation • Motion estimation is more effective for more bits and considerable motion • Better than intra-coding for lossless case and most of lossy cases

21. Questions?

22. AppendixSequence Rate Trace (7 bits)

23. AppendixSequence Rate Trace (6 bits)

24. AppendixSequence Rate Trace (5 bits)