Advance in Scalable Video Coding

1. Advance in Scalable Video Coding Proc. IEEE 2005, Invited paper Jens-Rainer Ohm, Member, IEEE

2. Outline • Introduction • Scalability in Existing Standard • Principles of Scalable Predictive Coding • Drift control • Interframe Wavelet Coding • Conclusion 謝俊瑋 NTU CSIE, CMLab

3. Introduction • Scalable video coding is attractive due to the capability of reconstructing lower resolution or lower quality signals from partial bit streams. • A simple and flexible solution for transmission over heterogeneous network. • Allow simple adaptation for a variety of storage devices and terminals. 謝俊瑋 NTU CSIE, CMLab

4. Scalability in Existing Standard • MPEG-2 • First general-purpose video compression standard which also include tools providing scalability. • Layered coding, support spatial, temporal, SNR scalability. • Number of layers is restricted to maximum of 3. 謝俊瑋 NTU CSIE, CMLab

5. Scalability in Existing Standard(2) • MPEG-4 • More flexible scalability tools, including spatial and temporal scalability within a more generic framework. • SNR scalability with fine granularity and scalability at level of video objects. • AVC can in principle be run in different temporal scalability modes. 謝俊瑋 NTU CSIE, CMLab

6. Principles of Scalable Predictive Coding • Prediction should not use any decoded information from higher layers, otherwise drift effect would occur. • However the rate-distortion performance toward higher rates will be worse than single-layer coder. 謝俊瑋 NTU CSIE, CMLab

7. 謝俊瑋 NTU CSIE, CMLab

8. Drift control • It is possible to track the drift within the local loop of the encoder that would occur in a decoder only receiving the base-layer information. • Drift compensation • Drift clipping • Drift leaking 謝俊瑋 NTU CSIE, CMLab

9. Drift compensation • It assume that the decoder side is not aware of the drift compensation made. • Usual MC decoder loop could be used without any modifications. • The encoder have to find the balance between the penalties for the base and enhancement decoding. 謝俊瑋 NTU CSIE, CMLab

10. Drift clipping • The drift is dynamically limited if a maximum value Dmax is reached. • A good choice for Dmax is approximately by the base-layer quantizer step size. 謝俊瑋 NTU CSIE, CMLab

11. Drift leaking • The accumulation of drift is limited by multiplying D’ = a * D (a < 1) • The best selection of drift coefficient is dependent on the operational target and sequence characteristic. 謝俊瑋 NTU CSIE, CMLab

12. 謝俊瑋 NTU CSIE, CMLab

13. Interframe Wavelet Coding • To overcome the limitation which are caused by drift problem, it would be desirable to discard the temporal recursion. • Motion-Compensated Temporal Filtering 謝俊瑋 NTU CSIE, CMLab

14. Spatiotemporal wavelet decomposition 謝俊瑋 NTU CSIE, CMLab

15. Spatiotemporal wavelet decomposition (2) 謝俊瑋 NTU CSIE, CMLab

16. Lift structure 謝俊瑋 NTU CSIE, CMLab

17. Operation of Motion Compensation 謝俊瑋 NTU CSIE, CMLab

18. 謝俊瑋 NTU CSIE, CMLab

19. Conclusion • The fully open loop property of MCTF provides high flexibility in bit stream scalability. • Combination of MCTF with the new drift-controlled prediction strategies is also a promising path. • Decoder could integrate additional signal synthesis element whenever the received information is incomplete. 謝俊瑋 NTU CSIE, CMLab

20. Conclusion (2) • When low delay is required, the update step must be omitted, or the number of temporal wavelet decomposition levels must be low. • Seamless transition between intraframe and interframe coding methods. 謝俊瑋 NTU CSIE, CMLab