MCTF in Current Scalable Video Coding Schemes

1. MCTF in Current Scalable Video Coding Schemes Student: Chia-Yang TsaiAdvisor: Prof. Hsueh-Ming HangInstitute of Electronics, NCTU

2. Outline • Overview • MCTF in Interframe Wavelet • MCTF in JSVM • Comparison • References

3. Outline • Overview • Scalable Video Coding • MCTF in Interframe Wavelet • MCTF in JSVM • Comparison • References

4. Scalable Video Coding • Ability to adjust • Different client requirements • Scalabilities • Rate/SNR • Spatial • Temporal

5. MCTF MCTF = Motion Compensated Temporal Filtering

6. Rate/SNR Scalability • Progressive approximation 300kbps PSNR=32.2 dB 500kbps PSNR=34.6 dB 1000kbps PSNR=38.2 dB GOP Header Motion Info. Image Data

7. Wavelet decomposition provides spatial scalability Bit-plane Coder Spatial Scalability

8. Temporal Scalability 30Hz Video Sequence 15Hz Video Sequence H H H H H H H H1 7.5Hz Video Sequence H2 H2 H2 H2 3.25Hz Video Sequence H3 H3 H4 L

9. Scalable Video Coding • History HHI (AVC-based) JSVM UNSW (wavelet) RPI (wavelet) MSRA (wavelet) 2005 2004.3 2004.7

10. Approaches An AVC/H.264-based approach (also DCT-based)

11. Approaches A wavelet-based approach with t+2D structure.

12. Approaches A wavelet-based approach with 2D+t structure

13. Lifting Scheme • 5/3 lifting scheme

14. Outline • Overview • MCTF in Interframe Wavelet • Barbell lifting • In-band MCTF • Base-layer structure • MCTF in JSVM • Comparison • References

15. Barbell Lifting Scheme • Purpose: • Improve the accuracy of motion field. • Methods: • Take (5,3) wavelet kernel. • Use “barbell function” to generate prediction /update values.

16. Barbell Lifting Scheme

17. Barbell Lifting Scheme Prediction Stage Update Stage

18. In-Band MCTF • Purpose: • Improve coding performance with spatial scalability • Methods: • Leaky motion compensation • Mode-based temporal filtering

19. In-Band MCTF • The forming of different quality reference of LL • Low quality reference as IP_DIR • High quality reference as IP_LBS

20. In-Band MCTF • Leaky motion compensation • leaky factor • Attenuate the prediction based on the unknown information at the decoder • make a good trade-off between drifting errors and coding efficiency

21. In-Band MCTF • Mode-based temporal filtering • Mode I: Low quality reference • Mode 2: High quality reference • Mode is selected by RD cost

22. Base-Layer Structure • Purpose: • Coding efficiency improvement in low rates • AVC compatible • Methods: • Insert AVC encoding module into MCTF

23. Base-Layer Structure Encoder Decoder

24. Outline • Overview • MCTF in Interframe Wavelet • MCTF in JSVM • Base layer structure • Inter-layer prediction • Adaptive prediction/update steps • Comparison • References

25. Base Layer Structure • Purpose • Coding efficiency improvement in low rates • Compatibility to AVC • Methods • Unrestricted MCTF (UMCTF) • Hierarchical B pictures

26. Base Layer Structure • UMCTF • Update step is omitted. • Hierarchical B pictures • Fully compatible to AVC Main profile • Non-dyadic decomposition is available

27. Non-Dyadic Decomposition

28. Inter-Layer Prediction • Purpose • Reduce redundancy between layers • Methods • Inter-layer texture prediction • Inter-layer motion prediction

29. Inter-Layer Prediction

30. Adaptive Prediction/Update Steps • Purpose: • Delay (Memory)control • Method: • Sub-partitioning of GOP

31. Adaptive Prediction/Update Steps

32. Outline • Overview • MCTF in Interframe Wavelet video • MCTF in JSVM • Comparison • Cons and pros • Experimental results • References

33. Wavelet Based SVC • Key features • 3D wavelet decomposition • Open-loop prediction structure • Spatial-temporal resolution scalability • SNR scalability

34. Wavelet Based SVC • Advantages • Nature for multi-resolution scalability • Open-loop prediction structure • Provides elegant SNR scalability without impairing full exploitation of spatial-temporal correlation • Simplifies the R-D model of the bitstreams. • Facilitates the bitstream truncation • each subband is independent with other subbands

35. Wavelet Based SVC • Disadvantages • Decomposition modes (coding modes) selection • Texture & side information trade off • Intra-prediction • Badly-matched blocks • Downsampling filter problems

36. AVC Based SVC • Key features • MCTF/Hierarchical B structure for temporal scalability • Hierarchical B structure with close-loop structure for base layer • Multiple spatial layers for spatial scalability • Multiple FGS layers at each spatial resolution for SNR scalability • DCT coding of all the frames

37. AVC Based SVC • Advantages • All the RDO and intra-prediction can be used. • It guarantees the quality of the first testing point. • MPEG filter for low resolution video • the target low resolution video is visually good.

38. AVC Based SVC • Disadvantages • Redundancy between spatial layers

39. Experiments

40. Experiments

41. References [1] “Draft of joint scalable video model (JSVM)3.0 reference encoding algorithm description”, ISO/IEC JTC1/SC29/WG11, N7311, Poznan, July 2005. [2] D. Zhang, J. Xu, H. Xiong, and F. Wu, “Improvement for in-band video coding with spatial scalability”, ISO/IEC JTC1/SC29/WG11, M11681, HongKong, Jan. 2005. [3] V. Bottreau, G. Pau, and J. Xu, “Vidwav evaluation software manual”, ISO/IEC JTC1/SC29/WG11, M12176, Poznan, July. 2005. [4] X. Ji, J. Xu, D. Zhao, and F. Wu, “Responses of CE1d: base- layer”, ISO/IEC JTC1/SC29/WG11, M11127, Redmond, July 2004. [5] R. Xiong, J. Xu, and F. Wu, “Coding performance comparison between MSRA wavelet video coding and JSVM”, ISO/IEC JTC1/SC29/WG11, M11975, Busan, April 2005. [6] R. Xiong, J. Xu, and F. Wu, “Response to VidWav EE1”, ISO/IEC JTC1/SC29/WG11, M12286, Poznan, July 2005. [7] J. Reichel, K. Hanke and B. Popescu, “Scalable Video Model V1.0”, ISO/IEC JTC1/SC29/WG11, N6372, Munich, March 2004.