MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding

1. MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding Authors from: University of Georgia Speaker: Chang-Kuan Lin

2. Reference • S. Chattopadhyay, S. M. Bhandarkar, K. Li, “FGS-MR: MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding,” ACM NOSSDAV 2006. • W. Li, “Overview of Fine Granularity Scalability in MPEG-4 Video Standard,” IEEE Trans. on Circuits and Systems for Video Technology, Vol. 11, No. 3, pp. 301-317, Mar. 2001. • H. Radha, M. van der Schaar, and Y. Chen, “The MPEG-4 fine-grained scalable video coding method for multimedia streaming over IP,” IEEE Trans. on Multimedia, vol.3, pp. 53–68, Mar. 2001.

3. Outline • Introduction • MPEG-4 Fine Grained Scalability • Motivation • FGS-AQ vs. FGS-MR • Experimental Results • Conclusion

4. Introduction • MPEG4 Fine Grained Scalability (FGS) profile for streaming video • Base Layer Bit Stream • must exist at the decoder • has coarsely quantized DCT coefficients • provides the minimum video quality • Enhancement Layer Bit Stream • can be absent at the decoder • contains encoded DCT coefficient differences • provides higher quality • can be truncated to fit the target bit rate

5. FGS Encoding Block Diagram

6. Motivation • Base Layer video quality is usually not satisfactory • in order to provide a wide range of bit rate adaptation • MPEG4 FGS Adaptive Quantization (FGS-AQ) for Base Layer video does not provide good rate-distortion (R-D) performance • parameter overhead at the decoder • Proposed FGS-MR • no parameter overhead to transmit • transparent the codec • better rate-distortion performance

7. Outline • Introduction • MPEG-4 Fine Grained Scalability • Motivation • FGS-AQ vs. FGS-MR • FGS-AQ • FGS-MR • MR-Mask Creation • MR-Frame • Experimental Results • Conclusion

8. FGS Adaptive Quantization (AQ) • Goals • To improve visual quality • To better utilize the available bandwidth • Method • Define different quantization step sizes for different transform coefficients • within a macro-block(low freq. DCT coeff. => small step size) • for different macro-blocks(different quantization factors) • Disadvantages • R-D performance degrades due to FGS-AQ parameter overhead

9. Proposed Multi-Resolution FGS (FGS-MR) • Goal • To improve the visual quality • To better utilize the available bandwidth • No transmission overhead and hence maintaining the R-D performance • Method • Apply a low-pass filter on “visually unimportant” portion of the original video frame before encoding.

10. Two Equivalent Operations • Apply a low-pass filter on the spatial domain of an image • Truncate DCT coefficients in the corresponding transform domain of an image

11. FGS-MR Process (Step 1) • MR-Mask creation • Use Canny edge detector to detect edges • Weight Mask • an weight parameter wi, j for each pixel p(i, j) of an image, 0 ≦ wi, j ≦1 • wi, j = 1, if p(i, j) is on the edge • 0 < wi, j ≦1, if p(i, j) is near edge • wi, j = 0, if p(i, j) is in non-edge region

12. Original (5.12Mbps)

13. MR-Mask

14. FGS-MR Process (Step 2) • MR-Frame Creation • VI = (I-W) VL +W VH • VF = Iteration( VI, G(σI)) • Note • VI contains abrupt changes in resolution • VF is a smooth version of VI Parameters • Vo: original video frame • VL: low resolution frame from the convolution of Vo and G(σL) • VH: high resolution frame from the convolution of Vo and G(σH) • VI : intermediate video frame • VF : final multi-resolution frame • I: matrix with all entries as 1 • W: MR-mask weight matrix • G(σ): Gaussian filter with standard deviation of σas LPF • σL >σH

15. Original (5.12Mbps)

16. FGS-AQ (0.17Mbps, PSNR = 22.77dB)

17. FGS-MR (0.17Mbps, PSNR = 26.5dB)

18. Determine Parameters • σL, σH, and σI • to control the bit rate • W (weight matrix) • to control the quality of the encoded video frame • Figure of merit function: δ=Q/C • Q = 2^( PSNR(σL, σH, σI)/10 ) • or PSNR = 10log(Q) • C: compression ratio • The authors empirically determine the parameters • σL = 15, σL = 3, and varying σI

19. Outline • Introduction • MPEG-4 Fine Grained Scalability • Motivation • FGS-AQ vs. FGS-MR • FGS-AQ • FGS-MR • Experimental Results • Rate Distortion • Resource Consumption • Conclusion

20. Experiments • Video 1 • 320x240, fps = 30 • A single person walking in a well lighted room • Video 2 • 176x144, fps = 30 • A panning view across a poorly lighted room. • No moving object

21. Rate Distortion Performance • Vary σI from 3 to 25 to adjust the target bit rate

22. Power Consumption • Energy used and hence power consumed by wireless network interface card (WNIC): T: time duration S: data size b: the bit rate of streaming video B: available BW ER: energy used by WNIC during data reception Es: energy used by WNIC when sleeping

23. Power Consumption Comparison

24. Conclusion • The rate distortion performance of FGS-MR is better than FGS-AQ. • FGS-MR can be seamlessly integrated into existing MPEG4 codec. • My comment • Processing time issue of FGS-MR • Empirical determined filter parameters