240 likes | 460 Views
Overview of Error Resiliency Schemes in H.264/AVC Standard. Sunil Kumar, Liyang Xu, Mrinal K. Mandal, and Sethuraman Panchanathan. Elsevier Journal of Visual Communication and Image Representation, to appear in2005.
E N D
Overview of Error Resiliency Schemes in H.264/AVC Standard Sunil Kumar, Liyang Xu, Mrinal K. Mandal, and Sethuraman Panchanathan Elsevier Journal of Visual Communication and Image Representation, to appear in2005. S. Wenger, “H.264/AVC over IP,”IEEE Trans. Cir. Syst. Video Technol., vol. 13, pp. 645–656, July 2003
Outline • Introduction • Error-Resilience Tools in H.264/AVC • Experimental Results • References
Introduction • H.264/AVC Video Coding Standard • http://vc.cs.nthu.edu.tw/home/paper/codfiles/cycho/200310132110/Overview_of_the_H.264_AVC_Video_Coding_Standard.ppt
Introduction • Error-resilience tools in older video compression standard • Picture segmentation (slices, GOBs); • Placement of Intra MBs, Intra slices, and intra pictures; • Reference picture selection (with and without feedback); • Data partitioning MPEG 1 part 2 MPEG 2 part 2 H.261 H.263 H.263
p2 p4 p6 p8 I0 p9 p1 p3 p5 p7 Error-Resilience Tools in H.264/AVC • Reference picture selection • Feedback-based • Without feedback • e.g. Video redundancy coding • Picture segmentation • Slice • Flexible macroblock ordering (FMO) • 7 modes • Overhead bits highly depends on the picture format, the content, and the QP. (S. Wenger and M. Horowitz) • < 5% penalty at QP = 16; on average 20% at QP = 28. Encoder Decoder feedback channel Channel 1 Channel 2 Checker board mode Interleaving mode
P1, n-3 P1, n-2 P1, n-1 S1, n P1, n+1 S 21, n SI1, n Error-Resilience Tools in H.264/AVC • SP/SI frame http://vc.cs.nthu.edu.tw/home/paper/codfiles/cycho/200402181718/The_SP-_and_SI-Frames_Design_for_H.264_AVC.ppt • Instantaneous refreshing • Gradual refreshing
Intra inter Intra Intra Error-Resilience Tools in H.264/AVC • Intra placement • Used to combat drifting effects. • ConstrainedIntraPrediction flag • An Intra block can not be predicted from an Inter block. • IDR slice and IDR picture • An IDR picture contains only IDR slices. • An IDR slice can only be part of an IDR picture. • An IDR picture invalidates all short-term reference memory buffers. I P IDR P
Error-Resilience Tools in H.264/AVC • Data partitioning • For unequal error protection (UEP). • Three different partition types in H.264. • Type A • Header information (MB types, QP, MVs) • Type B • Intra coded block patterns (CBPs) and Intra coefficients • Type C • Inter CBPs and Inter coefficients • Importance • A > B > C
Error-Resilience Tools in H.264/AVC • Redundant slices (RSs) • For a MB, an encoder can place redundant representation of the same MBs into the same it stream. • e.g. • One slice is coded using different QP. • If the slice of low QP is available, the decoder discards the RS; otherwise, the RS is reconstructed by the decoder Slice AQP1 Decoder Slice AQP2
Error-Resilience Tools in H.264/AVC • Parameter sets • Including picture size, entropy coding method, MV resolution, and so on. • Sequence parameter set • Containing all information related to the picture sequence between two IDR pictures. • Picture parameter set • Containing all information related to all slices in a picture. • e.g. Sending multiple copies of SPSs so to enhance the arrival rate. • e.g. SPSs can be sent out-of-band.
Error-Resilience Tools in H.264/AVC • Intra block refreshing by RD control • Better than periodically inserting intra MBs.
Experimental Results Stephan Wenger
Experimental Results • Error concealment scheme • Intra • Inter
paris foreman Stephan Wenger
Experimental Results • Experiment 1: One picture, one packet, without any error-resilience • Packet loss rates > 3% unprotected H.264 video becomes unusable. Paris Foreman Stephan Wenger
PLR = 0% PSNR = 35 Experimental Results • Experiment 2: One picture, one packet, with intra macroblock refresh • Very good PSNR results • Very high amount of Intra MBs of significant size • A coarser quantizer is needed for CBR – less spatial detail. Stephan Wenger Foreman
Experimental Results • Experiment 3: 2-4 slices in one frame • Unfavorable PSNR results for Foreman. • 2nd best PSNR results for Paris. ? Packet header overhead Paris Foreman Stephan Wenger
Experimental Results • Experiment 4: Slices interleaving • Better than Exp. 3 for Foreman. • Worse than Exp. 3 for Paris. Paris Foreman Stephan Wenger
Experimental Results • Experiment 5: Data partitioning • Partition A is sent twice (3% PLR) or 3 times (>3% PLR). • Degraded PSNR due to CBR. Paris Foreman Stephan Wenger
Experimental Results • Experiment 6: FMO • Randomly inserting Intra MB. (3% of PLR) • Best PSNR results Paris Foreman Stephan Wenger
Experimental Results QCIF, 10fps,100Kbps M. M. Ghandi and M. Ghanbari
Experimental Results QCIF, 10fps,100Kbps M. M. Ghandi and M. Ghanbari
References • S. Kumar, L. Xu, M. K. Mandal, and S. Panchanathan, “Overview of Error Resiliency Schemes in H.264/AVC Standard,”Elsevier Journal of Visual Communication and Image Representation (Special issue on H.264/AVC Video Coding Standard), to appear in 2005. • S. Wenger, “H.264/AVC over IP,”IEEE Trans. Cir. Syst. Video Technol., vol. 13, pp. 645–656, July 2003. • S. Wenger and M. Horowitz, “Scattered slices: A new error resilience tool for H.26L,” in JVT-B027, Feb 2002. • M. M. Ghandi and M. Ghanbari, “Layered H.264 video transmission with hierarchical QAM,”Elsevier J. of Visual Communication and Image Representation (Special issue of H.264/AVC), to appear in 2005.