Overview of the H.264/AVC Video Coding Standard

1. Overview of the H.264/AVC Video Coding Standard T. Wiegand, G.J. Sullivan, G. Bjøntegaard and A. Luthra, IEEE Transaction on Circuits and Systems for Video Technology, Vol. 13, no. 7, Jul. 2003. Presented by Peter

2. H.264/AVC • Latest Video coding standard • Basic design architecture similar to MPEG-x or H.26x • Better compression efficiency • Up to 50% in bit rate savings • Subjective quality is better • Advance functional element

3. History of H.264/AVC • Initiate by the Video Coding Experts Group (VCEG) in early 1998 • Previous name H.26L • Target to double the coding efficiency • First draft was adopted in Oct. of 1999 • In Dec. of 2001, VCEF and the Moving Pictures Experts Group (MPEG) formed a Joint Video Team (JVT) • Approved by the ITU-T as H.264 and ISO/IEC as International Standard 14496-10 (MPEG-4 part 10) Advanced Video Codec (AVC) in Mar. 2003

4. Timeline of Video Development

5. Design Features Highlights • Features for enhancement of prediction • Directional spatial prediction for intra coding • Variable block-size motion compensation with small block size • Quarter-sample-accurate motion compensation • Motion vectors over picture boundaries • Multiple reference picture motion compensation • Decoupling of referencing order form display order • Decoupling of picture representation methods from picture referencing capability • Weighted prediction • Improved “skipped” and “direct” motion inference • In-the-loop deblocking filtering

6. Design Features Highlights • Features for improved coding efficiency • Small block-size transform • Exact-match inverse transform • Short word-length transform • Hierarchical block transform • Arithmetic entropy coding • Context-adaptive entropy coding

7. Design Features Highlights • Features for robustness to data errors/losses • Parameter set structure • NAL unit syntax structure • Flexible slice size • Flexible macroblock ordering (FMO) • Arbitrary slice ordering (ASO) • Redundant pictures • Data Partitioning • SP/SI synchronization/switching pictures

8. Directional spatial prediction for intra coding • Intra prediction is to predict the texture in current block using the pixel samples from neighboring blocks • Intra prediction for 44 and 16  16 blocks are supported in H.264 Figs. from [2]

9. Directional spatial prediction for intra coding - 4  4 example Mode 7 is selected Figs. from [2]

10. Directional spatial prediction for intra coding – 16  16 example Mode 3 is selected Figs. from [2]

11. Variable block-size motion compensation with small block size • Partitioned in 2 stages • In the 1st stage, determine first 4 modes • 1616, 168, 816, 88 • If mode 4 (88) is chosen, further partition into smaller blocks for every 88 block • 84, 48, 44 • At most 16 motion vectors may be transmitted for a 1616 macroblock • Large computational complexity to determine the modes Fig. from [3]

12. Variable block-size motion compensation with small block size

13. Multiple reference picture motion compensation – P Slices • More than one prior coded picture can be used as reference for MC prediction • Reference index parameter is transmitted for each MC 1616, 168, 816 or 88 • For smaller blocks within the 88 use 1 reference index • P macroblock can also be coded in P-Skip type Fig. from [1]

14. Multiple reference picture motion compensation – B Slices • Utilize two distinct lists of reference pictures • Four different types of inter-picture predict • List 0, list 1, bi-predictive, and direct prediction • Bi-predictive • weighted average of MC list 0 and list 1 • Direct prediction • Inferred from previously transmitted syntax • Either list 0 or list 1 prediction or bi-predictive • Similar macroblock partitioning as P slices is utilized • B_Skip mode is supported

15. Small block-size transform • Transformation is applied on 44 blocks • Close to 44 DCT transform • Inverse-transform mismatches are avoided • The transform matrix is given as

16. Short word-length transform • Post-scaling matrix in forward transform • Pre-scaling matrix in inverse transform • Only integer operations and shifting are needed in transformation and quantization

17. Hierarchical block transform • For macroblock is coded in 1616 Intra mode and chrominance blocks • DC coefficients are further grouped and transformed • Hadamard transform is used for chrominance block • Intended for coding of smooth areas Figs. from [4]

18. Some results – Foreman QCIF @ 10 Hz Fig. from [1]

19. Some results – Foreman CIF @ 30 Hz Fig. from [1]

20. Profiles • 3 profiles - Baseline, Main and Extended Profile • 15 levels • Picture size: up to 250M pixels/s • Bit Rate: up to 240M bps

21. Potential Applications • Baseline (low latency) • H.320 conversational video services • 3GPP conversational H.324/M services • H.323 with IP/RTP • 3GPP using IP/RTP and SIP • 3GPP streaming using IP/RTP and RTSP • Main (moderate latency) • Modified H.222.0/MPEG-2 • Broadcast via satellite, cable, terrestrial or DSL • DVD and VOD • Extended • Streaming over wired Internet • Any (no requirement on latency) • 3GPP MMS • Video mail

22. References • T. Wiegand, G.J. Sullivan, G. Bjøntegaard and A. Luthra, “Overview of the H.264/AVC Video Coding Standard,” IEEE Transaction on Circuits and Systems for Video Technology, Vol. 13, no. 7, Jul. 2003. • I.E.G. Richardson, “H.264/MPEG4 Part 10: Intra Prediction,” available at http://www.vcodex.com • I.E.G. Richardson, “H.264/MPEG4 Part 10: Inter Prediction,” available at http://www.vcodex.com • I.E.G. Richardson, “H.264/MPEG4 Part 10: Transform and Quantization,” available at http://www.vcodex.com