Supplement, Chapters 6

1. Supplement,Chapters 6 MC Course, 2009

2. Chapter 6H.264/MPEG4 Part 10 Macroblock syntax elements Reference picture management 1/8 motion estimation Motion vector prediction Intra prediction Deblocking filter Transform and quantization Direct and weighted predictions Data partition Raw byte sequence payload

3. H.264 Encoder/Decoder Forward path Reconstruction path unfiltered unfiltered Chapter 6 2009

4. H.264 Profiles FRExt High profiles Chapter 6 2009

5. H.264 Slice Types There is minimal inter-dependency between coded slices which can help to limit the propagation of errors Chapter 6 2009

6. Slice Syntax Skipped (not coded) MBs Chapter 6 2009

7. Macroblock Syntax Elements Chapter 6 2009

8. Reference Picture Management • Every reconstructed frame can be marked as • Unused for reference • Short term picture • Long term picture • None (directly output) • Adaptive memory control (Encoder  Decoder) • Managing the short and long term picture indexes. • Short term picture may be assigned a long term frame index • Short/long term picture may be marked as unused for reference • Instantaneous Decoder Refresh Picture (IDR) • An encoder sends an IDR to clearthe reference picture buffer Chapter 6 2009

9. Reference Picture Management (P-slice) Example: Current frame number=250 Number of reference frames=5 Chapter 6 2009

10. Macroblock to slice group map types Chapter 6 2009

11. Slice Groups: dispersed 4 groups Chapter 6 2009

12. Slice Groups: Foreground and Background Map Chapter 6 2009

13. Slice GroupsBox-out, Raster and Wipe Chapter 6 2009

14. Interpolation of Chroma: eight-sample a= round ([(8-dx)(8-dy)A+dx(8-dy)B+(8-dx)dyC+dxdyD]/64) Chapter 6 2009

15. Motion Vector Prediction leftmost topmost Chapter 6 2009

16. Motion Vector Prediction (2) • For transmitted partitions excluding 16  8 and 8  16 partition sizes: MVp= median (MVA , MVB , MVC ) • For upper 16  8 partitions: MVp = MVB For lower 16  8 partitions: MVp = MVA • For left 8  16 partitions: MVp = MVA For right 8  16 partitions: MVp =MVC • For skipped macroblocks: MVp= (as if the block were encoded in 16  16 Inter mode) Chapter 6 2009

17. Intra Prediction • Signaling intra prediction modes • The choice of intra prediction mode for each 44 block must be signaled to the decoder and this could potentially require a large number of bits. • However, intra modes for neighboring 44 blocks are often correlated. Mode 1 Mode 1 Mode 1 probably Chapter 6 2009

18. Intra Prediction 1:H 3 • Signaling intra prediction modes • Calculate the “most probable” prediction mode of E in both the encoder and the decoder • Most probable mode • Minimum of the prediction modes of A and B • Flag: prev_intra4x4_pred_mode • 1: Most probable mode = Prediction mode of E • If most probable mode  prediction mode of E • prev_intra4x4_pred_mode = 0 • rem_intra4x4_pred_mode is used and sent to the decoder • only 3 bits are needed for 9 modes Most probable mode Prediction mode of E: 0, …, i-1, i, i+1, i+2, …, 8 rem_intra4x4_pred_mode: 0, …, i-1, -, i , i+1, …, 7 Chapter 6 2009

19. 5 6 7 11 8 12 1 2 4 9 10 3 88 chroma 1616 luma Deblocking Filter • Edge filtering order • Boundary strength (bS) Chapter 6 2009

20. Deblocking Filter • Filter decision • (p2, p1, p0, q0, q1, q2) is filtered only if • bS > 0 and • |p0-q0| < , |p1-p0| <  , and |q1-q0| ≦ . •  and  depend on QPs of block p and q. • Filter implementation • bS  {1, 2, 3} • (p1, p0, q0, q1) → (p’0, q’0) • If |p2-p0|<, (p2, p1, p0, q0) → (p’1) (luma only) • If |q2-q0|<, (q2, q1, q0, p0) → (q’1) (luma only) • bS = 4 • If luma block, |p2-p0|< , and |p0-q0|<round(/4) • (p2,p1,p0,q0,q1)→p’0, (p2,p1,p0,q0) →p’1, (p3,p2,p1,p0,q0) →p’2 otherwise (p1, p0, q1) → p’0 • If luma block, |q2-q0|< , and |p0-q0|<round(/4) • (q2,q1,q0,p0,p1)→q’0, (q2,q1,q0,p0) →q’1, (q3,q2,q1,q0,p0) →q’2 otherwise (q1, q0, p1) → q’0 4-tap filter Chapter 6 2009

21. Digital Cosine Transform Y = A X AT X = AT Y A Chapter 6 2009

22. Digital Cosine Transform (2) Chapter 6 2009

23. Digital Cosine Transform (3) scaling factor core To ensure orthogonal, b is modified a = ½, b = , d = ½ (2nd, 4th rows of C, 2nd, 4th columns of CT)2, … d = c/b 0.414 To simplify the implementation: d  0.5 Chapter 6 2009

24. Transform and Quantization • 4x4 residual transform and quantization Quantization Pre-scaling matrix Transform round( / Qstep) Input matrix (Implemented by right shift) MF / 2qbits • Integer arithmetic using +, - and SHIFT • 16-bit arithmetic qbits=15+floor(QP/6) Chapter 6 2009 Table lookup

25. Transform and Quantization Chapter 6 2009

26. Transform and Quantization • 4x4 rescaling (in inverse quantizer) Table lookup  Z ×Qstep × × 64 2floor(QP/6) The output doubles for every (QP/6) Quantized value scaling factor to avoid rounding errors Chapter 6 2009

27. Transform and Quantization • A complete process of Transform, quantization, rescaling, and inverse transform Chapter 6 2009

28. Direct Prediction • Spatial direct mode • Temporal direct mode Chapter 6 2009

29. Weighted Prediction • Three types • P slice with ‘explicit’ weighted prediction • B slice with ‘explicit’ weighted prediction • B slice with ‘implicit weighted prediction • ‘Explicit’ prediction • The weighted factors are determined by the encoder. • ‘Implicit’ prediction • The weighted factors are calculated based on the relative temporal positions of the reference pictures. Chapter 6 2009

30. Data Partitioned Slices • Different partitions can be placed in separate NAL unit. • Partition A • Slice header, macroblock header. • Partition B • Coded residual data for I and SI macroblocks. • Partition C • Coded residual data for inter coded macroblocks. Chapter 6 2009

31. Raw Byte Sequence Payload (RBSP) Chapter 6 2009