MPEG Video (Part 2)

1. MPEG Video (Part 2)

2. MPEG Encoding Revisit • Overall MPEG bitstream organization. • I-Frames • Examples of many encoding techniques: • Subsampling (chrominance planes) • Transform Coding (DCT, zig-zag) • Run-length Encoding (AC coeffs) • Predictive Encoding (DC coeffs) • Entropy Encoding (Huffman encoding) • Quantization (All coefficients)

3. P and B Frame Encoding • P and B frames • Motion compensation. • Search techniques • The problem with error measurements • Skipped macroblocks • Quantization control • Variable bitrate vs. Constant bitrate • DCT Artifacts • Spider noise • Blockiness

4. DCT Artifacts • Spiders • High frequency noise. • Shows up most with diagonal edges. • Blockiness • Quantization noise in lower frequencies between macroblocks creates an artificial edge. • Shows up a lot with smooth gradients.

5. P-Frames • Two types of macroblocks in P-Frames: • I-Macroblocks. • Just like macroblocks in a I-Frame • DC term is differentially encoded from DC predictor • AC terms • RLE’d as (run,value) pairs. Huffman encoded. • P-Macroblocks

6. Luminance Blocks U Block V Block Block Pattern (3- 9 bits) Motion Vector (variable) Q Scale (5 bits) Macroblock Type (1-6 bits) Macroblock Address Increment (variable) P-Macroblocks Macroblock Type determines if Q Scale, Motion Vector, or Block Pattern exist. One or all of the blocks may be absent in a P-Macroblock.

7. Address Increment • Each macroblock has an address. • MB_WIDTH = width of luminance / 16 • MB_ROW = row # of upper left pixel / 16 • MB_COL = col. # of upper left pixel / 16 • MB_ADDR = MB_ROW * MB_WIDTH + MB_COL • Decoder maintains PREV_MBADDR. • MB address increment added to PREV_MBADDR provides current macroblock address.

8. MB Type • Huffman coded. • 7 possible codes (1 - 6 bits) • Determine the following: • Intra or non-intra. • Q scale specified or not. • Motion vector exists or not. • Block pattern exists or not. • Not all combinations are possible. • Not all possible combinations are feasible.

9. Quantization Scale • 5 bits. • Zero is illegal. • Encoded as 1-31 which results in q-scale values of (2-62). • Odd values impossible to encode. • Decoder maintains current q-scale. • If not specified, current q-scale used. • If specified, current q-scale replaced.

10. Motion Vector • Two components: • Horizontal and vertical offsets. • Offset is from upper left pixel of macroblock. • Positive values indicate right and down. • Negative values indicate left and up. • Offsets are specified in half pixels. • Motion vector is used to define a predictive base for the current macroblock from the reference picture.

11. Motion Vector Illustrated Previously Decoded I- or P- Frame P-Frame Prediction base does not have to be macroblock aligned. If predictive base is half-pixel aligned, bilinear interpolation is used. Whatever luminance pixels are picked out, corresponding chrominance pixels used to form chrominance prediction.

12. P-Frame Encoding

13. Motion Vector Encoding • If no motion vector is present, then motion vector is understood to be (0,0). • Horiz. component followed by vertical. • Decoder maintains motion vector predictor. • Set to 0,0 at beginning of picture or slice or whenever an I-macroblock is encountered. • Difference between predictor and value is Huffman encoded.

14. Predictive Base • P-Macroblocks always specify a predictive base: • Either motion vector picks out an area, or • No motion vector implicitly implies 0,0 (i.e., predictive base is same macroblock in reference frame.)

15. Block Pattern • The goal of motion compensation is to find predictive base that matches most closely with macroblock. • If match is really good, then no appreciable difference will need to be encoded at all. • Block pattern indicates which blocks have enough error to warrant coding. • Absence of block pattern indicates no blocks needed coding.

16. Block Pattern Encoding • Huffman code • 3-9 bits • Encodes values 1-63 • 0 not needed because then pattern wouldn’t be encoded. • Each bit in the block pattern value corresponds to a particular block: 5 4 3 2 1 0 V U Lower right luminance Lower left luminance Upper right luminance Upper left luminance

17. Block • Difference between pixels in prediction and macroblock is encoded as block: • 9-bit input values • Still produces 12-bit coefficients • Sometimes called error blocks.

18. P-Frame Review • Macroblocks are either I-macroblocks or P-macroblocks. • I-macroblocks just like macroblocks in I-frame. • P-macroblocks define predictive base and encode the difference.

19. Skipped Macroblocks • If P-macroblock has (0,0) motion vector and no appreciable difference to encode, then can be skipped altogether. • Skipped macroblock detected when address increment for next coded macroblock is detected.

20. Decoder State Updates • DC predictors are reset whenever a P-macroblock or skipped macroblock is encountered. • Motion vector predictors reset whenever I-macroblock is encountered.

21. B-Frames • B-frames have 4 macroblock types: • I-macroblocks • P-macroblocks • Predictive base specified from previous reference frame. • B-macroblocks • Predictve base specified from subsequent reference frame. • Bi-macroblocks. • Predictive base specified from both reference frames.

22. Luminance Blocks U Block V Block Block Pattern (3- 9 bits) Backward Motion Vector (variable) Forward Motion Vector (variable) Q Scale (5 bits) Macroblock Type (1-6 bits) Macroblock Address Increment (variable) B-Frame Macroblock Structure Macroblock Type determines if Q Scale, Motion Vectors, or Block Pattern exist. One or all of the blocks may be absent in a P-, B- or Bi-macroblock.

23. Two Motion Vectors • Separate predictors are maintained for both components of both the forward and backward motion vectors. • Reset whenever it doesn’t get used. • If both are specified, then predictive base is formed by bilinear interpolation of both prediction areas. • Error encoded in blocks as per P-macroblocks.

25. Putting it all together Input Buffer VLC Decoder ZZ and DeQuant IDCT Forward MC Prev. Frame Store Adder Display Bi- directional MC Future Frame Store Backward MC

26. Motion Compensation • Provides most of MPEG’s compression. • Relies on temporal coherence. • Finding a good motion vector essentially a search problem. • Evaluating how goodness of a motion vector can be a bit tricky. • MC is what makes MPEG asymmetric. • Harder to encode than to decode.

27. Error Measurements • Regardless of search algorithm, need to determine which motion vector is best. • Simple measures: • Mean Squared Error • Mean Absolute Error • Minimum Difference Variance • Fundamental problem is no good correlation between any simple metric and perceptual quality.

28. VBR vs. CBR • Two ways to handle bitrate: • Variable Bit Rate (VBR) • Allows compressed bitrate to vary • Constant Bit Rate (CBR) • Bitrate constant over some averaging window. • MPEG buffer model. • Optional (don’t have to use it). • Provides in the sequence header parameters to a buffer model that can describe bitrate behavior.

29. VBR Q-scale adjustments • In general, VBR used to maintain quality. • Q scale is adjusted to provide maximum compression given quality limit. • Need some metric for quality. • Same issue for judging perceptual quality crop up here. • Common solution: q scale statically set for I-, P-, and B-frames. • A variation on this is differentiating among macroblock types.

30. CBR Q-scale adjustments • To achieve CBR, q-scale used to control bitrate. • Higher q-scale provides better compression at the expense of quality. • Lower q-scale provides better quality at the expense of compression. • Algorithms for controlling how q-scale is adjusted can get pretty complicated. • Common solution is to have target I, P, and B frame sizes and then adjust q-scale as macroblocks are encoded to hit the target.