MPEG-1 Video (Part 1)

1. MPEG-1 Video (Part 1) Ketan Mayer-Patel CS 294-9 :: Fall 2003

2. Encoding Techniques • Subsampling • Transform Coding • Run-length Encoding • Predictive Encoding • Entropy Encoding • Quantization CS 294-9 :: Fall 2003

3. Bitstream Organization Picture Data Seq. End Code Seq. Header Width Height Frame Rate Buffer Control GOP Header Time Code Picture Header Temporal Ref Picture Type Motion Vector Parameters All headers begin with 23 zeroes followed by 9 bits that indicate header type. Encoding process will never produce 23 zeroes. CS 294-9 :: Fall 2003

4. P I B B B B P B B P B B P B B I Frame Types • 3 Frame Types: I, P, B I : All information for frame present. P: Predictively encoded from previous I or P. B: Predictively encoded from previous I or P and next I or P. CS 294-9 :: Fall 2003

5. Frame Order • Predictive relationships create an obvious problem: B-frames depend on the future. • Obvious solution: send the frames out of order. P I B B B B P B B P B B P B B I 4 1 2 3 5 6 7 8 9 10 11 12 13 14 15 16 1 3 4 2 6 7 5 9 10 8 12 13 11 15 16 14 CS 294-9 :: Fall 2003

6. Source Input • Before we describe how I-frames are encoded, we should describe our input. • 3 planes of Y, U, V • 8 bits per pixel. • Y range [0,255]. • U and V range [-128,127] • Planes are all of the same size. • Pixels colocated between frames. CS 294-9 :: Fall 2003

7. Chrominance Subsampling • First step: downsize chrominance. • 4:2:0 (with chrominance samples centered) • Requires bilinear interpolation. • U and V biased by 128 to put in range [0,255] • Compression Ratio: 2:1 • Wow, doing well already. CS 294-9 :: Fall 2003

8. Subsampling In General • Severe loss of data. • Exploits imperceptibility of data loss. • In this case: human not as sensitive to color. • What if we were using images as input to feature extractor? • Depending on what was being extracted, subsampling might not be such a good idea. • Compression gain is directly related to subsampling factors. CS 294-9 :: Fall 2003

9. Macroblocks • Y is cut into 8x8 tiled pixel regions. • U and V cut into 8x8 tiled pixel regions. • Macroblock defined as 4 Y tiles that form a 16x16 pixel region and associated U and V tiles. • Macroblocks organized in row order fashion from top to bottom. • Compression gain: none. CS 294-9 :: Fall 2003

10. Discrete Cosine Transform • Each tile (aka block) in a macroblock is transformed with a 2D DCT. • DCT is an orthonormal, separable, frequency basis much like a Fourier transform. • 1-D case: 8 pixel values are transformed into 8 DCT coefficients. • 2-D case: apply 1-D transform to all of the rows and then apply 1-D transform to all of the columns. CS 294-9 :: Fall 2003

11. DCT Basis Functions DC AC CS 294-9 :: Fall 2003

12. DCT Properties • 8-bit pixel values produces 12-bit signed coefficient values. • Fast algorithms exist for computation. • 13 multiplies and 29 additions • Fixed point integer math. • Good perceptual properties. • Losing higher freq. results in a bit of blurring. • Ringing fairly minimal. CS 294-9 :: Fall 2003

13. Transform Coding Properties • No loss of data • Except for numerical errors • No compression either. • Used to rearrange the data into a form to make another coding technique more effective. CS 294-9 :: Fall 2003

14. Coefficient Quantization • Each block is now 64 coefficients instead of 64 pixel values. • Each coefficient quantized independently. • Allows larger quantization factors to be used with higher frequency coefficients. • Quantization is controlled by two parameters: • Quantization table. • Set in picture header or system header. • Two different tables, one each for intra and non-intra blocks. • Quantization factor. • Can be set on a per macroblock basis. Used to scale the table. • Can take value from (2-62) CS 294-9 :: Fall 2003

15. Quantization Properties • Data loss relative to quantization step. • Compression in two ways: • Smaller range to represent. • In our case 12 bit signed values turn into 9-bit signed values. • Creates runs of the same number. • In our case: runs of zeroes. CS 294-9 :: Fall 2003

16. Run Length Encoding • High quantization step size for higher frequency components results in lots of zero coefficients. • Run Length Encoding provides better representation. • Convert 2D matrix into 1D ordering of coefficients. • Reorganize as (run, value) pairs. • Run specifies number of zeroes to insert in the ordering before value appears. • Special marker that indicates nothing left but zeroes. CS 294-9 :: Fall 2003

17. Zig-Zag ordering • In order to group as many of the zeroes together, zig-zag ordering used. CS 294-9 :: Fall 2003

18. RLE Properties • Compression related to avg. size of run. • No data loss. CS 294-9 :: Fall 2003

19. DC Term Encoding • At this stage, each block in our macroblock is represented as a set of RLE’d DCT coefficients. • DC term is always coded even if it is zero. • Coded as difference between last DC term and current DC term. • Blocks are ordered within a macroblock. • Why code the difference? • Avg. pixel value of one block is likely to be correlated to nearby block. CS 294-9 :: Fall 2003

20. DC Term Encoding Cont’d • Now DC term is expressed as difference from previous DC term (DC_DIFF) • Encoded as two parts: • Size of difference (i.e., log(DC_DIFF)) • Size number of bits that provides the value. • Size is encoded as a Huffman code. CS 294-9 :: Fall 2003

21. Differential Encoding • Useful when values being encoded are well correlated. • Distribution of differences is expected to not be uniform. • No compression per se, but increases the efficiency of entropy encoding techniques (i.e., Huffman coding) CS 294-9 :: Fall 2003

22. AC Term Encoding • AC terms are given as (run,value) pairs. • Encoded in one of two ways: • Huffman code for (run, abs(value)) followed by single bit for sign of value. • Special Huffman code indicating ESCAPE, followed by 6 bits for run and either 8 or 16 bits for value. • 6 bits for run simply encode 0 through 63 • First 8 bits of value put value at –128 to 127. • If first 8 bits is -128, next 8 bits provide codes for –128 through –255 • If first 8 bits is 0, next 8 bits provide codes for 128 through 255. CS 294-9 :: Fall 2003

23. Entropy Coding • Huffman codes are a form of entropy encoding. • Relies on uneven distribution of values to be encoded. • Length of code associated with values inversely related to weight in distribution. • The more likely the value is to occur, the small the code length relative to all the other codes. • No data loss. • Compression depends on distribution. CS 294-9 :: Fall 2003

24. Stepping Back A Bit Picture Header Picture Data Row Major Scan of Encoded Macroblocks First Non-zero AC Coeff. (variable bit length) Luminance Blocks U Block V Block Last Non-zero AC Coeff. (variable bit length) Q Scale (5 bits) Macroblock Type (1 or 2 bits) Macroblock Address Increment (1-bit) DC Bits (0-8 bits) DC Size (2-7 bits) EOB (2 bits) CS 294-9 :: Fall 2003

25. Slices • One last level of organization. • Macroblocks grouped into slices. • Typically, one row of macroblocks in one slice. • Other groupings also possible. • Slice starts with a slice header. • Contains qscale. and indicates row in which slice starts. • Decoder state is reset. • DC predictors for Y, U, and V set to 1024. • Prev. macroblock address set to address of first macroblock in slice row (may not be first macroblock in slice). CS 294-9 :: Fall 2003

26. I-Frame Review • All macroblocks are intra-coded. • Blocks DCT’d and quantized to produce coefficients. • DC terms encoded differentially. • AC terms encoded with entropy codes associated with (run,value) pairs. • Escape code with fix length encoding for seldom used possibilities. • In general, compression ratio is 10:1 to 20:1 CS 294-9 :: Fall 2003