Image and Video Compression Fundamentals

1. Image and Video Compression Fundamentals Heejune AHN Embedded Communications Laboratory Seoul National Univ. of Technology Fall 2013 Last updated 2013. 9. 07

2. 1. Driving Force of Video Compr. • Uncompressed Video Bandwidth • Ver. Resolution x Hor. Resolution x Time Resolution x Colors • Eg. CCIR 601 (TV Quality) 720x480x30x24 = 248,832,000 bps • Typical Storage and Network • DVD 4.7 GB (about 80 sec for CCIR) • ADSL 100Mbps < CCIR BW

3. Typical values • Typical Video Bandwidth • ITU CCIR 601 L(858x525) C(429 x525) 30fps => 216.0Mbps • CIF L (352x288) C(176x144) 30fps => 36.5Mbps • QCIF L (176x144) C(88x72) 15fps => 4.6Mbps • Typical Storage /Transmission Capacity • Terrestrial TV broadcasting channel ~20 Mbps • CD/DVD-5 640MB/4.7GB • Ethernet/Fast Ethernet <10/100 Mbps • ADSL/VDSL downlink 2048 kbps/100Mbps • Wireless cellular (2G/3G/3G+) 9.6/384/2000kbps

4. 2. Image and Video Compression • Information Theory • 1950’sClaude Shannon (Bell Lab)pioneered. • Providing Mathematical Limits for Information Processing/Communications • Coding • Source Coding • How to Reduce the data • for information representation • Channel coding • How to Transmit Data • though Noise/Distored Channels • Note : TDMA, FDMA, CDMA, OFDMA, and MIMO are all for the channelization methods Claude Elwood Shannon (April 30, 1916 – February 24, 2001)

5. Typical Visual Comm. System • Typical path Info output channel decoder demodulator modulator Source decoder channel coder Source coder Info source Channel (wired/wirless/storage)

6. Codec • Codec = enCOder&DECoder • Codec Types • Lossless compression • X == X’ • Used for document file (ZIP), Medical Images (JPEG lossless) • Entropy coding (Arithmetic coding, Huffman coding), Predictive coding • Lossy compression • X ~ X’ • Used for Entertainment, Communication Multimedia • (DCT), Quantization Encoder Decoder X’ Y X

7. Uncompressed, Zipped, H264-encoded of same video • Video Compression System Feature • Source model • Note: zip is source-independent encoding • Human Visual System • HVS does not notice many distortions

8. + + Entropy Decoder Entropy Coder - + line of pixels above Predictor Predictor Decoder Encoder current line of pixels 3. Predictive Coding • DPCM (Differential Pulse Coded Modulation) • Highly Correlated pixel values in Spatial Domain • Code current (S0) using previously coded ones (S1, S2, S3 etc) • Coder Block Diagram

9. original 0 0 0 1 1 0 0 0 0 0.5 0 0.5 • DPCM example

10. Motion Compensation Prediction • Temporal domain prediction • How to use the temporal correlation? • Model and representation methods Two successive video frames Changedetectionmask

11. Model based MC • 2D/3D Model • dx, dy, dz and rotations • Estimate (ie. Calculate) the parameters in encoder and use for decoder • Difficulties • Too high Shape encoding, Estimation Complexity for now • In MPEG-4 Object Oriented coding Moving area picked up by change detector Moving areas missed by change detector

12. Real Motion MV X(t+1) X(t) • Block Based MC • Segment Fixed Size Block and find best matching displacement • Easier Implementation in HW and SW

13. 4.Transform coding • Transform • Spatial Domain to Frequency Domain • Easy for quantization • Energy Compaction Properties and HVS properties • No Compression itself

14. Block transform • (fixed-size) Block Transform • Easy for implementation • Normally 2-D separable Transform

15. Transform types • KL Transform is proved optimal • DCT is fixed and similar to KL for image signals • Wavelet and Fractal Transform etc (1) (2) (3) (4) (5) (1) Karhunen Loève transform [1948/1960] (2) Haar transform [1910] (3) Walsh-Hadamard transform [1923] (4) Slant transform [Enomoto, Shibata, 1971] (5) Discrete CosineTransform (DCT) [Ahmet, Natarajan, Rao, 1974]

16. Transform size • The Larger Block, The more efficient, but The more Computationally complex • 8x8 or 4x4 are used for Standards

17. 5. Quantization • Approximation of Values • Lossy Coding (key data reduction) • Applied to 2D transform Coefficient

18. Quantizer output D D Quantizer input D+a • Qstep (or qscale) • Distortion Range • The Larger/Coarse Q step • The More Compression • The Larger Distortion • Rate Distortion Theory • In Video Coding • Applied to 2D transform Coefficients • HVS • Smaller in low freq • Larger in high frequency

19. 6. Entropy Coding • Statistical redundancy in video coding • Many zeroes in quantized transform coefficients • Unequal histogram of control info, like motion vectors and coding type • Entropy coding • Principle • “Shorter Code words for More Frequency events” • Variable Length Coding (VLC) • Huffman coding • Integer VLC: each code words are integer length • Used for most Standards • Arithmetic Coding • Fractional Length Coding • Started from H.263+ but used in H.264 practically

20. VLC coding in Image Coding • Zigzag scan used for more statistical correlation • 2-D Run-Length Code (num of zeros, no zero value) Q (8) Transformed 8x8 block Zig-zag scan Run-level coding

21. Coder Control Control Data DCT Coefficients Intra-frame DCT Coder Quant - Intra-frame Decoder Decoder DeQ Entropy coder 0 Motion- Compensated Predictor Intra/Inter Motion Data Motion Estimator 7. Codec Design • Hybrid Codec • Most Standards Codec • MC => DCT => Quant => Entropy Coding

22. Complexity Consideration • Asymmetric Complexity • Encoders are more complex for most standards • Non-real time Encoding but Real time Encoding (e.g. Broadcasting, Storage) • One time encoding many time decoding • Encoder and decoder Cost • Parallel Processing and HW/SW implementation (in MPEG-2) • Motion Compensation (~ 55%) • DCT/DCT (~15%) • VLC encoding/Decoding (~15%) • Other (post processing) (15%)