Tutorial on Image Compression

1. Tutorial onImage Compression Richard Baraniuk Rice University dsp.rice.edu

2. Agenda • Image compression problem • Transform coding (lossy) • Approximation • linear, nonlinear • DCT-based compression • JPEG • Wavelet-based compression • EZW, SFQ, EQ, JPEG2000 • Open issues

3. Image Compression Problem

4. Images • 2-D function • Idealized view some function space defined over

5. Images • 2-D function • Idealized view some function space defined over • In practice ie: an matrix

6. Images • 2-D function • Idealized view some function space defined over • In practice ie: an matrix (pixel average)

7. Quantization 111 110 101 100 011 010 001 000 • Approximate each continuous-valued pixel valuewith a discrete-valued variable • Ex: 3-bit quantization = 8-level approximation • Human eye “blind”to 8-bit quantization= 256 levels

8. From Images to Bits

9. The Need for Compression • Modern digital camera megapixels

10. How Much Can We Compress? [M. Vetterli +] • 2(256x256x8) possible images ~500,000 bits [David Field] • Dennis Gabor, September 1959 (Editorial IRE) “… the 20 bits per second which, the psychologists assure us, the human eye is capable of taking in …” • Index all pictures ever taken in the history of mankind 100 years x 1010 ~44 bits • Search the Web google.com: 5-50 billion images online ~33-36 bits • JPEG on Mona Lisa ~200,000 bits • JPEG2000 takes a few less, thanks to wavelets …

11. From Images to Bits is it Lena?

12. Lossy Image Compression • Given image approximate using bits • Error incurred = distortionExample: squared error

13. Rate-Distortion Analysis achievable region

14. Rate-Distortion Analysis

15. Rate-Distortion Analysis better compression schemes push the R-D curve down

16. Rate-Distortion Analysis achievable region

17. Rate-Distortion Analysis better compression schemes push the R-PSNR curve up

18. Lossy Transform Coding

19. Image Compression • Space-domain coding techniques perform poorly • Why? smoothness strong correlations redundancies too many bits

20. Transform Coding • Quantize coefficients of an image expansion coefficients basis, frame

21. Transform Coding • Quantize coefficients of an image expansion coefficients basis, frame quantize to total bits

22. Wavelet Transform • Standard 2-D tensor product wavelet transform

23. Transform Coding Transform – sparse set of coefficients (many 0)

24. Transform Coding Quantize – approximate real-valued coefficients using bits – sets small coefficients = 0

25. Quantization and Thresholding 111 110 101 100 011 010 001 000 • Quantization thresholds small coefficients to zero

26. Quantization and Thresholding • Quantization thresholds small coefficients to zero 111 110 101 100 011 010 001 000 deadzone

27. Transform Coding Quantize – approximate real-valued coefficients using bits – sets small coefficients = 0

28. Transform Coding bits Entropy code – reduce excess redundancy in the bitstreamEx: Huffman coding, arithmetic coding, gzip, …

29. Transform Coding/Decoding bits bits

30. Sparse Approximation

31. Computational Harmonic Analysis • Representation • Analysis study through structure of should extract features of interest • Approximation uses just a few terms exploit sparsityof coefficients basis, frame

32. Wavelet Transform Sparsity • Many(blue)

33. Sparseness Approximation few big sorted index many small

34. Nonlinear Approximation few big sorted index • -term approximation: use largestindependently • Greedy / thresholding

35. Linear Approximation index

36. Linear Approximation • -term approximation: use “first” index

37. Error Approximation Rates • Optimize asymptotic error decay rate • Nonlinear approximation works better than linear as

38. Compression is Approximation • Lossy compression of an image creates an approximation coefficients basis, frame quantize to total bits

39. NL Approximation is not Compression • Nonlinear approximation chooses coefficients but does not worry about their locations threshold

40. Location, Location, Location • Nonlinear approximation selects largestto minimize error (easy – threshold) • Compression algorithm must encode both a set of and their locations(harder)

41. Local Fourier CompressionJPEG

42. JPEG Motivation • Image model:images are piecewise smooth • Transform: Fourier representation sparse for smooth signals edge texture texture

43. JPEG Motivation • Image model:images are piecewise smooth • Transform: Fourier representation sparse for smooth signals • Deal with edges: local Fourier representation (DCT on 8x8 blocks)

44. JPEG and DCT • Local DCT(Gabor transformwith square windowor wavelet packets) • Divide image into 8x8 blocks • Take Discrete Cosine Transform(DCT) of each block

45. Discrete Cosine Transform (DCT) • 8x8 block • Project onto64 differentbasis functions(tensor productsof 1-D DCT) • Real valued • Orthobasis

46. Discrete Cosine Transform (DCT) • 8x8 block • Project onto64 differentbasis functions(tensor productsof 1-D DCT) • Real valued • Orthobasis increasingfrequency

47. Discrete Cosine Transform (DCT) • 8x8 block • Project onto64 differentbasis functions(tensor productsof 1-D DCT) • Real valued • Orthobasis rapid coefficientdecay for smooth block

48. JPEG Quantization more bits 8 6 4 2 0 fewer bits

49. JPEG Quantization more bits 8 0 6 4 zero 2 0 fewer bits • Quasi-linear approximation in each block(fixed scheme)

50. JPEG Compression 256x256 pixels, 12,500 total bits, 0.19 bits/pixel