JPEG

1. JPEG Hao Jiang Computer Science Department Sept. 27, 2007

2. What is JPEG? • JPEG: Joint Photographic Expert Group — an international standard in 1992. • Works for both color and grayscale images. • Targets at natural images. • Applications include satellite, medical imaging, general photography ...

3. The System of JPEG 8x8 block DCT Quantization Quantization tables Header DC Entropy coding Tables Difference coding Data Run length coding AC

4. Color Space Conversion • JPEG first converts RGB to YUV or YCrCb. • Y is the luminance component (brightness). Y = 0.299 R + 0.587 G + 0.144 B • U and V are color components U = B – Y V = R - Y Y U V

5. Color Subsampling • JPEG down-samples the “color channels” by half and partitions images into 8x8 small blocks. U V Y1 Y2 Y3 Y4 Intensity Component Color components

6. DCT (Discrete Cosine Transform) • DCT converts each image 8x8 block into another 8x8 block. • The energy in DCT domain is concentrated into very few coefficients. Image block DCT DCT coefficients

7. Image block DCT Coefficients

8. Definition of DCT A linear transfrom F(u,v) = åx=07åy=0 7 h(u,v,x,y) f(x,y) where h(u,v,x,y) is the linear weighting function. DCT: The inverse transform IDCT recovers the original data from DCT coefficients.

9. For DCT, h(u,v,x,y) can be decomposed into g(u,x) *g(v,y) F(u,v) = åx=07åy=0 7 h(u,v,x,y) f(x,y) = åy=07g(v,y)[åx=0 7g(u,x) f(x,y)] followed by Horizontal 1D DCT Vertical 1D DCT

10. Given u and v, h can be viewed as an image of x and y. u v DCT coefficients are projection of image block into these patterns.

11. Quantization • Recall that quantization can be used to collapse the input into smaller number of values. • For DCT coefficients, we want to quantize different coefficients in different details: • DC coefficient should have the most quantization levels. • Quantization for AC coefficients can be coarser and coarser as the frequency increases.

12. Y quantization table U, V quantization table 17 18 24 99 99 99 99 99 18 21 26 66 99 99 99 99 24 26 56 99 99 99 99 99 47 66 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 Quantization: Fq(u,v) = round(F(u,v)/Q(u,v)) De-quantization: Fr(u,v) = Fq(u,v) * Q(u,v)

13. quantization dequantization

14. The Quality Factor • For most current JPEG encoders, we can choose a quality factor from 1 to 100. • The method of controlling the compression quality is by scaling the quantization table. For example, Fq(u,v) = round(F(u,v)* (quality) / Q(u,v))

15. DC Coefficients Encoding D(m,n) D(m,n+1) D(m,n+2) D(m,n+3) We compute the difference of each two successive quantized coefficients D(m,n) – 0 => d(m,n) D(m,n+1) – D(m,n) => d(m,n+1) D(m,n+2) – D(m,n+1) => d(m,n+2) D(m,n+3) – D(m,n+2) => d(m,n+3)

16. DC Coefficients Encoding • Then, we encode each DC difference value by (size, coefficient) • 1 -1, 1 • 2 -3, -2, 2, 3 • -7 …-4, 4, …,7 • 4 • … • 11 -2047,…,-1024,1024,…,2047 • 7 will be coded as (3, 7) • The first number 3 is Huffman • coded. • The second 7 is encoded as • 1’s complement 111. • (-7 will be 000) The Size Table Can we encode the dc difference value directly?

17. AC Coefficients Encoding • AC coefficients are not differentially encoded. • Instead, we first do run-length coding. The run length code: (0,11), (0,-12), (1, -12), (0, 10), (0,16), (0, 0) Indicates all are 0 from here Zig-zag scanning

18. AC Coefficients Encoding (cont) • For AC coefficients, we now have a bunch of symbols like • The “value” has large number of possible values in [-1023, 1023]. Direct Huffman coding is infeasible. • Instead, we generate symbols like (zero-run length, value) ( zero-run length, size, value) 1’s complement Huffman coded

19. JPEG File Format Start of image Frame End of image … Tables Header Scan Scan Scan Tables Header Segment Restart Segment …. Block Block Block

20. JPEG Extensions • The basic mode of JPEG supports sequential coding (the order is from top to bottom and left to right). • JPEG extensions support progressive modes: • Spectrum selection. • Successive approximation. • Hierarchical mode.

21. JPEG 2000 • JPEG 2000 is a new standard based on wavelet transform. • JPEG2000 does not partition an image into small blocks. • Wavelet decomposes the whole image into different “bands” and then encodes the coefficients in different bands smartly.

22. LL LH HL HH The wavelet decomposition of Lena image using Haar Wavelet.