Improved Shape-adaptive image compression by morphological segmentation

1. National Taiwan University Presenter: Tzu-Heng Henry Lee Research Advisor: Jian-Jiun Ding , Ph. D. Assistant professor Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University Improved Shape-adaptive image compression by morphological segmentation

2. Introduction to Shape-Adaptive Image Compression • Morphological Segmentation Using Erosion • Shape-Adaptive Transform Algorithm • Quantization • Coding Technique of the Image Segment • Simulations • Conclusion and Future Work Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University overview

3. The idea is to exploit high correlation between the color values in the neighboring pixels within the same image segment. • Characteristics in an image segment usually share the similar color values(the color intensity variations are low). • The arbitrarily-shaped image segment can be completely represented by its shape and internal contents [1]. Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University Introduction

4. JPEG images normally display various kinds of undesired distortion artifacts such as • blocking, • blurring, and • Ringing. • Compressions with low bit-rates • Lossy quantization process is used to compress the DCT coefficients. Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University Motivation - Distortion artifact pproblem

5. key features that distinguish the improved algorithm are built around two central components: • Morphological segmentation, and • Shape-adaptive DCT with orthogonal bases. Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University The improved compression algorithm

6. Q: Why do we include this stage in our algorithm? A: The color values at the edge of an segmented object usually vary significantly. The contour region of a segment contains a great portion of the high frequency components Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University Morphological Segmentation Using Erosion

7. Contour sub-region The overall internal region Interior sub-region • This allows us to compress the contour sub-region and the interior sub-region of an arbitrarily image segment separately. • So we can minimize quantization noise and enhance overall quality of image compression. Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University Morphological Segmentation Using Erosion

8. Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University Divide the image segment by morphological erosion

9. Traditional Method: • fill zeros outside the contour of the arbitrarily image and treat the whole image block as a traditional image block [2]. • Drawback: • This increases the high-order transform coefficients which are later truncated. • Leads to performance degradation. Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University Shape-Adaptive Transform Algorithm

10. Based on the concept of KLT(Karhunen-Loeve). • Generic transform that does not need to be computed for each image can be derived. • Lower compuational complexity. • Provides a good compromise between information packing ability and computational complexity [A1]. Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University Discrete Cosine Transform (DCT)

11. DCT produces less blocking artifacts compared to DFT. • 1-D point of view. • The implicit n-point periodicity of the DFT boundary discontinuities High freq • Truncation  Gibb’s phenomenon • The DCT which has the implicit 2n-point periodicity does not produce such discontinuities [1]. Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University Discrete Cosine Transform (DCT)

12. DCT-based. • Since the height and width of an arbitrarily-shaped image segment are usually not the same, • we redefine the forward DCT as • for and Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University Shape-Adaptive Transform Algorithm

13. The inverse DCT can also be re-written as • and the DCT basis is expressed as Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University Shape-Adaptive Transform Algorithm The DCT bases are not yet customized for a particular arbitrarily-shaped image segment.

14. Since we are using the traditional DCT bases, we can simply project these basis functions into subspace SB: • A linear combination of can be used to describe the arbitrarily segment vector P(x, y). • This operation removes the components of outside subspace SB [2]. Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University Shape-Adaptive Transform Algorithm

15. An example of the projection operation: • shape matrix:formed by filling 1’s in the pixel position inside the contour of the arbitrary shape. Zeroes are filled in the region outside the contour. Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University Shape-Adaptive Transform Algorithm

16. The 88 DCT bases with the shape of our example. Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University Shape-Adaptive Transform Algorithm

17. The number of orthogonal bases M is less than HW • The same basis function could be repetitively chosen. • Generally the HW shape-projected bases are not orthogonal because the number of transform coefficients may exceed the image segment size [2]. Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University Orthogonalization of the Shape-Projected Bases

18. One of the methods to obtain orthogonal basis functions in the subspace SB is to use the Gram-Schmidt algorithm [2], [3], [4], [5]. • We use the Gram-Schmidt process to reduce the bases to M orthogonal ones. Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University Orthogonalization of the Shape-Projected Bases

19. Before we use the Gram-Schmidt process to reduce the bases to M orthogonal ones, we reorder the HW shape-projected bases by the zig-zag reordering matrix [6]: Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University zig-zag reordering

20. make the low frequency components to concentrate on the top-left corner • move the less important high frequency components to concentrate on the bottom-right corner of the matrix. • This is because the low frequency components contain a significant fraction of the total image energy [7]. Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University zig-zag reordering

21. Low-frequency AC coefficients are placed before high-frequency AC coefficients. • Makes upcoming entropy coding process much easier. • By keeping higher frequency coefficients (which are more likely to be zero after quantization) together, we can form long runs of zeros [8]. Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University zig-zag reordering

22. JPEG – fixed quantization matrix for 8X8 blocks • Our method – The length of the quantization array corresponding to the arbitrary-shape DCT coefficients is not fixed. • We define an extendable and segment shape-dependent quantization array Q(k) as a linearly increasing line: for k = 1, 2,…, M. Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University Quantization

23. Need to encode the quantized arbitrary-shape DCT coefficients to bit stream. • We use the same coding technique that is used in JPEG. • The quantized coefficients are a series of integer values with large values at the beginning(DC terms) of the series followed by a large amount of zeros at the back(AC terms). Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University Coding Technique of the Image Segment

24. The DC coefficient is treated separately from the AC coefficients. • Difference Encoding: It is encoded as the difference between the present DC term and the one from the previous block. • The AC-terms are encoded by zero-run length coding(ZRL) and the Huffman coding [6], [7]. Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University Coding Technique of the Image Segment

25. We directly combine all the encoded bit streams of all image segments. • In the ZRL coding process, we truncate the successive zeroes in the end of the coefficients, and replace them with an end-of-bit (EOB) symbol. • We can divide the bit stream to each image segment by the EOB symbol in the decoding process. Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University Coding Technique of the Image Segment

26. Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University Coding Technique of the Image Segment(Encoding)

27. Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University Coding Technique of the Image Segment(Decoding)

28. Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University Simulation Results Shape-Adaptive Compression with Morphological Segmentation JPEG Shape-Adaptive Compression

29. Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University Simulation Results (Top) Original image (11080 bytes). (Left) JPEG compressed image (RMSE: 3.9931 and data size: 1128 bytes). (Right) Compressed image using our proposed algorithm (RMSE: 2.7502 and data size: 410 bytes) (Top) Original image (11080 bytes). (Left) JPEG compressed image (RMSE: 4.2714 and data size: 1428 bytes). (Right) Compressed image using our proposed algorithm (RMSE: 2.9509 and data size: 419 bytes)

30. The complexity of Gram-Schmidt orthogonal process: O(n2) • n - the number of points of an image segment Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University Complexity Issues and Proposed Solution What if an image segment is large? It would cost a lot of computational time

31. Solutions: • Segment the image in more detail such that the number of points of an image segment is confined in an acceptable range. • A number of bases smaller than the dimension of the image segment can be chosen to avoid n being too large Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University Complexity Issues and Proposed Solution

32. The JPEG method has a poorer because it cannot utilize the characteristics of the image. • Significant improvements on the distortion artifacts caused by the quantization process are made by using the shape-adaptive compression algorithm with morphological segmentation. • A higher compression rate with a comparable RMSE. Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University Conclusion

33. Improvements on Huffman Coding algorithm. • More efficient ways to segment the image. • Improvements on compression efficiency. • Elimination of the flaws on the erosion operation(small segment problems ) Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University Future work

34. [1] R. C. Gonzalez and R. E. Woods, Digital Image Processing Second Edition, Prentice Hall, New Jersey, 2002. [2] S. F. Chang and D. Messerschmitt, “Transform coding of arbitrarily shaped image segments,” Proc. 1st ACM Int. Conf. Multimedia Anaheim, CA, pp. 83- 90, 1993. [3] M. Gilge, T. Engelhardt, and R. Mehlan, “Coding of arbitrarily shaped image segments based on a generalized orthonormal transform,” SignalProcess: Image Commun., vol. 1, pp. 153–180, Oct. 1989. [4] J. Apostolopoulos and J. Lim, “Coding arbitrarily-shaped regions,” Proc. SPIE Visual Commun. Image Process., pp. 1713-1726, May 1995. [5] R. Stasinski and J. Konrad, “A new class of fast shape-adaptive orthogonal transforms and their application to region-based image compression,” IEEE Trans. on Circuitsand systems for Video Technology, vol. 9, pp. 16–34, 1999. [6] W. B. Pennebaker and J. L. Mitchell, JPEG Still Image Data Compression Standard.New York: Van Nostrand Reinhold,1993. [7] C. K. Wallace. The JPEG still picture compression standard. Communications of the ACM, 34(4):31-44, 1991. [8] T. Acharya amd A. K. Ray, Image Processing Principles and Applications, John Wiley & Sons, New Jersey. Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University References

35. Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University Questions? Duh?