Radhika Veerla radhika.veerla@uta EE Graduate student, UT Arlington Supervising Professor: Dr.K.R.Rao

1. EE 5359 Multimedia Processing Project Report Implementation of AIC based on I-frame only coding in H.264 and comparison with other still frame image coding standards such as JPEG, JPEG 2000, JPEG-LS and JPEG-XR Radhika Veerla radhika.veerla@uta.edu EE Graduate student, UT Arlington Supervising Professor: Dr.K.R.Rao The University of Texas at Arlington July 24th, 2008

2. Outline • Review: • Advanced Image Coding • H.264 standard • JPEG, JPEG-LS, JPEG2000, JPEG-XR • Implementation of AIC and results • Comparison with other standards and results • Comparison in terms of rate-distortion metrics – PSNR and SSIM • Conclusions and future work • References

3. Advanced Image Coding Block Diagram (a) Encoder [1] (b) Decoder [1]

4. Advanced Image Coding It is a still image compression system which is a combination of H.264 and JPEG standards. • It is based on the basic technologies: • Use of transform to reduce spatial correlation • Quantization for the control of bitrate • Entropy coding for reduction in statistical correlation Features: • No sub-sampling- higher quality / compression ratios • 9 intra prediction modes as in H.264 • Predicted blocks are predicted from previously decoded blocks • Uses DCT to transform 8x8 residual block instead of transform coefficients as in JPEG • Employs uniform quantization • Uses floating point algorithm • Coefficients transmitted in scan-line order • Makes use of CABAC similar to H.264 with several contexts

5. Cr Cr Y, Cb, Cr Blks Res Q Huff FDCT  ZZ CC Cb Cb A A C +  B B G G R R Y Pred Blk Table Predictor Y Y mode Block Predict Mode Select and Store Dec Y DecY DecCb DecCr + Res Q1 IDCT  + ModeEnc Y,Cb,Cr Blks Res Q1 IHuff IDCT  IZZ ICC + A A D + Pred Blk Table DecY DecCb DecCr ModeDec and Store mode Block Predict M-AIC Algorithm (a) M-AIC Encoder [36] (b) M-AIC Decoder [36] CC - color conversion, ICC - Inverse CC, ZZ – zig-zag scan, IZZ – inverse ZZ, AAC – adaptive arithmetic coder, AAD – AA decoder.

6. H.264 Block Diagram [2] • It achieves higher compression and better performance than previous coding standards • In addition to the basic techniques mentioned in AIC: • - Motion compensated prediction for reduction of temporal correlation and • It includes enhanced coding tools: • - Adaptive intra-picture prediction • - Small block size transform with integer precision • - Multiple reference pictures and generalized B frames • - Content adaptive inloop deblocking filter • - Improved entropy coding – CABAC and CAVLC • There is an increase in the complexity at the encoder and decoder

7. H.264 Main Profile Intra-Frame Coding • Transform block size reduced from 8x8 to 4x4 • H.264 relies on spatial prediction taking the advantage of inter-block spatial correlation • Uses multiplier-less integer transforms and implemented in 16-bit fixed point architectures • Block DCT with inter-block correlation is competitive with global wavelet coding used in JPEG2000 • Improved entropy coding – CABAC or CAVLC • H.264 High Profile Intra-Frame Coding • H.264 Fidelity range extensions support higher-resolution color spaces, high bit depths, 8x8 intra prediction and 8x8 transform • Advantage- improves coding efficiency by adding 8x8 integer transform, prediction schemes associated with adaptive selection between 4x4 and 8x8 transforms • It is also observed that both main and high profiles produce similar results, so we considered only main profile for all the images.

8. JPEG Baseline Encoder and Decoder (a) Encoder [6] (b) Decoder [6] • Features: • 8x8 block based DCT – used in applications which need very good compression for a given level of visual distortion • Scalar quantization • Different quantization tables for luminance and chrominance components • Employs Huffman coding

9. JPEG 2000 • Based on wavelet/subband coding technique • EBCOT scheme for coding wavelet coefficients • Adaptive context-based binary arithmetic coding • Features: • - Lossy and lossless representations embedded with the same code-stream • - Region-of-interest (ROI) coding • - Support for continuous-tone, bi-level and compound image coding Block diagram of JPEG 2000 (a) encoder (b) decoder

10. Adaptive VLC table switching Adaptive VLC table switching 8x8 blocks Quantization tables Features: Employs fully reversible transforms HD photo format supports high dynamic range encoding possible through floating point algorithm. It minimizes objectionable spatial artifacts preserving high frequency detail and outperforms other lossy compression standards in this regard. Its image quality is comparable to JPEG2000 with computational and memory requirements closely comparable with JPEG. Quantization tables Reversible int-int mapping LBT Scalar quantization VLC Encoding VLC Decoding Scalar Dequantization Reversible int-int mapping inverse LBT Original image Reconst-ructed image Coded image Block based encoder JPEG-XR Coded image a) HD photo encoder b) HD photo decoder

11. JPEG-LS • This algorithm combines the simplicity of Huffman coding with the compression potential of context models. • Based on LOw COmplexity LOw COmpression for Images (LOCO-I) algorithm • Supports lossless and near-lossless compression of continuous tone images • Line interleaved mode – One row per image component is processed at a time in an interleaved manner. Both gray and color images are compressed in this mode. • Low complexity and cost in algorithm design and implementation Context modeling Prediction Error encoding Digital source data Compressed image data Run mode Basic block diagram of JPEG-LS

12. Applications of all the codecs • AIC - digital cameras, medical imaging, mobile phones, internet browsers, personal entertainment appliances • JPEG - digital cameras, fixed-bit rate image transmission devices, security systems, cost-sensitive image compression systems, printers, scanners, facsimile • JPEG 2000 - digital cinema, video surveillance, defense and medical imaging, computer graphics/ animation • JPEG XR - digital cameras, storage of continuous tone still images • JPEG LS – still imaging, video applications, biometrics, medical imaging, documents • H.264 – Broadcast, interactive, conversational and video streaming applications

13. Test images Lena (512x512) Airplane (512x512) Peppers (512x512) Sailboat (512x512) Splash (512x512) Couple (256x256) Lena (256x256) Lena (128x128) Cameraman (256x256) man (256x256) Lena (64x64) Lena (32x32)

14. Evaluation MethodologySoftwares, parameters and test image formats used for comparison

15. Codec Settings H.264 • Main and high profiles in 4:2:0 coding mode • ProfileIDC = 77 # Profile IDC (77=main, FREXT Profiles: 100=High) • LevelIDC = 40 # Level IDC (e.g. 20 = level 2.0) • IntraProfile = 1 # Activate Intra Profile for FRExt (0: false, 1: true) • Deblocking filter: off • QPISlice = 12 # Quant. param for I Slices (0-51) • YUVFormat = 1 # YUV format (0=4:0:0, 1=4:2:0, 2=4:2:2, 3=4:4:4) The command line arguments for JM13.2 software are: • Encoder: lencod –f encoder.cfg • Decoder: ldecod - i bitstream.264 - o output.yuv –r reference (input).yuv JPEG Encoder: • cjpeg – quality N inputfile.bmp outputfile.jpg where quality factor –N denotes the scale quantization tables to adjust image quality. Quality factor varies from 0 (worst) to 100 (best); default is 75. Decoder: • djpeg – outfile outputfilename.bmp –outputfileformat inputfile.jpg

16. Codec Settings (contd.) JPEG2000 Encoder: • jasper --input inputfilename.bmp --output outputfilename.jp2 –output-format jp2 –O rate=0.01 (or) • jasper –f inputfilename.bmp –F outputfilename.jp2 –T jp2 –O rate=0.01 where rate specifies target rate as a positive real number. Rate=1 corresponds to no compression HD photo • No tiling • One-level of overlap in the transformation stage • No color space sub-sampling • Spatial bit-stream order • All sub-bands are included without any skipping Encoder: • Wmpencapp –i input.bmp –o output.wdp –q 10 • Increase in the quality factor ‘q’ leads to lowering of PSNR resulting in lossy compression. q=0 is the case of lossless compression. Decoder: • WMPDecApp command line converts HD photo files to different uncompressed file formats. • For example: wmpdecapp –i input.wdp –o output.bmp –c 0 • where ‘c’ denotes format, c – 0 for 24bppBGR, c-2 for 8bppGray JPEG-LS • Line interleaved mode is considered in the project. • Error value is varied from 1 to 60. Error value of zero corresponds to no compression. • T1, T2, T3 are thresholds. While giving the settings the following condition needs to be met. Error value+1<T1<T2<T3. • Default RESET value of 64 is considered in the project

17. Image Quality Measures • Criteria to evaluate compression quality • Objective quality measure- PSNR, MSE MSE and PSNR for a NxM pixel image are defined as where x is the original image and y is the reconstructed image. M and N are the width and height of an image and ‘L’ is the maximum pixel value in the NxM pixel image. L=255 for 8 bit depth image. • Compression ratio and rate are related as where 24 is used for color images and 8 for gray scale. (1) (2)

18. Structural Similarity Method • This method emphasizes that the Human Visual System (HVS) is highly adapted to extract structural information from visual scenes. Therefore, structural similarity measurement should provide a good approximation to perceptual image quality. • The SSIM index is defined as a product of luminance, contrast and structural comparison functions. [29] where μ is the mean intensity, and σis the standard deviation as a round estimate of the signal contrast. C1 and C2 are constants. M is the numbers of samples in the quality map.

19. Original and output decoded images: Original AIC, quality -52.83bpp,HDphoto, quality-28, 2.88bpp, JPEG quality-94, 2.94bpp, 36.61dB, SSIM-0.914 37.74dB, SSIM-0.928 35.6dB, SSIM-0.926 JPEG-LS error value-11, 2.8bpp, JPEG2000 rate=0.12, 2.95bpp, H.264 quantization parameter-16, 2.83bpp, 32.425dB, SSIM- 0818 37.53dB, SSIM-0.923 46.81dB, SSIM-0.917

20. SSIM simulation results AIC JPEG JPEG 2000

21. SSIM results (contd.) HD Photo JPEG -LS H.264

22. Results for images of same resolutions

23. Simulation results for Lena (512x512x24)

24. Simulation results for Airplane (512x512x24)

25. Simulation results for Peppers (512x512x24)

26. Simulation results for Sailboat on lake (512x512x24)

27. Simulation results for Splash (512x512x24)

28. Simulation results for Couple (256x256x24)

29. Simulation results for Cameraman (256x256x8)

30. Simulation results for Man (256x256x8)

31. Results for images of different resolutions

32. Simulation results for Lena (32x32x24)

33. Simulation results for Lena (64x64x24)

34. Simulation results for Lena (128x128x24)

35. Simulation results for Lena (256x256x24)

36. SSIM result for Lena (512x512x24) image • It is observed that almost all the codecs have similar performance except JPEG-LS. • AIC outperforms up to 5.5bpp and remains in competition beyond that bpp range.

37. Conclusions • Performance evaluated using different R-D curves like PSNR and SSIM. • AIC outperforms JPEG by about 5dB and performs similar to or surpasses the JPEG2000 performance below 2bpp. • Typical bit rate range for AIC is 0-2bpp for color images and 0-4bpp for gray scale images • H.264 outperforms every other codec for images of all resolutions, but does not work for gray-scale images. The main concern is its complexity. • HD photo performs second in performance to H.264 up to typical bits per pixel range of 0-6bpp approx. for color images and its range increases with decrease in image resolution and 0-3bpp for gray scale images. • JPEG 2000 performs close to HD photo. The gap increases with decrease in resolution however in favor of HD photo. • JPEG LS takes the lead at bit range higher than approximately 5bpp. • Typical bit range for JPEG LS is above 4.5 bpp • The limitation of JPEG reference software is that it has low dynamic range. • At low bit range, performance of JPEG is better than JPEG-LS. • AIC is preferred because of its optimal performance with reduced complexity and increased speed. Suitable for web pages. • Based on SSIM measurement, AIC outperforms up to 5.5bpp and remains in competition even beyond that bpp range. Future work • Implement CABAC entropy coding • Consider lossless compression

38. References [1] AIC website: http://www.bilsen.com/aic/ [2] T. Wiegand et. al, “Overview of the H.264/AVC Video Coding Standard,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, pp.560-576, July 2003. [3] G. Sullivan, P. Topiwala and A. Luthra, “The H.264/AVC Advanced Video Coding Standard: Overview and Introduction to the Fidelity Range Extensions,” SPIE Conference on Applications of Digital Image Processing XXVII, vol. 5558, pp. 53-74, Aug. 2004. [4] I. Richardson, H.264 and MPEG-4 Video Compression: Video Coding for Next-Generation Multimedia, John Wiley & Sons, 2003. [5] P. Topiwala, “Comparative study of JPEG2000 and H.264/AVC FRExt I-frame coding on high definition video sequences,” Proc. SPIE Int’l Symposium, Digital Image Processing, San Diego, Aug. 2005. [6] P. Topiwala, T. Tran and W.Dai, “Performance comparison of JPEG2000 and H.264/AVC high profile intra-frame coding on HD video sequences,” Proc. SPIE Int’l Symposium, Digital Image Processing, San Diego, Aug. 2006. [7] T. Tran, L.Liu and P. Topiwala, “Performance comparison of leading image codecs: H.264/AVC intra, JPEG 2000, and Microsoft HD photo,” Proc. SPIE Int’l Symposium, Digital Image Processing, San Diego, Sept. 2007. [8] D. Marpe, T.Weigand and G. Sullivan, “The H.264/MPEG4 advanced video coding standards and its applications”, IEEE Communications Magazine, vol. 44, pp.134-143, Aug. 2006. [9] A. Skodras, C. Christopoulus and T. Ebrahimi, “The JPEG2000 still image compression standard,” IEEE Signal ProcessingMagazine, vol. 18, pp. 36-58, Sept. 2001. [10] D.S. Taubman and M.W. Marcellin, JPEG 2000: Image compression fundamentals, standards and practice, Kluwer academic publishers, 2001. [11] W.B. Pennebaker and J.L. Mitchell, JPEG: Still image data compression standard, Kluwer academic publishers, 2003. [12] D. Marpe, V. George, and T.Weigand, “Performance comparison of intra-only H.264/AVC HP and JPEG 2000 for a set of monochrome ISO/IEC test images”, JVT-M014, pp.18-22, Oct. 2004. [13] D. Marpe et.al, “Performance evaluation of motion JPEG2000 in comparison with H.264 / operated in intra-coding mode”, Proc. SPIE, vol. 5266, pp. 129-137, Feb. 2004. [14] Z. Xiong et.al, “A comparative study of DCT- and wavelet-based image coding,” IEEE Trans. on Circuits and Systems for Video Tech., vol.9, pp. 692-695, Aug. 1999. [15] G. K. Wallace, “The JPEG still picture compression standard,” Communication of the ACM, vol. 34, pp. 31-44, April 1991. [16] G. J. Sullivan, “ ISO/IEC 29199-2 (JpegDI part 2 JPEG XR image coding – Specification),” ISO/IEC JTC 1/SC 29/WG1 N 4492, Dec 2007

39. References (contd.) [17] H.264/AVC reference software (JM 13.2) Website: http://iphome.hhi.de/suehring/tml/download/ [18] JPEG reference software website: ftp://ftp.simtel.net/pub/simtelnet/msdos/graphics/jpegsr6.zip [19] Microsoft HD photo specification: http://www.microsoft.com/whdc/xps/wmphotoeula.mspx [20] JPEG2000 latest reference software (Jasper Version 1.900.0) Website: http://www.ece.ubc.ca/mdadams/jasper [21] JPEG-LS reference software website http://www.hpl.hp.com/loco/ [22] M.D. Adams, “JasPer software reference manual (Version 1.900.0),” ISO/IEC JTC 1/SC 29/WG 1 N 2415, Dec. 2007. [23] M.D. Adams and F. Kossentini, “Jasper: A software-based JPEG-2000 codec implementation,” in Proc. of IEEE Int. Conf. Image Processing, vol.2, pp 53-56, Vancouver, BC, Canada, Oct. 2000. [24] M. J. Weinberger, G. Seroussi, and G. Sapiro, “LOCO-I: A low complexity, context-based, lossless image compression algorithm”, Hewlett-Packard Laboratories, Palo Alto, CA. [25] M.J. Weinberger, G. Seroussi and G. Sapiro, “The LOCO-I lossless image compression algorithm: principles and standardization into JPEG-LS”, IEEE Trans. Image Processing, vol. 9, pp. 1309-1324, Aug.2000. [26] Ibid, “LOCO-I A low complexity context-based, lossless image compression algorithm”, Proc. 1996, pp.140-149, Snowbird, Utah, Mar. 1996. [27] K. Sayood, “Introduction to Data Compression”, Third Edition, Morgan Kaufmann Publishers, 2006. [28] M.Ghanbari, “Standard Codecs: Image Compression to Advanced Video Coding”. IEE, London, UK, 2003. [29] Z. Wang and A. C. Bovik, “Modern image quality assessment”, Morgan and Claypool Publishers, 2006. [30] Special Issue on JPEG-2000, Signal Processing: Image Communication, vol. 17, pp. 1-144, Jan 2002. [31] A. Stoica, C. Vertan, and C. Fernandez-Maloigne, “Objective and subjective color image quality evaluation for JPEG 2000- compressed images,” IEEEInt’l Symposium on Signals, Circuits and Systems, vol. 1, pp. 137 – 140, July 2003. [32] J. J. Hwang and S. G. Cho, “Proposal for objective distortion metrics for AIC standardization”, ISO/IEC JTC 1/SC 29/WG 1 N4548, Mar 2008. [33] H. R. Wu and K. R. Rao, “Digital video image quality and perceptual coding,” Boca Raton, FL: Taylor and Francis, 2006. [34] Test images found in: http://www.hlevkin.com/default.html#testimages [35] Information collected for various topics included in the material: www-ee.uta.edu/dip [36] Zhengbing Zhang, Radhika Veerla, K.R.Rao, “A modified advanced image coding”, to appear in Proceedings of CANS’ 2008, Romania, Nov. 8-10, 2008.

40. Thank you