From 2D Extrapolation to 1D Interpolation: Content-adaptive Image Bit-depth Expansion

1. From 2D Extrapolation to 1D Interpolation:Content-adaptive Image Bit-depth Expansion PengfeiWan, Oscar C. Au, KetanTang, YuanfangGuo, Lu Fang Department of Electronic and Computer Engineering The Hong Kong University of Science and Technology

2. OUTLINE • Introduction to Bit-depth Expansion(BDE) • Existing Methods: A General Framework • Proposed Method: LMM Reconstruction • Experiments • Conclusions

3. Introduction: What is BDE? ‘original’ image pixel (high bit-depth, HBD) Truncation/quantization contouring artifacts input image pixel (low bit-depth, LBD) 1 1 1 1 1 1 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 ? 0 1 0 ? 0 0 0 0 0 ? 0 ? 1 1 A naive method: zero padding Bit-depth expansion output image pixel (HBD) Estimate the lost LSBs !

4. Introduction: What is BDE? ground-truth 8-bit image output of proposed method output of zero padding Bit-depth expansion from 4 bit to 8 bit for RGB image.

5. OUTLINE • Introduction to Bit-depth Expansion(BDE) • Existing Methods: A General Framework • Proposed Method: LMM Reconstruction • Experiments • Conclusions

6. Existing Methods: A General Framework • Filtering-based methods: e.g. Daly’s de-contouring method[8] • Pros: linear operation, fast implementation • Cons: local-window filtering is unsuitable for removing contours • Non-linear methods: e.g. Cheng’s method[9] • Pros: operation is spatially varying, better performance • Cons: large error in local minima and maxima regions

7. OUTLINE • Introduction to Bit-depth Expansion(BDE) • Existing Methods: A General Framework • Proposed Method: LMM Reconstruction • Experiments • Conclusions

8. Proposed Method: Flowchart

9. Proposed Method S1: Neighborhood Flooding Flooding: exchange locational information with immediate neighbors until convergence. Purpose: 2D interpolation problem in non-LMM regions becomes 1D interpolation.

10. Proposed Method S2: LMM Regions Detection • Detect LMM regions by thresholdingDM/UM • LMM map refinement by morphological operations

11. Proposed Method S3: Marking Skeletons for LMM With the help of skeletons, 2D extrapolation problem in LMM regions becomes 1D interpolation.

12. Proposed Method S3: Marking Skeletons for LMM • LMM map with skeletons

13. Proposed Method S4: LSBs Reconstruction Where

14. OUTLINE • Introduction to Bit-depth Expansion(BDE) • Existing Methods: A General Framework • Proposed Method: LMM Reconstruction • Experiments • Conclusions

15. Experiments • Gains 0.63, 3.33, 4.01 dB in PSNR and 0.004, 0.020, 0.064 in SSIM over the best of remaining methods • for 6-bit, 4-bit, 2-bit to 8-bit expansion respectively. • Compared with zero padding, the gain is 6.49, 9.35, 6.32 dB in PSNR and 0.011, 0.138, 0.282 in SSIM.

16. Experiments Daly’s Method Zero padding Proposed Method Cheng’s Method Error maps of selected BDE methods

17. OUTLINE • Introduction to Bit-depth Expansion(BDE) • Existing Methods: A General Framework • Proposed Method: LMM Reconstruction • Experiments • Conclusions

18. Conclusions • Proposed LMM reconstruction method is general. • it can be applied for any 2D surface reconstruction scenarios. • it can be used for generating HDR images from 8-bit images. • Key part of proposed method: skeleton marking • it can guarantee thin but complete skeletons for arbitrary LMM regions. • Significantly over-performs existing BDE methods. • appealing reconstruction performance even for 2bit-8bit expansion. • Time-efficient parallel implementations are possible.

19. [1] R. Mantiuk, G. Krawczyk, K. Myszkowski, and H. Seidel,“Perception-motivated high dynamic range video encoding,” in ACM SIGGRAPH, 2004, pp. 733–741. [2] P. E. Debevec and J. Malik, “Recovering high dynamicrange radiance maps from photographs,” in ACM SIG-GRAPH 2008 classes, 2008, pp. 31:1–31:10. [3] F. Banterle, P. Ledda, K. Debattista, and A. Chalmers, “Inverse tone mapping,” in GRAPHITE ’06, 2006, pp. 349–356. [4] G.W. Larson, H. Rushmeier, and C. Piatko, “A visibility matching tone reproduction operator for high dynamic range scenes,” Visualization and Computer Graphics, IEEE Transactions on, vol. 3, no. 4, pp. 291–306, 1997. [5] R.Mantiuk, A. Efremov, K.Myszkowski, and H. Seidel, “Backward compatible high dynamic range mpeg video compression,” in ACM SIGGRAPH, 2006, pp. 713–723. [6] “Meeting report of the fourth meeting of the joint collaborative team on video coding, daegu, kr,” Joint Col-laborativeTeam on Video Coding, 20-28 January, 2011. [7] R. Ulichney and S. Cheung, “Pixel bit-depth increase by bit replication,” Color Imaging: Device Independent Color. Proc. of SPIE, vol. 3300, pp. 232–241, 1998. [8] S. Daly and X. Feng, “Decontouring: Prevention and removal of false contour artifacts,” in Proc. SPIE Hu- man Vision and Electronic Imaging IX, 2004, vol. 5292, [9] C. H. Cheng, O. C. Au, C. H. Liu, and K. Y. Yip, “Bitdepth expansion by contour region reconstruction,” in Proc. of IEEE Int. Sym. on Circuits and Systems, 2009, pp. 944–947. [10] C. H. Liu, O. C. Au, H.W.Wong, M. C. Kung, and S. C. Chao, “Bit-depth expansion by adaptive filter,” in Proc. of IEEE Int. Sym. on Circuits and Systems, 2008. [11] M. Winken, D. Marpe, H. Schwarz, and T. Wiegand, “Bit-depth scalable video coding,” in Proc. IEEE Int. Conf. Image Processing, 2007, vol. 1, pp. I–5–I–8. [12] F. Banterle, P. Ledda, K. Debattista, and A. Chalmers, “Expanding low dynamic range videos for high dynamic range applications,” in the 24th Spring Conference on Computer Graphics, 2010, pp. 33–41. [13] B. T. Phong, “Illumination for computer generated pictures, ”Commun.ACM, vol. 18, no. 6, pp. 311–317, June 1975. [14] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” Image Processing, IEEE Trans-actions on, vol. 13, no. 4, pp. 600–612, 2004.

20. Thank you ! Q&A