Multi-View Image Coding with Disparity-Compensated Lifted Wavelets

1. Multi-View Image Coding with Disparity-Compensated Lifted Wavelets Steven Lansel Nandhini Nandiwada Santhanam EE 398 – Image and Video Compression Winter 2006

2. Overview • Problem Overview • Disparity Vector Selection • Transform Explanation • Pareto-like Condition • Overall R-D Curve • Optimal Bit Allocation • Conclusions Multi-View Image Coding with Disparity-Compensated Lifted Wavelets

3. + + + + Problem Overview • Images are taken of a scene from multiple angles • Lifted Haar scheme is used to encode two images • Transform uses disparity vectors to match similar blocks in the two images • Optimal allocation of bits between disparity vectors and L/H images is desired SPIHT L - P U U-1 P-1 - H SPIHT Multi-View Image Coding with Disparity-Compensated Lifted Wavelets

4. Disparity Vector Selection • For each block in image B, the best block in image A is found • Disparity vectors for each block are chosen to minimize J=D+λR where D is the sum of squared errors and R is the bit rate for the vector • Bit rate for vectors follows Laplacian model Multi-View Image Coding with Disparity-Compensated Lifted Wavelets

5. Transform Explanation Input B Input A α is percent of connected pixels in A Output H Multi-View Image Coding with Disparity-Compensated Lifted Wavelets

6. Transform Explanation (2) Input B Input A For each pixel in L use the pixel in the same location in A and the first pixel in B that is mapped to that pixel in A. Output H Output L Not used for unconnected pixels Multi-View Image Coding with Disparity-Compensated Lifted Wavelets

7. Inverse Transform and Distortion Equations • Inverse transform equations: • Distortion equations for non-orthonormal transform: Not used for unconnected pixels Multi-View Image Coding with Disparity-Compensated Lifted Wavelets

8. Transform Characteristics based on Lambda Multi-View Image Coding with Disparity-Compensated Lifted Wavelets

9. Rate-Distortion Curves when Compressing L or H Image Multi-View Image Coding with Disparity-Compensated Lifted Wavelets

10. Optimal Bit Allocation using a Given Set of Disparity Vectors • Total Distortion: • Cost Function: • Pareto-like Solution: • Total Rate Constraint: Multi-View Image Coding with Disparity-Compensated Lifted Wavelets

11. R-D Dependence on Lambda for DVs Enlarged region from left plot Multi-View Image Coding with Disparity-Compensated Lifted Wavelets

12. Overall Rate-Distortion Curve Multi-View Image Coding with Disparity-Compensated Lifted Wavelets

13. Optimal Bit Allocation Multi-View Image Coding with Disparity-Compensated Lifted Wavelets

14. Conclusion • Disparity-compensation matches similar blocks in the two input images • Resulting transform is not orthonormal • Disparity vectors must be determined by finding the best λ for a given total rate • Allocation of bits between L and H images is achieved by satisfying weighted equation and total bit rate constraint Multi-View Image Coding with Disparity-Compensated Lifted Wavelets

15. References • Aditya Atul Mavlankar, “Robust Video Communication using Motion-Compensated Lifted 3-D Wavelet Coding,” Master’s Thesis, Technische Universität München Lehrstuhl für Kommunikationsnetze Fachgebiet Medientechnik, September 2004. • B. Pesquet-Popescu and V. Bottreau, "Three-Dimensional Lifting Schemes for Motion Compensated Video Compression," Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Salt Lake City, UT, May 2001, pp. 1793-1796. • Chuo-Ling Chang, Xiaoqing Zhu, Prashant Ramanathan, and Bernd Girod, "Inter-view Wavelet Compression of Light Fields with Disparity-Compensated Lifting," Proc. SPIE Visual Communications and Image Processing, VCIP-03, Lugano, Switzerland, July 2003, pp. 14-22. Multi-View Image Coding with Disparity-Compensated Lifted Wavelets