Jin Yong Lee Ho Chen Wey Du Sik Park IEEE Transactions on CSVT 2011

1. A Fast and Efficient Multi-View Depth Image Coding Method Based on Temporal and Inter-View Correlations of Texture Images Jin Yong Lee Ho Chen Wey Du Sik Park IEEE Transactions on CSVT 2011

2. Outline • Introduction • Proposed Method • Temporal Correlation • Texture and Depth View Synthesis • Inter-View Correlation • Evaluations • Coding Performance • Encoding Complexity Analysis • Subjective Quality Assessment

3. Introduction • Encode video information of each view individually with the H.264/AVC • A multi-view video coding structure with hierarchical B pictures • Multi-view video plus depth

4. Outline • Introduction • Proposed Method • Temporal Correlation • Texture and Depth View Synthesis • Inter-View Correlation • Evaluations • Coding Performance • Encoding Complexity Analysis • Subjective Quality Assessment

5. Temporal Correlation • Stationary regions have similar pixel values for successive frames • Use sum of the squared difference(SSD) to measure the temporal correlation • If SSD => Strongly Correlated I P Texture images Depth images

6. Temporal Correlation B I P Texture images Depth images

7. Temporal Correlation

8. Outline • Introduction • Proposed Method • Temporal Correlation • Texture and Depth View Synthesis • Inter-View Correlation • Evaluations • Coding Performance • Encoding Complexity Analysis • Subjective Quality Assessment

9. Texture and Depth View Synthesis • Synthesis the next depth view by 3D image warping

10. Texture and Depth View Synthesis • The pixel position(x,y) at the reference frame can be projected into a 3D point (u,v,w) • The corresponding pixel location of the virtual image ()

11. Texture and Depth View Synthesis • Some pixels in the synthesized image are missing or undefined • Occlusion • Pixel position quantization • Only consider the neighboring pixelsaround the hole • If a view is warped to the right view position, the hole is filled with its left pixel

12. Texture and Depth View Synthesis

13. Outline • Introduction • Proposed Method • Temporal Correlation • Texture and Depth View Synthesis • Inter-View Correlation • Evaluations • Coding Performance • Encoding Complexity Analysis • Subjective Quality Assessment

14. Inter-View Correlation • A depth image of I-view(), a texture image of I-view() and P-view() are encoded • Synthesized a virtual texture image from the reconstructed and • If SSD => skip the block P-view I-view Synthesized Image Texture images Depth images depth texture

15. Inter-View Correlation

16. Evaluations • Coding performance • Encoding complexity analysis • Subjective Quality Assessment

17. Coding performance • Simulation conditions • Test sequences

18. Coding performance BDBR : Average bit-rate difference in % over the whole range of PSNR BDPR :Average PSNR difference in dB over the whole range of bit-rates Balloons Newspaper Kendo Champagne tower Book Arrival Pantomime

19. Coding Performance Book Arrival Balloons Kendo Pantomime Champane tower Newspaper

20. Encoding Complexity Analysis • ETR and RM indicate the total encoding time reduction and the reduced number of the modes performing the RD optimization in percentage • SB and VS represent the proportion of skipped macro blocks in depth image and the time required for the view synthesis process

21. Encoding Complexity Analysis

22. Subjective Quality Assessment • 15 professional subjects • Two stereoscopic view pairs were rendered using the depth images decoded by the original method and the proposed method respectively

23. Subjective Quality Assessment • Y-axis indicates a differential score that subtract a score of the original method from the proposed method