1 / 35

Overview of Multi-view Video Coding

Overview of Multi-view Video Coding. Yo-Sung Ho; Kwan-Jung Oh; Systems, Signals and Image Processing, 2007 and 6th EURASIP Conference focused on Speech and Image Processing, Multimedia Communications and Services. 14th International Workshop on. Outline. Introduction

morrison
Download Presentation

Overview of Multi-view Video Coding

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Overview of Multi-view Video Coding Yo-Sung Ho; Kwan-Jung Oh;Systems, Signals and Image Processing, 2007 and 6th EURASIP Conference focused on Speech and Image Processing, Multimedia Communications and Services. 14th International Workshop on

  2. Outline • Introduction • Applications of Multi-view Video • Requirements for Multi-view Video Coding • Test Data Sets and Test Conditions • Joint Multi-view Video Model (JMVM) • Conclusion

  3. Introduction

  4. Introduction • Multimedia Demands • holography • two-view stereoscopic system with special glasses • multi-view video • Multi-view Video: • FVV, FVT, 3DTV • What is multi-view video? • Why we need multi-view video coding (MVC)? • MVC has been studied in the past. • MVP, MCP, DCP, MPEG4 MAC, H.263/4

  5. Application of MVC

  6. Application of MVC • Free Viewpoint Television (FTV) • Three-dimensional TV (3DTV) • Immersive Teleconference

  7. FTV • What is FTV (Free Viewpoint Television)? • Application of FTV: • Entertainment • Education • Sightseeing • Surveillance • Archive

  8. FTV

  9. 3DTV • What is 3DTV? • Interaction may not be required • To broadcast on 3DTV

  10. 3DTV

  11. 3DTV • Capture by various types of multiple cameras • 1D parallel • 2D parallel • 1D arc • …etc. • Intermediate view reconstruction (IVR)

  12. Immersive Teleconference • What is immersive teleconference? • Interaction

  13. Requirements for Multi-view Video Coding

  14. Requirements for Multi-view Video Coding • Requirements for multi-view video coding: • Compression related requirements • System support related requirements

  15. Compression Related Requirements • Compression efficiency • View scalability • Free viewpoint scalability • Spatial/Temporal/SNR scalability • Backward compatibility • Resource consumption • Low delay • Robustness

  16. Compression Related Requirements • Resolution, bit depth, chroma sampling format • Picture quality among views • Temporal random access • View random access • Spatial random access • Resource management • Parallel processing

  17. System Support Related Requirements • Synchronization • View generation • Non-planar imaging and display systems • Camera parameters

  18. Test Data Sets and Test Conditions

  19. Test Data Sets and Test Conditions

  20. Test Data Sets and Test Conditions

  21. Joint Multi-view Video Model

  22. Joint Multi-view Video Model

  23. Random Access • GGOP contains frames. • For accessing any frame within a GGOP, we have to decode maximum number of frames. • b4(S5/T7), following 18 referencing frames.

  24. Time-first coding order

  25. Encoder Complexity • Minimum decoded picture buffer (DPB) • EX: GOP_length=16, number_of_views=8, the DPB size = 42 • MVC codec will have the same coding delay as single view video coding since time-first coding is mandated.

  26. GOP structures and view prediction structure

  27. GOP structures and view prediction structure

  28. GOP structures and view prediction structure

  29. Illumination compensation • ICA MC: illumination change-adaptive motion compensation • Macroblocks(MB) mode in h.264/MPEG-4 AVC: Inter 16*16 mode, Direct 16*16 mode (include B_Skip), and P_Skip mode • DVIC: difference value of illumination change • ICA ME: illumination change-adaptive motion estimation

  30. SAD calculation for the motion estimation of S*T blocks: • In order to compensate the illumination change

  31. 1 • 1 • Illumination compensated residual signal

  32. Other Technical Issues • View-temporal prediction structure • Single video v.s. Multi-view video • Three main coding structures • Encode multiple video sequences separately • Utilizes inter-view correlation only • Utilizes both temporal and inter-view correlation • View interpolation prediction • Decoder side disparity estimation • Computing depth at encoder side and transmitting this to the decoder • Motion/Disparity vector coding • Highly correlated each other

  33. Conclusion

  34. Conclusion • The multi-view video includes multi-viewpoint video sequence captured by several cameras at the same time. • Compress multi-view video efficiently • MPEG and JVT are leading the standardization of MVC.

More Related