210 likes | 419 Views
Depth Image-Based Temporal Error Concealment for 3-D Video Transmission. Yunqiang Liu, Jin Wang, and Huanhuan Zhang IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 20, NO. 4, APRIL 2010 Professor: Jar - Ferr Yang Presenter: Jen - Hung Yeh. Outline. Introduction
E N D
Depth Image-Based Temporal Error Concealment for3-D Video Transmission Yunqiang Liu, Jin Wang, and Huanhuan Zhang IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 20, NO. 4, APRIL 2010 Professor: Jar - Ferr Yang Presenter: Jen - Hung Yeh
Outline • Introduction • Depth Image-Based Temporal Error Concealment • A. Error Concealment for Depth Stream • B. Error Concealment for 2-D Video Stream • Homogeneous MB • Boundary MB • Simulation Results • Conclusion
Introduction(1/2) • 3-D television (3DTV) system has recently received increasing attention and is believed to be the next logical development. • In DIBR-based 3-D video application, it requires transmitting 2-D video and its corresponding depth information. • In the video transmission over unreliable channels such as wireless or internet, the transmission errors, such as packet losses and bit errors, are inevitable.
Introduction(2/2) • Therefore , there is a need for good error concealment (EC) algorithms to compensate the large drop in received video quality. • It explores the correlation between the 2-D video stream and the depth stream and proposes a novel temporal error concealment approach by taking advantage of the depth information.
Depth Image-Based Temporal Error Concealment • A. Error Concealment for Depth Stream • B. Error Concealment for 2-D Video Stream
Error Concealment for Depth Stream(1/2) • The MV from the corresponding 2-D frame can be taken as the recovered MV for the corrupted MB in the depth stream. • In case of the error occurring at the same location in the depth stream and the video stream, the DMVE algorithm is performed in the depth sequence to recover the lost MV.
Error Concealment for Depth Stream(2/2) • The depth map does not contain any texture information; moreover, it cannot distinguish the different objects at the same distance with the camera, even if they have relative motion. • It is not appropriate to take directly the MV of depth map as the recovered MV for 2-D video.
Error Concealment for 2-D Video Stream • In the following steps, we will onlyconsider the situation that the error occurs in the 2-D video. 8-category histogram for the depth value The lost MBs Belong to two categories & The difference between their average depth values>T1 NO YES Homogeneous MB Boundary MB
Homogeneous MB Find MV candidates of the MBs around the The MV of the corresponding MB in depth map Filter these MVs candidate The referenced depth block should coincide with that of the . MVi <T2 YES NO The MV with minimum discard
Homogeneous MB • (SAD represents sum of absolute differences) is calculated to measure the similarity between the two depth MBs • For 3-D video , we introduce a new matching criterion with consideration of the corresponding depth, which is defined as
Boundary MB • It concludes four steps as follows: • Step 1: MB Segmentation: • Step 2: MV Candidates’ Initialization: • Step 3: MV Selection: • Step 4: Motion Compensation:
Step 1: MB Segmentation: The lost MB The pixels with the large depth value NO YES Background • Foreground We take the foreground part as an example to describe the approach.
Step 2: MV Candidates’ Initialization: Foreground Background • If the depth value of the grid is close to the average depth of the foreground area, both the MV of the grid in 2-D video and depth map are chosen as the MV candidate for the EC of foreground area. • And the collocated MV in the reference frame is also taken as the MV candidate. The neighbors of the foreground area Strong correlation with the lost foreground part
Step 3: MV Selection: • Similar with the recovery of homogeneous MB, the matching criterion considers both the neighbor information and the corresponding depth. • The matching criterion is defined as:
Step 3: MV Selection: • measures the temporal correlation of the 2-D video , which is defined as follows: • measures the similarity between the current depth MB and its referenced MB , which is defined as:
Step 4: Motion Compensation: • The foreground area in the lost MB is recovered using the corresponding area in the reference frame according the recovered MV. The background area is recovered with a similar process.
Simulation Results(1/4) depth map 2-D video
Simulation Results(2/4) Fig. 2. Comparison of PSNR for each frame with 5% packet loss. (a) Little Girl. (b) Mobile Phone. (c) Orbi.
Simulation Results(3/4) Homogeneous MB Boundary MB
Simulation Results(4/4) Homogeneous MB Boundary MB
Conclusion • A depth image-based error concealment approach exploits the correlations between the 2-D video and its corresponding depth map to recover the lost blocks. • This strategy makes it preserve the temporal and spatial consistency on both 2-D video and depth information.