1 / 18

Yu Liu 1 , Feng Wu 2 and King Ngi Ngan 1

3D Object-based Scalable Wavelet Video Coding with Boundary Effect Suppression. Yu Liu 1 , Feng Wu 2 and King Ngi Ngan 1 1 Department of Electronic Engineering, The Chinese University of Hong Kong 2 Microsoft Research Asia, Beijing, 100080, China ISCAS 2007, New Orleans, USA, 27-30 May 2007.

aulani
Download Presentation

Yu Liu 1 , Feng Wu 2 and King Ngi Ngan 1

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 3D Object-based Scalable Wavelet Video Coding with Boundary Effect Suppression Yu Liu1, Feng Wu2 and King Ngi Ngan1 1Department of Electronic Engineering, The Chinese University of Hong Kong 2Microsoft Research Asia, Beijing, 100080, China ISCAS 2007, New Orleans, USA, 27-30 May 2007

  2. Outline • Introduction • Motivation • Object-based Motion Threading Using The Lifting Structure • 3D SA-DWT Via Lifting • Experimental Results • Conclusion

  3. Introduction • Standard video coding paradigm • Frame-based: MPEG-1/2, H.261/263/264 V.S. • Object-based: MPEG-4/7 • Advantages: • enables the accessibility and manipulability of object within a video sequence • allows the structure of video content to survive the process of acquisition, editing and distribution • some important features still need more efforts • scalability, • coding efficiency, • etc.,. DCT/ ICT

  4. Introduction • Alternative to traditional video coding standard —wavelet-based video coding • full spatio-temporal-quality scalability • non-redundant 3D subband decomposition • comparable with H.264-based JSVM scheme • MCTF-based 3D scalable wavelet video coding • Motion Compensated Temporal Filtering (MCTF) • motion compensation is incorporated into temporal wavelet transform • Spatial domain MCTF (SD-MCTF), T+2D • which performs MCTF prior to spatial wavelet transform • In-band MCTF (IB-MCTF), 2D+T • which performs MCTF after spatial wavelet transform

  5. Motivation • 3D Object-based Scalable Wavelet Video Coding • Object-based motion threading using the lifting structure • extends the motion threading technique from the frame-based coding to the object-based coding • attracted by the unique advantages of the object-based coding that do not exist in other coding schemes • 3D SA-DWT via Lifting • to suppress the boundary effects • which exist in spatial and temporal wavelet transforms due to the manner of object-based coding

  6. Object-based Motion Threading Using The Lifting Structure • Previous motion threading techniques [Xu01, Luo04] • Motion Threading (MTh)[xu01]: An efficient implementation of MCTF • employs long wavelet filters to exploit the long-term correlation across frames along motion trajectory • some limitations, such as many-to-one mapping, terminating and non-referred pixels • Advanced Motion Threading [Luo04] • many-to-one mapping: • well solved • terminating and non-referred • not well solved

  7. Object-based Motion Threading Using The Lifting Structure • Why object-based motion threading? • object-based coding with arbitrary regions of support • more accurate motion trajectory in the video object • Object-based MTh using the lifting structure • aims to reduce the boundary effects of artificially terminating/ emerging threads in the previous MTh.

  8. Object-based Motion Threading Using The Lifting Structure • Object-based motion threading • For the thread which is originally terminated in many-to-one mapping • it still can continue temporal filtering without being terminated by using lifting structure

  9. Object-based Motion Threading Using The Lifting Structure • Object-based motion threading • For the terminating and non-referred pixels • originally indicate the boundaries of the motion threads • using the symmetric extension over the motion thread boundary • instead of assigning the motion vector from adjacent motion thread shape adaptive temporal wavelet transform

  10. 3D SA-DWT Via Lifting • Boundaries • Temporal domain: the boundary of motion thread • Spatial domain: the boundary of arbitrarily shaped signals • Approaches for transforming 2D arbitrarily shaped signals to wavelet domain • Extension over the boundaries [Li00] • Boundary filters [Herley95] • Projections onto convex sets [You87] • Biorthogonal symmetric filters with symmetric extensions over the boundaries [Li00] • focus on the texture coding of 2D still image objects • not for the coding of 3D video objects

  11. 3D SA-DWT Via Lifting • Efforts in developing coding techniques for arbitrarily shaped 3D video objects • Opportunity provided by object-based motion threading technique • align a series of video object planes to form a 3D video object • 3D SA-DWT = (1D Shape Adaptive Temporal Wavelet Transform + 2D Shape Adaptive Spatial Wavelet Transform) • SD-MCTF (T+2D) • first apply 1D-SA-DWT in the temporal thread segment • then apply 2D-SA-DWT in the spatial signal segment • IB-MCTF (2D+T) • or vice versa

  12. 3D SA-DWT Via Lifting Example of aligning a series of video object planes to form a 3D video object symmetric extensions over the boundaries Temporal Thread Segment Spatial Signal Segment Biorthogonal 9/7 lifting wavelet Biorthogonal 5/3 lifting wavelet

  13. 3D SA-DWT Via Lifting • 3D multi-spatio-temporal resolution video object pyramid • non-redundant decomposition structure • full spatio-temporal-quality scalability for object-based coding • 3D EBCOT (Embedded Block Coding with Optimal Truncation) [Xiong05] • a bit-plane entropy coding scheme which encodes the wavelet coefficients coming from 3D SA-DWT • an extension of 2D EBCOT in the JPEG2000 image compression standard • Extension to object-based coding: 3D OB-EBCOT • Only samples inside the object are encoded by 3D object-based EBCOT • If neighboring samples fall outside the object, their value is set to zero in context modeling

  14. Experimental Results The test object sequences, (a) Akiyo, (b) Bream, (c) Children, (d) Coastguard, and (d) Foreman (a) (b) (c) (d) (e) • The proposed object-based scalable wavelet video coder (3D OB-EBCOT) includes two versions: • SD-MCTF-based: 3D OB-EBCOT (SD), • IB-MCTF-based : 3D OB-EBCOT (IB).

  15. Experimental Results • Table I lists the average PSNR (in dB) results given by different coders on the four foreground object sequences. • The comparisons of 3D OB-EBCOT with MPEG-4 in terms of PSNR plots (in dB) for the foreground objects, Children and Foreman.

  16. Experimental Results • Coding performance comparison of 3D OB-EBCOT with different coding schemes The 3D OB-EBCOT coder is an object-based scalable video coder and is able to achieve all of these different target bit rates in a single bitstream.

  17. Conclusion • 3D Object-based Scalable Wavelet Video Coder • object-based motion threading with lifting structure • to improve the previous motion threading techniques for object-based coding • 3D SA-DWT via lifting in a unified framework • to suppress the boundary effect problem existing simultaneously in spatial and temporal transforms of object-based coding

  18. Thank you for your attention!Q&A

More Related