1 / 16

Byung Cheol Song Shin- Cheol Jeong Yanglim Choi

V ideo S uper -R esolution A lgorithm U sing B i - directional O verlapped B lock M otion C ompensation. Byung Cheol Song Shin- Cheol Jeong Yanglim Choi. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 21, NO. 3, MARCH 2011. o utline. Introduction

oni
Download Presentation

Byung Cheol Song Shin- Cheol Jeong Yanglim Choi

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Video Super-Resolution Algorithm Using Bi-directional Overlapped Block Motion Compensation ByungCheol Song Shin-CheolJeong Yanglim Choi IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 21, NO. 3, MARCH 2011

  2. outline • Introduction • Motion-compensated SR • Basic concept • Hierarchical motion estimation • Bi-directional OBMC • Result • Learning-based SR • Hybrid super-resolution • Experimental Results

  3. Introduction • For the last few decades, many image interpolation algorithms have been developed to display high quality scaled images on cutting-edge digital consumer application such as HDTV,DSC. • Traditional interpolation methods usually suffer from several types of visual degradation. • To overcome the problem, many algorithms are proposed. But there are many problem.(Ex. structurally weak against textures, computational complexity)

  4. Introduction • Brandi et al. presented an interesting SR approach. • They defined the so-called key frames (KF) that sparsely exist in a video sequence and have HR resolution. The remaining frames in the video sequence, i.e., non-key frames(NKF) had LR resolution • Brandi et al. took advantage of the fact that few KF (encoded at HR) may provide enough HF information to the up-scale NKF (encoded at LR) • However, it rarely found true motion information because it made use of the conventional full-search motion estimation. • This paper presents a hybrid SR algorithm where each LR patch is adaptively selected between a temporally super-resolved patch and a spatially super-resolved patch using adjacent HR KFs. • Temporally super-resolved patch  MSR • Spatially super-resolved patch  LSR

  5. Motion-compensated SR Basic concept • A target frame in LR video sequence is interpolated by using forward and backward HRkey-frame(KF). • The KF interval N can be constant or variable. • These paper employ cubic convolution for Up-Scaling.

  6. Motion-compensated SR Hierarchical motion estimation • First, the MV for overlapping M×Mmatching block is searched by using a rate-constrained ME. • The conventional rate-constrained ME find the best v for each matching block by minimizing the rate as well as distortion • In order to maximize the ME performance andto concurrently reduce the computational burden, we adopt arate-constrained fast full search algorithm presented in [15]. λ : Lagrange multipier

  7. Motion-compensated SR Bi-directional OBMC • Assume that MVsbetween up-scaled LR frames are statistically very similar to those between corresponding HR frame. • Replace the unknown MVs for HR frames with the MVs obtains from the up-scaled LR frames. • Employ BOBMC based on bi-directional MVs .the BOBMC is performed on a 4×4block basis. • Each MV may be a forward or backward MV. The direction is determined according to SAD

  8. Motion-compensated SR Result Fan’s [12] Brandi’s [14] original MSR

  9. Motion-compensated SR Drawback • It cannot often provide acceptable visual quality due to non-translational motion, occlusion, inaccurate motion estimation and limited motion search range. • When such temporal motion compensation does not work well, we employ a learning-based SR in order to avoid the degradation of visual quality.

  10. Learning-based SR • On-the-fly learning stage • Construct a trained dictionary using adjacent LR KFs and HR KFs. • Inference stage • The best-matched patch for the is exhaustively searched from the dictionary.(the patch having minimum SAD is chosen as best-match) • Produce a spatially super-resolved HR patch from Seek optimal K-means clustering Produce Laplacian patch Optimal dictionary LMS algorithm

  11. Compare MSR & LSR MSR LSR Background is well- motion-compensated Better Quality Seldom be well- motion-compensated

  12. Hybrid super-resolution • The boundary between the temporally super-resolved patches and spatially super-resolved patches, we can observe blocking artifact because they are derived from different frames. • We apply a simple smoothing filter to the boundary pixels

  13. Experimental Results • PSNR performance of the MSR according to N Without/with little global motion  N seldom affects With large global motion

  14. Experimental Results BLI Bi-cubic NEDI[1] Fan’s[12] Barsiu’s N = 30 M =16 L = 4 Search range =±64 MSR Hybrid SR Brandi’s[14] [1] X. Li and M. T. Orchard, “New edge-directed interpolation,” IEEE Trans. Image Process., vol. 10, no. 10, pp. 1521–1527, Oct. 2001. [12] W. Fan and D. Y. Yeung, “Image hallucination using neighbor embedding over visual primitive manifolds,” in Proc. CVPR, 2007, pp. 1–7. [14] F. Brandi, R. Queiroz, and D. Mukherjee, “Super-resolution of video using key frames and motion estimation,” in Proc. IEEE ICIP, Oct.2008, pp. 321–324.

  15. Experimental Results N = 30 M =16 L = 4 Search range =±64

  16. Experimental Results N = 30 M =16 L = 4 Search range =±64

More Related