Wen-Hsiao Peng Chun-Chi Chen

An Inter-Frame Prediction Technique Combining Template Matching Prediction and Block Motion Compensation for High Efﬁciency Video Coding Circuits and Systems for Video Technology, 2013 IEEE Transactions on Wen-Hsiao Peng Chun-Chi Chen

Outline • Introduction • Background • Bi-prediction Combining TMP and BMC • Analysis LS and LMS • Experiment Results • Conclusion

Introduction • Inter prediction combines MVs from • TMP • BMC for Overlapped Block Motion Compensation. • Prediction performance of OBMC close to that of bi-prediction. • without having to signal the template MV

Introduction • TMP generally outperforms SKIP prediction. • TMP is inferior to block-based motion compensation. • Another MV is required to best complement the template MV.

Introduction • A key issue in video coders with motion-compensated prediction is how to trade off effectively between • accuracy of the motion field representation • required overhead • Based on HEVC version 6.0 • Achieve the bitrate reduction.

Outline • Introduction • Background • Template Matching Prediction • Block Motion Compensation • SKIP and Merge-SKIP • Signal Model • Prediction Error Surface • Bi-prediction Combining TMP and BMC • Analysis LS and LMS • Experiment Results • Conclusion

Template Matching Prediction • Obtains the MV at a current pixel by finding, in the reference frames, the best match for a template region composed of its surrounding reconstructed pixels.

Block Motion Compensation • The frames are partitioned in blocks of pixels and each block is predicted from a block of equal size in the reference frame.

Comparsion True motion BMC TMP

SKIP and Merge-SKIP • SKIP • H.264/AVC • Merge-SKIP • Weighted sum

Signal Model • Tao et al [19] • . • Zheng et al [24] • . [19] B. Tao and M. T. Orchard, “A parametric solution for optimal overlapped block motion compensation,” IEEE Trans. on Image Processing, vol. 10, no. 3, pp. 341–350, Mar. 2001. [24] W. Zheng, Y. Shishikui, M. Naemura, Y. Kanatsugu, and S. Itoh,“Analysis of space-dependent characteristics of motion- compensated frame differences based on a statistical motion distribution model,” IEEE Trans. on Image Processing, vol. 11, no. 4, pp. 377–386, Apr. 2002.

Signal Model • Mean-sqaured prediction error • . • Tao et al [19] • . • Zheng et al [24] • . [19] B. Tao and M. T. Orchard, “A parametric solution for optimal overlapped block motion compensation,” IEEE Trans. on Image Processing, vol. 10, no. 3, pp. 341–350, Mar. 2001. [24] W. Zheng, Y. Shishikui, M. Naemura, Y. Kanatsugu, and S. Itoh,“Analysis of space-dependent characteristics of motion- compensated frame differences based on a statistical motion distribution model,” IEEE Trans. on Image Processing, vol. 11, no. 4, pp. 377–386, Apr. 2002.

Signal Model • Block MV, vb , and block center, sc • vb = v(sc) • . • Template MV, vt, and template center, st • vt= v(st) • .

Signal Model Tao’s model Zheng’s model

Prediction Error Surface

Prediction Performance Comparsion • Encoding 50 frames

Outline • Introduction • Background • Bi-prediction Combining TMP and BMC • Overlapped Block Motion Compensation • Least Square Solution • Least Mean-Square Solution • Analysis LS and LMS • Experiment Results • Conclusion

Bi-prediction Combining TMP and BMC • Predictor is computed as a weighted average of two reference blocks. • Template MV, vt • Block MV, vb • TMP can better compensate for the movement of the top-left area of a prediction block. • BMC is thus aimed at reducing further the prediction residual in the remaining area.

Overlapped Block Motion Compensation • The weighting can be pixel adaptive. • . • ω is indicating their likelihood • The problem is to determine the OBMC weights so that the resulting predictor would produce a minimal residual. • .

Overlapped Block Motion Compensation • How to minimize the prediction residual by a suitable choice of the block MV and OBMC weights. • . • The approaches to solve the problem • Least Squares Approach • Least Mean-Square Approach

Least Square Solution • Rely on an iterative algorithm to solve for the optimal weights. • Estimating Block MVs : • . • Adapting OBMC Weights : • . • It’s convergence to a possibly local minimum is usually between 5 to 10 iterations.

Least Mean-Square Solution • Introduce statistical signal models. • Given that every block is to be predicted using OBMC based on two MVs • defaulting to the true MV • MV sampling the motion field at some point sb • determine a set of OBMC weights

Least Mean-Square Solution • Transform the problem of minimizing ξinto that of minimizing its expected value E[ξ]. • . • Fixing sbdetermine the : • . • Find the optimal sbthat yields the global minimum : • .

Analysis LS and LMS • . indicates the likelihood of vtbeing the true motion of a pixel at s relative to the other hypothesis vb. • Template MV is not as reliable for compensating pixels in the upper-left area as predicted by the theoretical results. Tao’s model Zheng’s model LS solution

Analysis LS and LMS • So, we would expect to drop to zero (or, equivalently,  to increase to unity) without amendment with amendment Multiple reference frames

Results • Reductions in mean-square error

Experiment Results Low-Delay B High Efficiency Random Access High Efficiency Random Access Main Low-Delay B Main

Experiment Results

Conclusion • We proposed a bi-prediction scheme that combines BMC and TMP predictors through OBMC. • TMP is inferior to BMC, but is, in general, superior to SKIP prediction. • The data dependency complicates the pipeline design and hinders parallel processing. • The proposed method restricted the use of TB-mode to 2Nx2N PUs only.

Wen-Hsiao Peng Chun-Chi Chen