Fine-granular Motion Matching for Inter-view Motion Skip Mode in Multi-view Video Coding

1. Fine-granular Motion Matching for Inter-view Motion Skip Mode in Multi-view Video Coding Haitao Yanh, Yilin Chang, Junyan Huo CSVT

2. Outline • Motivation • Introduction of motion skip mode • Methodology • Experimental results • Conclusion

3. Motivation • Global disparity can not well describe the inter-view corresponding relations in different image regions. • Fine granularity is introduced to obtain mare accurate motion information. Akko&Kayo, 640*480, 30fps

4. Introduction – Motion Skip Mode • Use global disparity vector to search for the corresponding macroblock. • Motion information is derived from the corresponding MB in the picture of neighboring view.

5. Introduction – Motion Skip Mode (cont’d) • Assume there is one inter-view reference picture and one temporal reference picture: • Iv,t : the picture in view v at time t • Bv,t : a 16×16 block in Iv,t • ,where V and Vref denote the coding view and the reference view • , where T and Tref denote the time instance of the coding picture and the reference picture

6. Introduction – Motion Skip Mode (cont’d) • Use mean absolute difference(MAD) to evaluate the matching error: • h, w: height and width of coding picture • accuracy: 16-pel w w (x, y) (x, y) (x, y) h h reference frame coding frame

7. Fine-granular motion matching • In H.264/AVC, 8×8 block is the basic unit to perform MC. • To estimate 8-pel accuracy global disparity vector between the coding picture and the inter-view reference picture. • DG: global disparity vector • XG, YG: x and y component of global disparity vector • S : search range with 8-pel accuracy • where

8. Fine-granular motion matching (cont’d) • After the estimation of DG, we need to find the optimal disparity of the coding macroblock BV,T. • A search window of (4×8-pel) ×(4×8-pel) centers at (x+xG,y+yG) • Each × sign indicates a search point. 8*8 MB 16*16 MB

9. Fine-granular motion matching (cont’d) • The 16×16 block centers at each search point, (x+xG+Δxi, y+yG+Δyi) for the ith search point. • Each 16×16 block is composed of four 8×8 blocks, {bi,j|j=1,2,3,4}. • Each 8×8 block bi,j has its own motion information mi,j, Mi ={mi,j |j=1,2,3,4} 8*8 MB 16*16 MB

10. Fine-granular motion matching (cont’d) • Disparity vector at each search point is represented as: • To find the optimal disparity Dopt, Lagrangian cost function is employed: where • Mi is the motion information of the block BVref,T(i) at the ith search point. • DREC(Mi) is measured as the sum of the squared differences (SSD) between the original MB and the reconstructed MB. • RREC(ΔDi ) is the sum of bits to encode the whole MB and ΔDi.

11. Fine-granular motion matching (cont’d) • To lower the complexity, the cost function, instead, is replaced for fast RD performance evaluation: where • Mi(x,y)x and Mi(x,y)y denote the MV components at x and y direction. • λMOTION = λMODE • The optimal motion information Mopt can be obtained once ΔDopt is determined.

12. Fine-granular motion matching (cont’d) • In case there are multiple inter-view reference pictures, the optimal incremental disparity ΔDopt and the index kopt of the selected reference picture can be obtained with: where ΔDi,k represent the incremental disparity at the ith search point in the kth inter-view reference picture.

13. Experimental environment • JMVM V5.0 • Test sequences: • QP: 22,27,32,37 • Search rage of disparity estimation: 96 • Size of the search window for the proposed fine-granular motion matching algorithm: (10×8-pel) × (10×8-pel)

14. Experimental results • Ration of motion skipped MBs:

15. Experimental results (cont’d)

16. Rate-distortion comparison • With/without base view: Without base view With base view

17. conclusion • 8-pel precision motion matching is applied to inter-view reference pictures. • Results show that the proposed algorithm increase the number of motion skip MBs. • Further improvement on overall RD performance for MVC can be achieved.