Flexible Fast Block Matching Algorithm Design based on Complexity-Distortion Optimization

1. Flexible Fast Block Matching Algorithm Design based on Complexity-Distortion Optimization Pol Lin Tai, Chii Tung Liu, Shih Yu Huang*, Jia Shung Wang Department of Computer Science National Tsing Hua University, Taiwan, R.O.C. *Department of Information Management Ming Chuan University, Taiwan, R.O.C.

2. Outline • Introduction - Block Motion Estimation • Traditional Fast Block Motion Estimation • Adaptive Complexity-Distortion Block Motion Estimation • Experimental Results • Conclusions

3. vector MAE Frame n-1 Frame n (-3,-3) 50 y (-2,-3) 70 -3 Motion vector (-1,-3) 45 -2       -1 (-2,2) 0 -1 0 -2 1 2 -3 x 3       1 2 (3,3) 120 3 motion vector arg min = MAE ( u , v ) Search area ( u , v ) Current block Full-Search Block Matching

4. Three-Step Search 1 x x x x x x x x 2 Referenced frame Current frame Motion vector 16 16 4 Search area Current block £ £ motion vector arg min MAE ( u , v ), and - P u, v P = ( u , v ) Fast Block Matching Algorithm •  fast block matching • - three-step search, four-step search • - new three-step search  reduce check point(u,v)

5. Problems of the Traditional BM (1) Fixed Computational Complexity MSE 1100 1000 FSBM 900 TSS 800 New-TSS 700 FSS 600 500 400 4 9 14 19 24 29 10log(check point)

6. Block 1 Block 2 block3 Block 4 Initial mv (0,0) 50 60 30 80 Step one (8 check points) :1+9+1+25 = 36 50 40 30 70 Step two (8 check points) 50 40 30 60 Step three (8 check points) 50 40 30 50 254=100 The search path for TSS Problems of the Traditional BM (cont.) (2) Complexity-Distortion Optimization

7. Complexity-Distortion Optimization BM  flexible computational complexity  better complexity-distortion performance MSE 1100 CDOBM 1000 FSBM 900 TSS 800 New-TSS 700 FSS 600 500 400 4 9 14 19 24 29 10log(check point)

8.  • In Rate-Distortion Optimization •  trellis system • - dynamic programming • Viterbi algorithm 50 60 30 80 0 20 50 40 30 70 50 40 30 60 In Complexity-Distortion Optimization  MSE Benefit ?  Non-Reusable  Heuristic search algorithm 50 40 30 50 Complexity-Distortion Optimization

9. 4 2 1 3 Block 1 Block 2 block3 Block 4 50 60 30 80 Initial mv (0,0) 50 40 20 65 Step one (8 check points) 50 34 20 60 Step two (8 check points) 50 30 20 50 Step three (8 check points) How to Define MSE benefit  In each checking step, select the block that could achieve the maximum improved distortion benefit to apply searching procedure

10. Block1 Block2 Block3 Block4 Block5 MSE: Assign benefit 200 300 500 300 100 400 200 400 500 100 Block1 Block2 Block3 Block4 Block5 benefit: 300 500 200 400 100 Block2 Block4 Block1 Block3 Block5 Sort Predictive Complexity-Distortion Benefit List Predictive Complexity-Distortion Benefit List

11. Block2 Block4 Block1 Block3 Block5 PCDB List 500 Select Block2 to check candidate blocks (MSEcheck=250) MSEcheck<MSE2 MSEcheck>MSE2 400 300 200 100 Benefit2=MSEcheck Benefit2= 0.7Benefit2 Update PCDB List Update PCDB List Block4 Block2 Block1 Block3 Block5 Block4 Block1 Block2 Block3 Block5 400 350 300 200 100 300 400 250 200 100 Updating PCDB List

12. Select the first block from PDCB list Checking candidate blocks Initialize PCDB list Updating PCDB list End Total complexity > target complexity ? Flexible Fast Block Matching Algorithm Design Start No Yes

13. Block2 Block4 Block1 Block3 Block5 PCDB List 500 Select Block2 to check candidate motion vector Search pattern FSBM 3-Step 400 300 200 100 Checking Candidate Motion Vector

14. Experimental Results • QCIF: “Miss America”, “Suzie” • SIF: “Football” • full-search block matching, 3-step search, new 3-step search, 4-step search • frame rate: 15Hz and 7.5Hz • search area: SIF (-15,15), QCIF (-7,7) • is set as 0.8

15. Experimental Results (cont.) The complexity-distortion performance comparison for the QCIF "Miss America" with (a) frame rate 15Hz Distortion (MSE) 19 17 flexible-New-TSS 15 13 flexible-TSS 11 flexible-FSS TSS FSS New-TSS FSBM 9 flexible-FSBM 7 Complexity (10ln(check point)) 0 10 20 30 40 50

16. Experimental Results (cont.) The complexity-distortion performance comparison for the QCIF "Miss America" with (a) framerate 7.5Hz Distortion (MSE) 41 36 flexible-New-TSS 31 flexible-TSS 26 21 flexible-FSS TSS 16 FSS FSBM New-TSS flexible-FSBM 11 Complexity (10ln(check point)) 0 10 20 30 40 50

17. Experimental Results (cont.) • The performance summary of the flexible BMAs The Worst Algorithm The Best Algorithm complexity video Low High Low High frame rate 15 FSBM New-TSS TSS FSS Miss America 7.5 FSS New-TSS TSS FSS 15 FSBM New-TSS TSS FSS Suzie 7.5 FSS New-TSS TSS FSS 15 FSS New-TSS FSBM FSS Football 7.5 TSS New-TSS FSBM FSS

18. Conclusions • Discuss how to utilize the limited computational complexity of the block matching algorithm to achieve the maximum quality of the compensated image under a target computational complexity • Propose predictive complexity-distortion benefit list technique • The flexible block matching algorithm design not only improves the efficiency of the traditional BMAs, but also provides a flexible motion estimation tool that allows user to terminate it at any computational complexity