Mode Decision and Fast Motion Estimation in H.264

1. Mode Decision and Fast Motion Estimation in H.264 K.-C. Yang Qionghai Dai, Dongdong Zhu and Rong Ding,”FAST MODE DECISION FOR INTER PREDICTION IN H.264,” ICIP 2004 Zhi Zhou and Ming-Ting Sun,”FAST MACROBLOCK INTER MODE DECISION AND MOTION ESTIMATION FOR H.264/MPEG-4 AVC,” ICIP 2004

2. Introduction • Mode decision • Motion estimation strategies • Full Search (FS) • Three Step Search (TSS) • Diamond Search (DS) • ? 8*8 8*4 4*8 16*16 16*8 8*16 4*4

3. FAST MODE DECISION FOR INTER PREDICTION IN H.264 Qionghai Dai, Dongdong Zhu and Rong Ding Tsinghua University, China ICIP 2004

4. Outline • Framework • Down Sampling • Complex Motion • Edge Detector • Motion Estimation Strategy • Simulation Results

5. Framework Down sampling Current image Pre- encode Mode Encode with original image Original image Output Reference image Motion vector Reference image

6. Down Sampling(1) • 2:1 down sampling • 16x168x8, 16x88x4, 8x164x8, 8x84x4 • 8x4?, 4x8?, 4x4? (Complex motion inside)

7. Down Sampling(2) • ME and mode decision on down sampled image • DIRECT, 8x8, 8x4, 4x8, 4x4 in the down sampled image • Remember the top 2 best mode

8. Complex Motion in Smaller Blocks • Complex motion • Edge detection on the original image • Use Sobel operator • Horizontal edges  • Vertical edges  • Others  ,

9. -1 1 -1 -2 -1 -2 2 -1 1 1 2 1 Edge Detector(1) • Edge vector • Ampij = |dxij|+|dyij| • Angij = (180°/π)  arctan(dyij/dxij) • Histogram(k) = ∑(i,j)SET(k) Ampij • SET(k) = all pels s.t. Angijak • a1 = [-22.25°, 22.25°] (horizontal) • a2 = (-90°,-67.5°)(67.5°,90°) (vertical) • a3 = o.w. dyi,j  dxi,j  dyi,j dxi,j 90° a2 a3 a1 0° a3 a2 -90°

10. Edge Detector(2) • Histogram(1) > 2Histogram(2) and Histogram(1) > 2Histogram(3) • 8x4 mode • Histogram(2) > 2Histogram(1) and Histogram(2) > 2Histogram(3) • 4x8 mode • O.W. • 4x4 and 8x8 mode 1: Horizontal edges 2: Vertical edges 3: others

11. Motion Estimation Strategy • MVFAST motion estimation • Initial motion vectors • S1 : (0, 0), MVs of its neighbors • S2 : scaled MVs obtained from pre-encoding process • If min SAD for a MV in S2 is smaller than T1, STOP • T1 can be the number of pels of the examined block type • ELSE, let L be maximum amplitude of MVs in S1 • L TH1 (small motion) • Small diamond pattern • TH1 < L TH2 (medium motion) • Large diamond pattern • L > TH2 (large motion) • Use MV with minimum SAD from initial MVs • Small diamond is then used

12. Simulation Results • Compare with • JM61 with MVFAST algorithm • Config 1. All 7 inter block modes • Config 2. 16x16 inter block only

13. Simulation Results Stefan

14. FAST MACROBLOCK INTER MODE DECISION AND MOTION ESTIMATION FOR H.264/MPEG-4 AVC Zhi Zhou and Ming-Ting Sun Department of Electrical Engineering, University of Washington, Seattle ICIP 2004

15. Outline • Diversity-Based Fast Block Motion Estimation • Fast Variable Block-Size Motion Estimation Algorithms Based on Merge And Split Procedures for H.264/MPEG-4 AVC • Framework of Fast Mode Decision And ME • Simulation Results • Conclusion

16. Relative Work • DIVERSITY-BASED FAST BLOCK MOTION ESTIMATION ICME 2003 • FAST VARIABLE BLOCK-SIZE MOTION ESTIMATION ALGORITHMS BASED ON MERGE AND SPLIT PROCEDURES FOR H.264/MPEG-4 AVC ISCAS 2004

17. ... a b a b ... c d c d ... a b a b ... c d c d ... ... ... ... Diversity-Based Fast Block Motion Estimation • Adaptive diversity search strategy (ADSS) • Perform DS with {a} and {d}, and denote the two obtained MV v1 and v2. • If (v1 == v2)Returnv1. • If (|v1-v2|<TH) Use DS for {b} and {c}. Else use TSS for {b} and {c}. • Check v1, v2, v3, and v4. Return the MV with minimum SAD. CIF_Foreman 68% v1 and v2 are the same 96% of them are final MVs TH = 2

18. Fast Variable Block-Size Motion Estimation Algorithms Based on Merge And Split Procedures for H.264/MPEG-4 AVC • Motion vector merging and splitting • ADSS for initial blocks • Small Diamond Pattern for refinement A B E MVE = (MVA+MVB)/2 MVF = (MVC+MVD)/2 MVG = (MVA+MVC)/2 MVH = (MVB+MVD)/2 A B MVB = MVC = MVD = MVE = MVA MVF = (MVB+MVD)/2 MVG = (MVB+MVE)/2 ... ... C D F C G H I D E F G H I MVI = (MVE+MVF+MVG+MVH)/2

19. Framework of Fast Mode Decision And ME Merge+ME 16x16 Splitting+ME Merge+ME Predicted MV ME ME Splitting+ME Done 8*8 ME ME Merge+ME Merge+ME Splitting+ME Splitting+ME good good good good Done 16*16 Done 8*8 Merge Splitting+ME Done 4*4, 4*8, 8*4 Done 16*16, 16*8, 8*16

20. Motion cost of 16*16 at (0, 0) or PMV < TH16x16? Yes Prediction MV No 8*8 ME by ADSS Four motion cost of 8*8 < TH8*8? Yes Best mode=16*16, MV=(0,0) or PMV No 8*16 and 16*8 ME by MV merging Best mode=8*8, MV=MV by ADSS Compare motion cost of 8*16, 16*8 with that of 8*8 Both are larger than 8*8 Otherwise A B 8*8 sub-MB mode decision 16*16 ME by MV merging Four 8*8 sub-MBs done? Choose best on from 16*16, 16*8 and 8*16 No C 1 modes 4 modes 0 modes Yes End

21. 8*4 and 4*8 ME by MV splitting Compare motion cost of 8*4, 4*8 with that of 8*8 Both are larger than 8*8 Otherwise Best mode=8*8 4*4 ME by MV splitting Choose best on from 4*4, 8*4 and 4*8 D E 5 modes 6 modes End

22. Threshold • TH16*16= (min cost of previous 20 16*16 blocks) + 600 • TH8*8= (min cost of previous 20 8*8 blocks) + 150

23. C D E Simulation Results • LCM-FFS: Low-Complexity Mode JM with Fast Full-Search • LCM-MSS: Low-Complexity Mode JM with Merge-Split Search

24. Conclusion • A new method for mode decision and ME • Block splitting and merging • Diversity-based fast ME • Thresholds • Compare with JM with FFS • Save about ½ computation • Quality degradation for 0.1~0.4 dB