Fast Mode Decision for H.264/AVC Based on Macroblock Motion Activity

1. Fast Mode Decision for H.264/AVC Based on Macroblock Motion Activity Huanqiang Zeng CanhuiCai, IEEE senior member Kai-Kuang Ma, IEEE senior member CSVT April,2009

2. Outline • Introduction • Overview of H.264/AVC’s mode decision • Fast mode decision methods • Proposed mode decision algorithm • Motion activity-based mode decision (MAMD) • Experimental results • Discussion

3. Overview of H.264/AVC’s mode decision (1/4) • For inter-frame MB, there are 11 candidate modes: • SKIP, 16×16, 16×8, 8×16, 8×8, 8×4, 4×8, 4×4, intra_4×4, intra_8×8 and intra_16×16 • intra_4×4, intra_8×8, intra_16×16 are denoted as I4MB, I8MB, and I16MB respectively. • For intra-frame MB, only I4MB, I8MB and I16MB are applicable. Do not need motion estimation operation Be introduced in H.264 FRExt profile, but most H.264 profiles do not support it.

4. Overview of H.264/AVC’s mode decision (2/4) • For SKIP mode MB: • All the DCT coefficients of the MB are set to zero. • No residual • Coded Block Pattern (CBP) is equal to zero • Its MVs can be produced from the MBs of the neighboring encoded MBs. • SKIP MB is highly beneficial to code areas with no motion or region with slow motion.

5. Overview of H.264/AVC’s mode decision (3/4) • For I4MB and I16MB: • The spatial correlation within each video frame will be exploited. • Only the residual block will be encoded. • Suitable for encoding a highly-textured region under fast motion.

6. Overview of H.264/AVC’s mode decision (4/4) Start End Time-consuming! Y Intra frame? N N Four 8×8 MB have finished encoding? Y Compute the cost of SKIP mode Perform ME process for 16×16, 16×8, and 8×16 Perform ME process for four 8×8 blocks. Compute the sub-MB mode (8×8, 8×4, 4×8, 4×4) Perform intra prediction. Compute the costs of I4MB and I16MB Select the mode that has the minimum cost among all the modes that have been checked

7. Fastmode decision methods • Optimize the Lagrangian function • To predict the best mode without checking all the modes. • The pixel difference between current MB and adjacent MBs • MVs of referenced MBs Cur_MB

8. Motivations • Test sequence and exhaustive mode decision • SKIP mode should be checked first. • Large block size → homogeneous region under slow motion Small block size → region containing fast moving objects • Current MB is intimately related to the motion activities of its spatially and temporally adjacent MBs. Dominant!

9. Region of support (ROS) • Region of support: • MBi, i={1,2,3,4} MB4 MB3 MB2 MB0 MB1 Previous frame Current frame

10. Motion Activity • Motion activity: • Use the maximum city-block lengthof the MVs taken from a region of support (ROS). Hard to distinguish! For all sequence at QP=28 For a MV=(mvx,mvy), the city-block length = |mvx|+|mvy|

11. Motion-activity classes • Motion activity and their involved modes: • Only the modes involved are required to be checked.

12. MAMD algorithm －threshold choosing • The motion activity of an MB is intimately related to its RD cost. • RD cost of SKIP mode: • Small enough → select SKIP mode as the best mode • Too large → likely to be a highly-textured region. Use modes in class 5 • Set two threshold: Tlow and Thigh

13. Motion-activity classes • Motion activity and their involved modes: • Only the modes involved are required to be checked. Tlow Thigh

14. MAMD algorithm －threshold choosing • These two thresholds should be QP dependent • The thresholds are empirically determinedwith a goal of achieving 90% degree of confidence.

15. MAMD algorithm －threshold choosing • The threshold values versus QP values:

16. MAMD algorithm (cont’d) • For class 2 to 4, establish the MV set {mvi|i=1,2,3,4} for current MB according to the ROS. • mvi=(xi , yi) MB4 MB3 MB2 MB0 MB1 Previous frame Current frame

17. MAMD algorithm (cont’d) • Compute the city-block length of each MV. • Find the maximum length L for the MV set. • Determine the motion activity of current MB. • Set L1=1 and L2=2 by previous statistics.

18. MAMD algorithm (cont’d) • If a MB has multiple MVs for different partition sizes, use the bottom-up merging procedure.

19. Flow chart

20. Note Check all the modes in class 2 to 4

21. Test condition • Software: JM10.2 • Test sequence: 6 QCIF-format and CIF-format seq. • Encode 300 frames • GOP structure: IPPP…. • QP=24, 28, 32, 36, 40 • RDO enabled • CABAC • One reference frame

22. Experimental results • For QCIF sequences QP↑, ΔT↑ AVG. -0.059 +0.19% -62.96%

23. Discussion • When Tlow↑, ΔT↑ • More MBs choose SKIP mode as the best mode • When Thigh↑, ΔT↓ • Less MBs choose intra mode as the best mode • However, “Mobile Calendar” is a highly-textured video sequence, its ΔT benefit from Tlow turns out be less then the ΔT increase due to the increment of Thigh. ↑

24. Experimental results • Compare CIF sequences with other two approaches

25. Experimental results • Compare CIF sequences with other two approaches [9],[10] [9] J. Bu, S. Lou, C. Chen, and J. Zhu, “A predictive block-size mode selection for inter frame in H.264,” in IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP),May 2006, vol. 2, pp. 917–920 [10] I. Choi, J. Lee, and B. Jeon, “Fast coding mode selection with rate-distortion optimization for MPEG-4 part-10 AVC/H.264,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 16, no. 12, pp. 1557–1561, Dec. 2006

26. Experimental results • The RD curve

27. Discussion • The sensitivity and robustness of the threshold values: “Foreman” sequence, QP=24

28. Conclusion • The evaluation of motion activity from spatially and temporally adjacent MBs. • Utilize the RD cost of SKIP mode as a criterion • Use QP dependent thresholds • Exploit the city-block lengths of MVs over ROS to predict the motion activity of current MB • Result (on average): • 62.96% time saving • 0.059 dB PSNR loss • 0.19% total bit rate increasing