Performance Complexity Trade-Offs in H.264 Motion Search

1. Performance Complexity Trade-Offs in H.264 Motion Search Pooja Agarwal (pagarwal@nvidia.com) Apoorv Gupta (agupta@nvidia.com) Paul Kim (pakim@nvidia.com)

2. Overview • Outline • Background on Motion Search • Motion Estimation in Reference Code • Our approaches to reduce Motion Search Complexity • Results • Conclusion • Definitions • Performance = Video quality (PSNR) • Complexity = Time taken for Motion Estimation

3. Motion Estimation: A computation intensive process • Finding the ‘closest’ block in the previously coded frame (reference frame) • Closest neighbor is generally the one which has the least mean square error (MSE) or sum of absolute differences (SAD). • Motion vector (MV) = Displacement • Full Search in R range => (2R+1)2 computations per block

4. Motion Estimation Complexity in H.264/AVC • Multiple reference frames • Supports a range of different block sizes => more computation per macroblock • Half-pel and quarter-pel accuracy • Generalized B-frame and weighted prediction

5. Motion Vector Prediction • What is it? Usually there is a high correlation among the MVs of the adjacent blocks. Use this to predict the motion vector, and calculate the difference (MVD). • Why do it? Motion vector encoding can contribute to a significant amount of bits per picture, especially at low bit rates. So transfer only the MVD.

6. MV Prediction in H.264 Reference Code • E – current block, and A,B,C are its neighbors • Predicted-MVE = median (MVA, MVB, MVC) • If one or more block are not available, modify the choice accordingly

7. Motion Search in H.264 reference Code For each macroblock do { Do ME for all blocks of size 16x16, 16x8, and 8x16 Do ME for all blocks of size 8x8 for 8x8 DCT Do ME for blocksize 8x8, 4x8, 8x4, and 4x4 for 4x4 DCT Choose the best pel motion vector Do sub pixel ME Choose the best sub-pel MV }

8. How to reduce Motion Search space? • Adaptive reduction in motion search space • Use a more accurate prediction for MV • Bias the search to the region around this predicted MV and the center of the search space C – Center of the search range ; MVb – New predicted motion vector; r – search range

9. Approaches to Motion Vector Prediction • SAD based motion vector prediction • Parent based motion vector prediction

10. Predict the motion vector from the neighbor which is ‘closest’ to the current block ‘closeness’ is measured in terms of SAD Predicted MV = MVAi such that SADi = min(SAD1, SAD2, SAD3, SAD4) where SADi = SAD(Current, Ai) Measured motion vector statistics sequence table, 50 frames SAD based Motion Vector Predictor

11. Parent based Motion Vector Predictor • H.264 reference encoder computes the best motion vector starting from biggest block • We can use the motion vector of parent as the predicted MV of the child • For example, say B is a 16x16 macroblock and A be one of its 16x8 partition, then Predicted-MV (A) = MV(B)

12. Results Qcif, 100 Frames, 30 Hz

13. Results • Zoomed in version of the previous plot

14. Results

15. Results Qcif, 100 Frames, 30 Hz

16. Results

17. Conclusion • Using our predicted motion vectors we were able to reduce the search space significantly • This led to 40% to 50% reduction in the Motion Estimation time across five test sequences (foreman, table, mobile, tempete, mother-daughter) • The maximum degradation in video quality was around 0.2 dB observed in `table’

18. Acknowledgement • Eric Setton for his valuable guidance throughout the project • Prof. Girod and David Rebollo-Monedero for their help

19. More Results

20. More Results

21. More Results

22. More Results