Feature-Based Intra-/InterCoding Mode Selection for H.264/AVC

1. Feature-Based Intra-/InterCoding Mode Selection for H.264/AVC C. Kim and C.-C. Jay Kuo CSVT, April 2007

2. Outline • Introduction • Overview of Proposed Algorithm • Feature Selection • Feature Space Partitioning • Coding Mode Prediction • Experimental Results

3. Introduction • Inter/Intra Mode Decision in H.264 • Skip mode, direct mode, intra modes, and inter modes • Full mode decision • Testing all possible modes and then choosing the best mode with smallest cost • Fast algorithms • Selection of optimal inter-prediction mode • Selection of optimal intra-prediction mode • Binary decision of intra/inter mode

4. Overview of Proposed Algorithm Motion activity Choose min(f0,f1) Risk-Free Compute risk- minimizing mode MB Risk-Tolerable Risk-Intolerable f0, Residual of inter prediction Full mode decision f1, Residual of intra prediction

5. Feature Selection (1/4) • Intra mode feature • Calculate SATD for 5 modes • DC, vertical, horizontal, diagonal down-left, and diagonal down-right • Let f1 or fIntra be the SATD of the MB of the chosen modes

6. Feature Selection (2/4) • Inter mode feature • MV is obtained by • MVFAST + Two more candidates • Residual of every visited point is remembered in the memory • Search points of a MB < 512 • Let f0 or fInter be SATD of MB residual of the chosen MV (i-1,j-1) (i,j) n-1 n

7. Feature Selection (3/4) • Motion activity classification • Motion activity, decision error, and skipped frames • Decision metric • df = f1 – f0 • Intra (Inter): df < 0 (df > 0) • Decision error probability • P(df<0inter)+P(df>0intra)

8. Feature Selection (4/4) • Motion activity, RD cost difference dc, and feature difference df • dc = (D1 +1R1) - (D0+ 0R0) • Positive (Negative) if inter (intra) is better Best intra mode Best inter mode Low motion medium motion High motion

9. Feature Space Partitioning • The 3-D feature space is partitioned into three regions (off-line) • Lp: normalized RD cost between the best mode and the wrongly selected mode Threshold Motion activity Inter mode feature Intra mode feature

10. Feature Space Partitioning • Let every cell has about equal training data |MV| f1 f0

11. Feature Space Partitioning • Getting training data from Akiyo, Hall Monitor, Foreman, Coastguard, Stefan, Table Tennis, and Mobile.

12. Coding Mode Prediction (1/4) • Risk-Free region • Distribution of f0 and f1 in a given motion class based on feature difference Risk free

13. Coding Mode Prediction (2/4) • Risk-tolerable/-intolerable region Risk-tolerable and Risk-intolerable

14. Coding Mode Prediction (3/4) Cost of deciding ~mi under mj • Risk-tolerable region • Risk function The chosen mode mj is the best mode m0: intra m1: inter For simplicity, let  stands for cost instead of R

15. Coding Mode Prediction (4/4) • Risk-minimizing mode selection • Mode selection rule 0 0 • Likelihood ratio • Parametric • Semi-parametric • Nonparametric

16. Experimental Results (1/6) • Environments • JM7.3a • 32 x 32 motion search range • Fast full search with 5 reference frames • No B-frame • QP= {10, 16, 22, 28, 34} • 5 skipped frames

17. Experimental Results (2/6) • QCIF Table Tennis

18. Experimental Results (3/6) • QCIF Table Tennis Saving time Computation complexity

19. Experimental Results (4/6) • QCIF Foreman • 5 skipped frames 0 skipped frames

20. Experimental Results (5/6) • QCIF Stefan

21. Experimental Results (6/6)