Performance Analysis and Implementation of Mode Dependent DCT/DST in H.264/AVC

1. Performance Analysis and Implementation of Mode Dependent DCT/DST in H.264/AVC PriyadarshiniAnjanappa Multimedia Processing Lab University of Texas at Arlington November 26, 2012 Advisor: Dr. K. R. Rao

2. Outline • Motivation • Overview of H.264 • Overview of HEVC • Mode Dependent DCT/DST • Implementation • Performance Analysis • Conclusion • Future Work

3. Motivation • Transform coding: Basic coding tool in image and video compression to reduce spatial redundancies • DCT-II used in recent video coding standards due to excellent energy compaction close to the optimal KLT • Integer approximation of DCT-II used in H.264 • Alternative Transforms proposed for HEVC • Integer approximation of DST-VII for 4x4 luma intra prediction residuals in HEVC • Revisiting Mode-Dependent DST/DCT implementation in HEVC to analyze the performance in H.264

4. Block Diagram of H.264 Encoder Typical H.264 encoder block diagram [1]

5. Block Diagram of H.264 Decoder Typical H.264 decoder block diagram [1]

6. Profiles in H.264 Profiles and corresponding tools in H.264 [2]

7. Intra prediction Intra prediction samples for a 4x4 block [3] A through M are previously predicted pixels (b) Directions for intra prediction modes for 4x4 and 8x8 blocks [3]

8. Intra prediction modes (a) Intra prediction modes for 4x4 blocks [4] (b) Intra prediction modes for 16x16 blocks [4]

9. Partitions for Inter prediction Macroblock partitions for inter prediction in H.264 [4]

10. Integer DCT matrices in H.264 4x4 Inverse DCT Matrix [5] 4x4 Forward Integer DCT Matrix [5]

11. Secondary Transform for DC Coefficients in H.264 4x4 and 2x2 Hadamard Transform matrices [6] The sixteen 4x4 blocks with DC coefficients drawn as smaller blocks on upper left corner [6]

12. Scan order in H.264 DC Scan order of transform coefficients [4]

13. Block Diagram of HEVC Encoder Typical HEVC encoder block diagram [7]

14. Block Diagram of HEVC Decoder Typical HEVC decoder block diagram [8]

15. Unified Intra prediction in HEVC : 33 directional modes + DC + Planar Directional mode for an 8x8 block [7] Modes and angular intra prediction directions [7]

16. Intra prediction directions in HEVC Illustration of Intra prediction directions with mode names mapped to the angular directions [9]

17. Partitions for Inter prediction in HEVC Partitions for inter prediction [7]

18. Quadtree coding structure in HEVC Subdivision of a CTB into CBs and TBs.Solid lines indicate CB boundaries and dotted lines indicate TB boundaries. Left: the CTB with its partitioning; right: the corresponding quadtree [7]

19. Core Transform : Integer DCT in HEVC Length-16 Integer DCT matrix in HEVC [7]

20. Integer DST in HEVC 4x4 Integer DST matrix used for intra prediction luma residual units in HEVC [7]

21. Adaptive scanning in HEVC Diagonal up-right scan (b) Horizontal scan (c) Vertical scan Scanning methods for 8x8 blocks in HEVC [7]

22. Mode Dependent DCT/DST • Motivation for DST [10]: • When prediction is performed from one-side, the energy in prediction error residuals increases as we go away from the boundary • A sine transform is better adaptable to these prediction residual statistics • Fast implementation of DST-VII is used in HEVC • For block size of NxN with j, j as row and column indices of the 2-D DST matrix for DST-VII is given as • Applied only to 4x4 luma intra prediction residuals

23. Mapping from intra prediction modes to DCT/DST in HM2.0 DCT/DST combinations applied to intra prediction modes[10]

24. Example of mapping a prediction mode to DCT/DST • Intra prediction mode: Vertical • DST in vertical direction • DCT in horizontal direction

25. Categories of unified intra prediction modes in HEVC • Category 1 oblique modes [12]: • Prediction is performed from the decoded pixels from either the top row or the left column. • The vertical mode 0 and the horizontal mode 1 are special cases of this oblique mode • Category 2 oblique modes [12]: • Prediction is performed from both the top row and the left column pixels • DC mode [12]: • Prediction is performed from an average of all available decoded pixels ; DCT is shown to be the optimal transform along both the directions

26. Category 1 oblique modes (a) Prediction from top row only [12] (b) Prediction from left-column only [12]

27. Category 2 Oblique modes A (dx,dy) pair denotes the horizontal and vertical distance respectively along the prediction direction for the pixel being predicted from the reference pixel [12]

28. Proposed Mapping from intra prediction modes to DCT/DST in H.264

29. Forward Integer DCT matrix in H.264 • Integer DCT matrix = Norms of the basis vectors along rows : • Row 1: 2 • Row 2: (3.1623) • Row 3: 2 • Row 4: (3.1623)

30. Inverse DCT matrix in H.264 • Inverse Integer DCT matrix = Norms of the basis vectors along rows : • Row 1: 2 • Row 2: (1.5811) • Row 3: 2 • Row 4: (1.5811)

31. Integer DST matrix in HEVC • Integer DST matrix = Norms of the basis vectors along rows : • Row 1: (128.0547) • Row 2: (128.1718) • Row 3: • Row 4: (128.0547)

32. Obtaining DST matrices for H.264 • To implement the same DST matrix in H.264, the norm along each row of the DST matrix is made equal to the norm of the corresponding row of the DCT matrices. • Divide rows of the DST matrix by the factor :

33. Forward DST matrix in H.264 • Forward Integer DST matrix = Norms of the basis vectors along rows : • Row 1: 2 • Row 2: 3.1622 • Row 3: 2 • Row 4:

34. Inverse DST matrix in H.264 • Inverse Integer DST matrix = Norms of the basis vectors along rows : • Row 1: 2 • Row 2: 1.5812 • Row 3: 2 • Row 4:

35. Implementation of Mode Dependent DCT/DST in H.264 • Mode dependent DCT/DST is implemented in the H.264 Reference Software JM 18.4 [13] in the Main profile • Matrix multiplication is implemented instead of fast implementation as the new transform matrices obtained are non-integer values • It is verified that the default fast DCT implementation in H.264 reference software and DCT matrix multiplication give the same results • Simulations are run on an i7 quad 4, 2.0 GHz processor, 8 GB RAM • Performance of default fast DST implementation in HEVC is also analyzed using HEVC reference software HM 8.0 [14] in the Intra Main profile

36. BD PSNR gives the absolute PSNR gain at the same bitrate and BD Bitrate gives the percentage bitrate savings at the same PSNR • BD PSNR and BD Rate [16] [17] are used to analyze the performance of the RD-curves • Video sequences[15] belonging to HD (1920x1080, 1280x720), WVGA (832x480) and WQVGA (416x240) are used for evaluating the performance • WVGA: Wide Video Graphics Array (Touch screen mobile phones support this resolution) [18] • WQVGA: Wide Quarter Video Graphics Array (Hand-held internet-enabled devices support this resolution) [18] • Only the first frame ( I frame) of the video sequences are considered for the analysis

37. Calculation of BD PSNR and BD Rate • BD PSNR and BD Rate represent the average PSNR and bitrate differences respectively between two rate distortion (RD) curves [16] • Fit a curve through four data points corresponding to QP = 16, 20, 24, 28 • The data points are Y-PSNR values for BD PSNR and bitrate values for BD Rate • A good interpolation curve through the four data points can be obtained by a third order polynomial of the form: SNR = a + b*bit + c*bit2 + d*bit3 where a, b, c, d are determined such that the curve passes through all the data points • The difference between the RD curves is dominated by high bitrates. Hence logarithmic value of bitrate is considered for calculating BD Rate • In the same way, we can do interpolation as a function of SNR as follows: SNR = a + b*SNR + c*SNR2 + d*SNR3 • Based on the interpolation , find an expression for the integral of the curve • The average difference is the difference between the integrals divided by the integration interval

38. Comparing Default H.264 and Default HEVC BD PSNR: 2.0212 dB BD Rate: -17.735% Comparing Default H.264 and Mode Dependent DCT/DST BD PSNR: -0.4913 dB BD Rate: 4.775%

39. Comparing Default H.264 and Default HEVC BD PSNR: 1.4722 dB BD Rate: -14.2433% Comparing Default H.264 and Mode Dependent DCT/DST BD PSNR: -0.4852 dB BD Rate: 5.1894%

40. Comparing Default H.264 and Default HEVC BD PSNR: 1.8785 dB BD Rate: -14.6427 % Comparing Default H.264 and Mode Dependent DCT/DST BD PSNR: -0.7315 dB BD Rate: 6.2691 %

41. Comparing Default H.264 and Default HEVC BD PSNR: 1.4754 dB BD Rate: -17.985 % Comparing Default H.264 and Mode Dependent DCT/DST BD PSNR: -0.3462 dB BD Rate: 4.6475 %

42. Comparing Default H.264 and Default HEVC BD PSNR: 1.3704 dB BD Rate: -20.3635 Comparing Default H.264 and Mode Dependent DCT/DST BD PSNR: 0.0665 dB BD Rate: -1.1473 %

43. Comparing Default H.264 and Default HEVC BD PSNR: 1.2773 dB BD Rate: -10.2026 % Comparing Default H.264 and Mode Dependent DCT/DST BD PSNR: -0.6996 dB BD Rate: 6.0669 %

44. Comparing Default H.264 and Default HEVC BD PSNR: 2.1567 dB BD Rate: -21.9681 % Comparing Default H.264 and Mode Dependent DCT/DST BD PSNR: -0.6369 dB BD Rate: 7.2692 %

45. Comparing Default H.264 and Default HEVC BD PSNR: 1.3025 dB BD Rate: -19.1702 % Comparing Default H.264 and Mode Dependent DCT/DST BD PSNR: -0.2102 dB BD Rate: 3.5532 %

46. Comparing Default H.264 and Default HEVC BD PSNR: 1.4856 dB BD Rate: -33.1268 % Comparing Default H.264 and Mode Dependent DCT/DST BD PSNR: -0.029 dB BD Rate: -0.4242 %

47. Comparing Default H.264 and Default HEVC BD PSNR: 2.053 dB BD Rate: -33.5926 % Comparing Default H.264 and Mode Dependent DCT/DST BD PSNR: 0.0216 dB BD Rate: -0.5157 %

48. Comparing Default H.264 and Default HEVC BD PSNR: 1.9721 dB BD Rate: -33.2696 % Comparing Default H.264 and Mode Dependent DCT/DST BD PSNR: 0.0851 dB BD Rate: -1.8388 %

49. Comparing Default H.264 and Default HEVC BD PSNR: 2.0659 dB BD Rate: -33.2327 % Comparing Default H.264 and Mode Dependent DCT/DST BD PSNR: 0.0422 dB BD Rate: -0.8327 %

50. Input sequence: RaceHorses_416x240_30.yuv