1 / 41

Analysis of Motion Estimation Algorithm ( HEVC), using Multi-core processing

Analysis of Motion Estimation Algorithm ( HEVC), using Multi-core processing. Shiba Kuanar Shiba.kuanar@mavs.uta.edu 1000449352 EE5359 – Interim Report. CONTENTS:. OVERVIEW OF HEVC SEARCH ALGORITHEM RESULTS REFERENCES. HEVC.

kristine-eris
Download Presentation

Analysis of Motion Estimation Algorithm ( HEVC), using Multi-core processing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Analysis of Motion Estimation Algorithm (HEVC), using Multi-core processing ShibaKuanar Shiba.kuanar@mavs.uta.edu 1000449352 EE5359 – Interim Report

  2. CONTENTS: • OVERVIEW OF HEVC • SEARCH ALGORITHEM • RESULTS • REFERENCES

  3. HEVC • High Efficiency Video Coding (HEVC) is the newest video coding standard of the ITU-T Video Coding Experts Group and the ISO/IEC Moving Picture Experts Group. • The main goal of the HEVC standardization effort is to enable significantly improved compression performance relative to existing standards in the range of 50% bit-rate reduction for equal perceptual video quality. • Video Coding Standards: Always have a trade between 1) Picture Quality 2) Compression Ratio 3) Computation complexity

  4. Fig 1:HEVC Encoder

  5. HEVC • Partitioning – into non overlapping macro blocks • Prediction - Forward/Backward based on current, past and future frame • Error signal - transformed and quantized • Entropy coded

  6. Figure 2: HEVC decoder block diagram [2]

  7. Fig 3: Partitioning of a Macro block CTB->CBs andTBs Solid lines indicate CB boundaries and dotted lines indicate TB boundaries Corresponding QUAD TREE

  8. Motion Estimation(ME) • Block Based ME algorithms are proposed to reduce the computation time. Algorithms will be discussed the current proposal “Test Zone (TZ) search” “Full search algorithm” • Proposed work is to analyze the existing algorithm and try to reduce motion estimation computation time using multicore programming. • “Motion estimation consumes more than 84% encoding time and coding complexity to encode • HM13.0 – latest HEVC software. • Block Matching Criterions used in BBME - Mean of Square Error - Mean of absolute difference (No multiplication) - Matching Pixel Count.

  9. Figure: 4. [22], Percentage “execution time” distributions of the HEVC encoder steps, for an HD sequence

  10. Block based Motion Estimation • In HEVC video coding, motion estimation (ME) plays a vital role in temporary redundancy reduction between the frames. • In BBME, the current frame is divided into N ×N pixel size macro blocks (MBs) and for each MB a certain area of the reference frame is searched, to minimize a block difference measure (BDM). • The block difference measure is usually a sum of absolute differences (SAD) between the current and the reference frame MB. • The displacement within the search area which gives the minimum BDM value is called a motion vector (MV). • MVs together with transformed and quantized block differences (residua) are entropy coded into the video bit stream. • To decrease such a huge computational burden, many fast BBME algorithms have been proposed. On the current proposal only two algorithms are going to be analyzed 1) Algorithm based on the search position number and 2) algorithm based on reduction of fast full search.

  11. Motion Estimation (ME)[19] • “Pel by Pel” or “Block by Block motion” algorithm used for ME. • ME improves the prediction accuracy between current frame and reference frame. • In Block matching algorithm (BMA), motion of a block of pels (M*N) within a frame interval is estimated. • The range of the motion vector is constrained by the search window. • Best match between the (M*N) block on current frame and corresponding block in previous frame calculated within a search window (M+2*m2) * (N+2*n1) (figure 5) • BMA uses minimum mean absolute error, mean squared error cost functions to find the best match.

  12. Figure 5:Motion Estimation of an (M*N) Block in the previous frame within an [19][(M+2*m2) * (N+2*n1)] search window. MV range is [-n1 to n1] pels and [-m2 to m2] rows (2d+ 1)² search points (SPs)

  13. Motion Estimation • The cost function computed every pel and line displacement within search window & compared to find the location of optimal cost function • To find the minimum cost function in figure 5, requires (2m2+1)*(2n1+1) times computation for every possible displacement constrained by the MV range. • Once ME carried out based on integer pel MV resolution surrounding the best match then MV can be extended to half a pel /line by interpolating the fractional pels. • So ME calculation computationally intensive and many ME algorithms are developed.

  14. Figure 6:Motion Estimation at the eight fractional pels surrounding the integer pel locations of the MV (pel a)

  15. Fig 7: BBME Algorithm for a) 2DLOG, b) TSS and c) NTSS search

  16. Fig 8: Logarithmic Search algorithm

  17. Figure 9: Various Search Patterns: a) Diamond search, b) Square Search, c) Horizontal Hexagonal search, d) Vertical Hexagonal Search, e)Rotating Hexagon Type - 1 and f) Rotating Hexagon Type - 2 [17, 20] • a) Diamond search b) Square Search c) Horizontal Hexagonal d) Vertical Hexagonal e) Rotating Hexagon Type – 1 f) Rotating Hexagon Type – 2

  18. Test zone (TZ) Search Algorithm Process (4 steps) • TZ Search algorithm is one of the fast search algorithms in HEVC motion estimation and reduces the encoding time. • 1) Motion Vector Prediction: TZS algorithm employs median predictor, left predictor, up predictor, upper right predictor. The minimum of these predictors is selected as a starting location for further search steps. • 2) Initial Grid Search: In this step, the algorithm searches the search window in using diamond or square patterns with different stride lengths ranging from1 through 64, in multiples of 2. • 3) Raster Search: The raster search is a simple full-search on a down-sampled version of the search window. • 4) Raster/Star Refinement: The Raster refinement is a fine refinement of the motion vectors obtained from the step three.

  19. Figure 10 Flowchart of TZS Algorithm [28]

  20. Test Sequence 1 -BasketballDrill [24] Resolution = 832x480 and frame = 50

  21. Test Sequence 2RaceHorses [24], Resolution = 832x480 and frame = 50

  22. BasketballDrill: TZ search(Fast Search algorithm)4:2:0 video formatPSNR(Y, U, V) = [6 * PSNR(Y) + PSNR (U) + PSNR (V)] / 8

  23. BasketballDrill: Full search (Fast Search algorithm)4:2:0 video formatPSNR(Y, U, V) = [6 * PSNR(Y) + PSNR (U) + PSNR (V)] / 8

  24. Project Results • The simulation conducted on HM Software 13.0 using BasketballDrill and RaceHorses test sequences [24] and with a single core and eight core systems. • The PSNR (dB), encoding time in sec and bitrate (kbps) are calculated by varying quantization parameter. The results are calculated for both TZ search and Full search algorithms. • The PSNR calculated using PSNR(y, u, and v) = [6 * PSNR(y) + PSNR (u) + PSNR (v)] / 8, for 4:2:0 sample video format. • The “PSNR – Bitrate” and “PSNR – Quantization Parameter” plotted for the TZ search & Full search.

  25. BasketballDrillPSNR – Bitrate Plot (TZ Search and Full Search)

  26. BasketballDrillQP – PSNR Plot (TZ Search and Full Search)

  27. BasketballDrill 832x480 and 50 framesSimulation values for theTestzone (TZ) search algorithm on eight core systems

  28. Eight core simulation resultsTZ search Encoding time for eight core processors.

  29. Encoding time ~ QP plot - TZ search for a single core and eight core processors. Multi core has a better encoding time as compared to single core on TZ search and there is a 20% improvement in encoding time keeping the video quality same.

  30. BasketballDrill 832x480 and 50 frameseightcoreSimulation values for theFull search algorithm using eight core systems

  31. “Encoding time in Sec ~ QP” Full Search plot. multicore has a better encoding time as compared to single core, on Full search and encoding time reduced by 12%, keeping the video quality same

  32. Future Work • Further simulations can be conducted on NVIDIA multi core processors using two hundred processors and windows7 operating system (64 bits) for the enhanced TZ search algorithms and with some early termination criteria.

  33. Acronyms • AVC: Advanced Video Coding • BBME: Block Based Motion Estimation • BDM: Block Difference Measure • BMA: Block matching algorithm • CABAC: Context Adaptive Binary Arithmetic Coding • CB: Coding Block • CSVT: Circuits and Systems for Video Technology • CTB: Coding Tree Block • CTU: Coding Tree Unit • CPU: Central Processing Unit • CU: Coding Unit • CUDA: Compute Unified Device Architecture • DCT: Discrete Cosine Transform

  34. Acronyms • GPU: Graphic Processor Unit • HEVC: High Efficiency Video Coding • ISO: International Organization for Standardization • ITU-T: International Telecommunication Union – Telecommunication Standardization Sector • JCT-VC: Joint Collaborative Team on Video Coding • MC: Motion Compensation • MCP: Motion Compensated Predication • ME: Motion Estimation • MPEG: Moving Picture Experts Group • OPENMP: Open Multiprocessing • PB: Prediction Block

  35. Acronyms • PCM: Pulse Code Modulation • PSNR: Peak signal-to-noise ratio • PU: Prediction Unit • SAD: Sum of Absolute Differences • SOA: Sample Adaptive Offset • SIMD: Single Instruction Multiple Data • TB: Transform Block • TU: Transform Unit • VCEG: Video Coding Experts Group • VBSME: Variable Block Size Motion Estimation • 2DLOG: Two dimensional logarithmic search procedure • TSS: Three step search • NTSS: New three step search

  36. References Reference thesis • 1 Thesis by S.Gangavati on “Complexity reduction of H.264 using parallel programming” which describes significant speed-up in encoding time on GPU using CUDA and CPU combined than on CPU by data and task parallelization, 2012. http://www-ee.uta.edu/Dip/Courses/EE5359/Sudeep_Thesis_Draft_2.pdf • 2 Thesis proposed by Pratik Meheta on “Complexity reduction for intra mode selection in HEVC using OpenMP” http://www-ee.uta.edu/Dip/Courses/EE5359/Pratik_Mehta_ThesisProposal.pdf References • [1] G. J. Sullivan et al, “Overview of the high efficiency video coding (HEVC) standard”, IEEE Trans. CSVT, vol. 22, pp.1649-1668, Dec. 2012. • [2] G. J. Sullivan et al “Standardized Extensions of High Efficiency Video Coding (HEVC). J. Sel. IEEE Journal of topics in Signal Processing” vol. 7, pp.1001-1016, Dec. 2013 • [3] F. Bossen et al, “HEVC complexity and implementation analysis”, IEEE Trans. CSVT, vol. 22, pp.1685-1696, Dec. 2012. • [4] H. Zhang and Z. Ma, “Fast intra prediction for high efficiency video coding”, Pacific Rim Conf. on Multimedia, PCM2012, Singapore, Dec. 2012. • [5] C.C. Chi et al, “Parallel scalability and efficiency of HEVC parallelization approaches”, IEEE Trans. CSVT, vol. 22, pp.1827-1838, Dec. 2012.

  37. References • [6] Introduction to parallel computing URL - https://computing.llnl.gov/tutorials/parallel_comp/#Whatis • [7] T. Wiegand and G.J. Sullivan, "Overview of the H.264/AVC video coding standard," IEEE Trans. on Circ. Sys. for Video Tech., vol. 13, pp. 560-576, no. 7, July 2003.  • [8] Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG, "Comments on Motion Estimation Algorithms in Current JM Software (JVT-Q089)", Joint Video Team Document, 17th Meeting: Nice, FR, 14-21 Oct., 2005.  • [9] N.Purnachand, L.N. Alves, A. Navarro “Improvements to TZ search motion estimation algorithm for multiview video coding” 19th International Conference on Systems, Signals and Image Processing IWSSIP, pp. 388 -391, 2012. • [10] B. Bross et al, “High Efficiency Video Coding (HEVC) Text Specification Draft 10”, Document JCTVC-L1003, ITU-T/ISO/IEC Joint Collaborative Team on Video Coding (JCT-VC), Mar. 2013 available on http://phenix.itsudparis.eu/jct/doc_end_user/current_document.php?id=7243 ​ • [11] J. Ascenso, C. Brites and F. Pereira, "Improving Frame Interpolation with Spatial Motion Smoothing for Pixel Domain Distributed Video Coding", in Proc. EURASIP Conference on Speech and Image Processing, Multimedia Communication and Services, Slovak Republic, June-July 2005.  • [12] W. Hong, “Coherent Block-Based Motion Estimation for Motion-Compensated Frame Rate Up-Conversion", IEEE International Conference on Consumer Electronics, pp. 165-166, Jan. 2010.  

  38. References • [13] G. Bjontegaard, "Calculation of average PSNR difference between RD curves", VCEG-M33, 2001. • [14] C. Fogg, “Suggested figures for the HEVC specification”, ITU-T/ISO/IEC Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC- J0292r1, July 2012. • [15] P. Hanhart et al, “ Subjective quality evaluation of the upcoming HEVC video compression standard”, SPIE Applications of digital image processing XXXV , vol. 8499, paper 8499 -30, Aug. 2012. • [16] M. Horowitz et al, “Informal subjective quality comparison of video compression performance of the HEVC and H.264/MPEG - 4 AVC standards for low delay applications”, SPIE Applications of digital image processing XXXV , vol. 8499, paper 8499 - 31, Aug. 2012. • [17] L.N.A. Alves and A. Navarro, " Fast Motion Estimation Algorithm for HEVC ", Proc IEEE International Conf. on Consumer Electronics -ICCE Berlin , Germany , vol.11 , pp. 11 - 14 , Sep. , 2012 • [18] X. Wang et al, Paralleling Variable Block Size Motion Estimation of HEVC on Multicore CPU plus GPU platform, IEEE International Conference on Image Processing (ICIP 2013), Melbourne, Australia, Sep.15-18, 2013. • [19] K.R. Rao, D.N. Kim and J.J. Hwang, “Video Coding Standards” – Springer 2014. • [20] M. Jakubowski and G. Pastuszak, “Block-based motion estimation algorithms-a survey,” Journal of Opto-Electronics Review, vol. 21, pp 86-102, Mar. 2013.

  39. References • [21] A. Abdelazim,W. Masri and B. Noaman "Motion estimation optimization tools for the emerging high efficiency video coding (HEVC)", SPIE vol. 9029, Visual Information Processing and Communication V, 902905, Feb. 17, 2014. • [22] M. Shafique et al,"An adaptive workload management scheme for HEVC encoding",Image Processing (ICIP),IEEE International Conference PP:1850-1854, Sept. 2013 • [23] Software repository for HEVC - http://hevc.hhi.fraunhofer.de/ • [24] Video test sequences – http://forum.doom9.org/archive/index.php/t-135034.html or http://media.xiph.org/video/derf/ • [25] HM Software - https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/ or https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/tags/HM-13.0rc1/ • [26] Y.S. Ho and K.J. Oh, “Overview of Multi-view VideoCoding,” IWSSIP and EC - SIPMCS -Proc. 14th Int.Workshop on Systems, Signals and Image Processing and 6th EURASIP Conf. Focused on Speech and ImageProcessing, Multimedia Communications and Services, pp.5-12, 2007. • [27] X.L. Tang, S.K. Dai, C.H. Cai, “An Analysis of TZSearch Algorithm in JMVC,” 1st InternationalConference on Green Circuits and Systems ICGCS, pp.516 -520, 2010.

  40. References • [28] JVT of ISO/IEC MPEG, ITU -T VCEG, MVC software Reference Manual- JMVC 8.5, Mar 2011. http://phenix.itsudparis.eu/jct/doc_end_user/current_document.php?id=7243 • [29] HM Software Manual - https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/ • [30] Special issue on emerging research and standards in next generation video coding, IEEE Transactions on CSVT, vol. 22, pp. 1646-1909, Dec 2012 • [31] Special issue on emerging research and standards in next generation video coding. IEEE Transactions on CSVT, vol. 23, pp. 2009-2142, Dec 2013 • [32] IEEE Journal of selected Topics in Signal Processing, vol. 7, pp. 931-1151, Dec 2013

  41. THANK YOU

More Related