1 / 23

Rate-Distortion Optimized Motion Estimation for Error Resilient Video Coding

Hua Yang and Kenneth Rose Signal Compression Lab ECE Department University of California Santa Barbara, USA Mar. 2005. Rate-Distortion Optimized Motion Estimation for Error Resilient Video Coding. Outline. Motion estimation (ME) for coding efficiency Conventional ME

zlata
Download Presentation

Rate-Distortion Optimized Motion Estimation for Error Resilient Video Coding

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Hua Yang and Kenneth Rose Signal Compression Lab ECE Department University of California Santa Barbara, USA Mar. 2005 Rate-Distortion Optimized Motion Estimation for Error Resilient Video Coding

  2. Outline • Motion estimation (ME) for coding efficiency • Conventional ME • Rate-constrained ME & rate-distortion (RD) optimized ME • Motion estimation for error resilience • Proposed end-to-end distortion based RDME • Intuition behind • End-to-end distortion analysis • Simulation results • Conclusions ICASSP 2005

  3. Coded frame n-1 Original frame n Motion Estimation for Coding Efficiency • Motion compensated prediction (MCP) • To remove inherent temporal redundancy of video signal • Both the motion vector and the prediction residue are encoded. ICASSP 2005

  4. Motion Estimation for Coding Efficiency • Conventional motion estimation • ME Criterion: minimize prediction residue • Ignoring the motion vector bit-rate cost ICASSP 2005

  5. Rate-constrained motion estimation : Lagrange multiplier • However, not yet the ultimate rate-distortion optimization for the best overall coding performance. Motion Estimation for Coding Efficiency • Motion estimation in low bit rate video coding • In low bit rate video coding, motion vectors may occupy a significant portion of total bit rate. • Efficient bit allocation between motion vector and prediction residue coding is necessary for better overall coding efficiency. ICASSP 2005

  6. Motion Estimation for Coding Efficiency • Motion estimation for low bit rate video coding (cont’d) • Rate-distortion optimized motion estimation (RDME) • Some references • [Girod `94] Theoretical analysis of rate-constrained ME • [Sullivan `98] Summary of rate-constrained ME • [Chung `96] Low complexity RDME for each MB using RD modeling • [Schuster `97] Joint RDME for multiple MB’s ICASSP 2005

  7. Error resilient video coding • RD optimization with end-to-end distortion • Coding mode selection: {Intra/Inter, QP} No mv for Inter-mode! Not comprehensively attack the RD optimization problem! Motion Estimation for Error Resilience • In the presence of packet loss: • Packet loss & error propagation • Internet – no QoS guarantee Wireless – inherent error-prone channel • Error propagation due to MCP • Error resilience via motion compensation • Multi-frame motion compensation (MFMC) [Budagavi `01] • Reference picture selection (RPS) [H.263+] • Error resilient rate-constrained ME [Wiegand `00] ICASSP 2005

  8. Motion Estimation for Error Resilience • We propose end-to-end distortion based RDME [accounting for packet loss] • The exact RD optimal ME solution for error resilience • Critical: accurate pixel-level end-to-end distortion estimation • Build on: recursive optimal per-pixel estimate (ROPE) [R. Zhang, S. Regunathan, and K. Rose `00] ICASSP 2005

  9. P1 I P2 P3 P1 P2 P4 P1 I For coding efficiency For error resilience Best trade-off • Conventional motion estimation completely ignores the error resilience information. • This error resilience information should be exactly considered for each pixel. Proposed RDME • Intuition for “error resilience via ME” ICASSP 2005

  10. Error concealment ROPE Error propagated distortion • DEP is explicitly affected by mv, whose minimization favors mv’s that point to reference areas with less encoder-decoder mismatch. Proposed RDME • ROPE-based end-to-end distortion analysis ICASSP 2005

  11. Packet loss impact Proposed RDME • The proposed RDME solution • Comparing with existent RDME • Source coding distortion  end-to-end distortion • mv affects not only the Rmv vs. Rres trade-off, but also more importantly, the coding efficiency vs. error resilience trade-off. • Comparing with existent RD optimized coding mode selection • Extended Inter mode with the mv parameter • Further optimize the Inter-mode performance ICASSP 2005

  12. Simulations • Objective: to check upper-bound performance • Joint {mv, QP} optimization • RD calculation via actual encoding • Simulation settings • UBC H.263+ • Encoding: I-P-P-…… • Transmission: independent packet loss, with a uniform p • Decoding: 50 different packet loss realizations for each p • Performance: average luminance PSNR ICASSP 2005

  13. Simulations • Simulation settings (cont’d) • Testing methods • Conventional ME (cME) • The proposed RDME (RDME) • Testing scenarios • Random Intra updating (rI): arbitrarily assigns MB’s to 1/p groups, and cycles through them updating one group per frame. • Optimal Intra updating (oI): RD optimized Intra/Inter mode selection. ICASSP 2005

  14. Simulation Results Random Intra Miss_am Foreman PSNR vs. Packet loss rate [QCIF, 10f/s, 48kb/s] ICASSP 2005

  15. Simulation Results Optimal Intra Miss_am Foreman PSNR vs. Packet loss rate [QCIF, 10f/s, 48kb/s] ICASSP 2005

  16. Simulation Results Random Intra Miss_am Foreman PSNR vs. Total bit rate [QCIF, 10f/s, p=10%] ICASSP 2005

  17. Simulation Results Optimal Intra Miss_am Foreman PSNR vs. Total bit rate [QCIF, 10f/s, p=10%] ICASSP 2005

  18. Simulation Results Conventional ME [29.58dB] RDME [33.83dB] Miss_am: QCIF, 10f/s, 48kb/s, p=10%, random Intra ICASSP 2005

  19. Simulation Results RDME [26.92dB] Conventional ME [23.92dB] Foreman: 1st 200f, QCIF, 10f/s, 112kb/s, p=10%, random Intra ICASSP 2005

  20. Besides Intra updating, RDME presents another good alternative for error resilience. Conclusions • Identify the new opportunity of achieving error resilience via motion estimation. • Propose an RD optimal ME solution, which further optimizes the Inter-mode performance. • Investigate the upper-bound performance. • With random Intra: substantial gain • With optimal Intra: significant gain at low bit rates. ICASSP 2005

  21. Conclusions • Originally, the power of Intra coded MB’s is only recognized as stopping past error propagation, while the proposed RDME reveals their new potential on reducing future error propagation. • Future work I: more comprehensive tests • Inaccurate p, bursty loss, or over actual networks, etc. • Future work II: complexity reduction • RD modeling, separate mv and QP optimization, sophisticated ME strategies, etc. ICASSP 2005

  22. References • [Girod `94] B. Girod, ``Rate-constrained motion estimation,'' Nov. 1994. • [Sullivan `98] G. J. Sullivan and T. Wiegand, ``Rate-distortion optimization for video compression,’’ Nov. 1998. • [Chung `96] W. C. Chung, F. Kossentini, and M. J. T. Smith, ``An efficient motion estimation technique based on a rate-distortion criterion,'' May 1996. • [Schuster `97] G. M. Schuster and A. K. Katsaggeslos, ``A theory for the optimal bit allocation between displacement vector field and displaced frame difference,'' Dec. 1997. • [Budagavi `01] M. Budagavi and J. D. Gibson, ``Multiframe video coding for improved performance over wireless channels,'' Feb. 2001. • [H.263+] ITU-T, Rec. H,263, ``Video codeing for low bitrate communications'', version 2 (H.263+), Jan. 1998. • [Wiegand `00] T. Wiegand, N. Farber, K. Stuhlmuller and B. Girod, ``Error-resilient video transmission using long-term memory motion-compensated prediction,'' June 2000. • [Zhang `00] R. Zhang, S. L. Regunathan, and K. Rose, ``Video coding with optimal intra/inter mode switching for packet loss resilience,'' June 2000. ICASSP 2005

  23. The End ICASSP 2005

More Related