Optimal End-to-end Distortion Estimation for Drift Management in Scalable Video Coding

1. Optimal End-to-end Distortion Estimation for Drift Management in Scalable Video Coding H. Yang, R. Zhang and K. Rose Signal Compression Lab ECE Department University of California, Santa Barbara SCL UCSB

2. Outline • Introduction • ROPE for scalable coding • R-D optimized mode selection • Simulation results • Conclusions SCL UCSB

3. Introduction • Scalable video coding • Drift problem • Multicast scenario & existent framework • Point-to-point scenario & proposed framework • Proposed coding approach SCL UCSB

4. Scalable video coding and drift problem • Scalable video coding • Error resilience • Multiple QoS • Drift problem • Whether to use enhancement layer information for prediction • If used better prediction improve coding gain • If lost mismatch / error error propagation SCL UCSB

5. Scalable video coding and drift problem • Drift management • Goal: Achieve a good trade-off. • Key : Accurately measure and thus effectively control the • amount of incurred error propagation. • H.263 and MPEG4 favor no-drift system. EI EP EP I P P SCL UCSB

6. Multicast Scenario & existent framework • Independent channels with different capacities • Some receivers have only access to the base layer, while others have access to both. • A coarse but acceptable base layer video quality is necessary. • Existent coding framework: EI EP EP I P P SCL UCSB

7. Point-to-point Scenario & proposed framework • Only one channel is considered. • Scalable coding only provides error resilience. • An acceptable base layer video quality is NOT necessary. • Proposed coding framework: • Research Purposes: • How much we can gain by using the proposed framework. • Investigate the importance of accurate end-to-end distortion estimation in effective management of drift. EI EP EP I P P SCL UCSB

8. Proposed coding approach • Macroblock(MB) based SNR scalable video coding • Objective: To minimize the expected end-to-end distortion given the packet loss rate and the total bit rate. • Drift management is fulfilled via R-D optimized coding mode selection for each MB. • To accurately estimate the end-to-end distortion, ROPE is adopted. SCL UCSB

9. Proposed coding approach • Coding mode selection is an efficient means to optimize the tradeoff between coding efficiency and error resilience. Intra: Stop error propagation & most bits. Inter B B: No new error & less bits. Inter E B: New error & least bits. EI EP EP I P P SCL UCSB

10. ROPE for Scalable Coding • Recursive Optimal per-Pixel Estimate (ROPE): • Take account of all the relevant factors as quatization, packet loss and error concealment. • Accurate & low complexity. • Adapt ROPE to scalable coding: • All the data of one frame is transmitted in one packet. • Channel is modeled as a Bernoulli process with packet loss only in the enhancement layer. SCL UCSB

11. Overall expected decoder distortion of pixel i in frame n: Original value , Encoder reconstruction values , Decoder reconstruction values , Quantized prediction residues Packet loss rate of the enhancement layer SCL UCSB

12. Calculation of and • Intra: • Inter B B: • Inter E B: SCL UCSB

13. Calculation of and assuming upward error concealment • Intra: • Upward: • Inter E E: SCL UCSB

14. RD Optimized Mode Selection • Unconstrained minimization: • J can be independently minimized for each MB. • The coding mode and quantization step size of each MB are jointly selected. • Sequential optimization: 1. 2. , . SCL UCSB

15. Simulation Results • UBC H.263+ codec with two-layer scalability • Mean luminance PSNR: average first over the frames and then over the packet loss patterns • Base layer bit rate: 75 kb/s, enhancement layer bit rate: 225kb/s, frame rate: 30 f/s, 150 frames, 50 packet loss patterns. SCL UCSB

16. (a) QCIF “Carphone” (b) QCIF “Miss_am” Fig.1 PSNR Performance of different coding frameworks SCL UCSB

17. (c) QCIF “Foreman” (d) QCIF “Salesman” Fig.1 (continued) • Gain of “B&E drift” over “E drift” : 0.92dB~ 2.83 dB • Gain of “E drift” over “no drift”: 1.25 dB ~2.20dB • With p increased, gain of “B&E drift” over “E drift” is unchanged, while gain of “E drift” over “no drift” is diminished quickly. SCL UCSB

18. (a) QCIF “Carphone” (b) QCIF “Miss_am” Fig.2 Performance of different distortion estimation methods SCL UCSB

19. (d) QCIF “Salesman” (c) QCIF “Foreman” Fig.2 (continued) • “ROPE-RD” always outperforms “QDE-RD”. • In most cases, “QDE-RD” performs even worse than “no drift”. SCL UCSB

20. Conclusions In the context of point-to-point video transmission over lossy networks: • Decoder drift due to prediction and packet loss should be controlled but not altogether disallowed. • Reaping the full benefits of drift management requires accurate estimation of end-to-end distortion. SCL UCSB