Xin Li and Louis Kerofsky Image Coding and Communication Group Digital Video Department

1. High Performance Resolution Scalable Video Coding via All-Phase Motion Compensated Prediction of Wavelet Coefficients Xin Li and Louis Kerofsky Image Coding and Communication Group Digital Video Department Sharp Labs of America

2. Goal “Really make wavelet methods work for video compression. While MPEG’s success and proliferation make medium-bitrate wavelet video a purely academic endeavor, low-bitrate compre- ssion remains a place where wavelet methods can prevail” W. Sweldens, “Wavelets: What Next?”, Proc. of IEEE, vol. 84, no. 4, pp. 685, 1996 “One of the critical issues to video coding that has been unfortunately ignored by our community is delay (latency)” Informal panel discussion during the session on internet video at VCIP’2001

3. What we already know about wavelets • Excellent energy compaction property in both the spatial and the frequency domain • What distinguishes wavelet coders from subband coders is the improved modeling and quantization strategy for wavelet coefficients s(n) d(n) Two channel perfect reconstruction filter bank

4. What we do not know about wavelets WT coefficients sx(n),dx(n) signal x(n) ? translational model WT coefficients sy(n),dy(n) signal y(n)=x(n+) wavelet spatial Challenge: How to exploit linear phase characteristics in the wavelet domain?

5. Phase Uncertainty frequency aliasing term Down-sampling operation introduces phase uncertainty

6. Toy Example 2 H0 x(n)   2 H1  2 H0 y(n)= x(n+1)   2 H1 introduces 1-bit phase uncertainty 2 It does not affect the coding of x(n) alone; but increases the difficulty of predicting y(n)

7. Phase-Shifting in the Wavelet Space se(n) e2 H0(z) x(n) de(n) H1(z) e2 so(n) o2 H0(z) do(n) H1(z) o2 Odd-phase and even-phase coefficients are linked to each other by the same phase shifting matrix T(z)

8. Phase Shifting Filter • Example: (Haar Filter)

9. All-Phase Motion-Compensated Prediction motion estimation and compensation current frame wavelet transform Phase (0,0) wavelet transform previous frame Phase (1,0) Phase shifting filter Phase (0,1) Phase (1,1) Motion field includes the optimal phase in addition to the optimal motion vector

10. Alternative Interpretation motion estimation and compensation current frame wavelet transform Phase (0,0) Wavelet transform without Decimation (overcomplete expansion) Phase (1,0) previous frame Phase (0,1) Phase (1,1) Since previous frame is also available at the decoder, overcomplete expansion does not require extra bits.

11. Multi-resolution All-Phase MCP lower resolution Phase Shifting Filter Inverse Wavelet Transform Inverse Wavelet Transform Motion Estimation higher resolution previous frame current frame

12. Efficiency Loss due to Resolution Scalability • Non-scalable coder Motion compensated prediction of LL band can employ the information contained in LH,HL and HH bands LH LL • Scalable coder HL HH Motion compensated prediction of LL band can NOT employ the information contained in LH,HL and HH bands

13. Trade Delay with Resolution

14. Experiment Results (I) : Rate-Distortion test sequence: flower-garden 100 frames

15. Experiment Results (II) : Delay Analysis