Speed up in feedback channel for a LDPCA base distributed video coding system on mobile device

1. Speed up in feedback channel for a LDPCA base distributed video coding system on mobile device 在手機裝置上對低密度奇偶校驗碼為基礎之分散式編碼中的回饋通道加速 Chen,chun-yuan陳群元Advisor: Prof. Wu, Ja-Ling 吳家麟 教授 2012/5/18

2. outline • Motivation and introduction • Mobile video trans • Traditional video codec • Distributed video codec • DVC architecture • Channel coding • Ldpca • The video communication system on Mobile device • Speed up DVC decoding time in feedback channel • Experiment Result • Conclusion

3. Digital Video Service Wireless sensor network mobile cameras phone mobile video conference Video surveillance Video compression is an essential component of broadcast and entertainment media Multimedia video everywhere!

5. outline • Motivation and introduction • Mobile video trans • Traditional video codec • Distributed video codec • DVC architecture • Channel coding • Ldpca • The video communication system on Mobile device • Speed up DVC decoding time in feedback channel • Experiment Result • Conclusion

6. Conventional Video Codec Heavyweight ENCODER Lightweight DECODER MPEG-2, H.264, HEVC(H.265)

7. Application of DVC Cloud Computational Resource H.264 encoded bitstream DVC to H.264 Transcoder DVC encoded bitstream H.264 decoder (Low Complexity) DVC encoder (Low Complexity) Video conferencing with mobile devices

8. outline • Motivation and introduction • Mobile video trans • Traditional video codec • Distributed video codec • DVC architecture • Channel coding • Ldpca • The video communication system on Mobile device • Speed up DVC decoding time in feedback channel • Experiment Result • Conclusion

9. Distributed Video Coding • Channel • decoder • H (Y) • H (X|Y) • Channel encoder • Source encoder X • RY =H (X|Y) • H (X, Y) • Correlation is exploited by motion estimation Virtual channel -------Distributed------- • Side Information creation • RY =H (Y) Y • Source encoder • Source decoder Slepian-Wolf Theorem(1973) Wyner-Ziv Theorem(1976)

10. Distributed Video Coding LDPC Encoder Channel Decoder LDPC Decoder Channel Encoder D. Varodayan, A. Aaron, and B. Girod, “Rate-Adaptive Codes for Distributed Source Coding,”EURASIP Signal Processing Journal, Special Issue on Distributed Source Coding,,November 2006.

11. Distributed Video Coding LDPC Encoder Channel Decoder LDPC Decoder Channel Encoder WZ frame WZ frame WZ frame Key frame Key frame Key frame WZ frame Key frame GOP size 2 GOP size 4 D. Varodayan, A. Aaron, and B. Girod, “Rate-Adaptive Codes for Distributed Source Coding,”EURASIP Signal Processing Journal, Special Issue on Distributed Source Coding,,November 2006.

12. S31 S21 S11 S12 S22 S32 S26 S36 S16 S17 S27 S37 S33 S23 S13 S15 S25 S35 S18 S28 S38 S213 S313 S113 S24 S34 S14 S29 S19 S39 S312 S212 S112 S114 S214 S314 S110 S210 S310 S111 S211 S311 S215 S115 S315 S216 S316 S116 Block1 Block3 Block2 Quantization … • DCT coefficient band b1 : { S11, S21, S31, …SN1 } • DCT coefficient band b2 : { S12, S22, S32, …SN2 } … • DCT coefficient band b16 : { S116, S216, S316, …SN16 } DC band AC bands DCT coefficients bands

13. Distributed Video Coding LDPC Encoder Channel Decoder LDPC Decoder Channel Encoder D. Varodayan, A. Aaron, and B. Girod, “Rate-Adaptive Codes for Distributed Source Coding,”EURASIP Signal Processing Journal, Special Issue on Distributed Source Coding,,November 2006.

14. Quantization Q8 Q8 2^5 2^4 2^3 2^2 2^7 2^6 2^5 2^4 2^6 2^5 2^4 2^3 2^4 2^3 2^2 2^2 2^3 2^2 2^2 2^5 2^4 2^3 2^2 2^2 2^2 2^4 2^3 2^2 63bits for one block 37bits for one block

15. Bit plane Extraction • For each DCT coefficient band… Bit planes of DC band: MSB 0 0 1 0 0 0 0 0 0 1 Bit plane 1: Bit plane 2: Independently Channel Encode (LDPCA) Bit plane 3: Zig-zag order 4 0 30 1 1 7 6 6 Bit plane 4: 7 7 3 5 0 0 0 0 0 1 1 1 1 0 Bit plane 5: LSB

16. outline • Motivation and introduction • Mobile video trans • Traditional video codec • Distributed video codec • DVC architecture • Channel coding • Ldpca • The video communication system on Mobile device • Speed up DVC decoding time in feedback channel • Experiment Result • Conclusion

17. LDPC decodingSum-Product Algorithm(Message Passing) Side Information (real number) +  0 -  1 decode output hard decision 4 6 7 1 2 3 5 a25 b25 c25 d25 e25 f25 g25 a1 b1 c1 d1 e1 f1 g1 a b c d e f g 甲 乙 丙 0 1 1 From DVC encoder (syndrome bits) Horizontal processing Vertical processing Kschischang, F.R., Frey, B.J., and Loeliger, H.-A. 2001. Factor graphs and the sum-product algorithm.IEEE Trans. Inform. Theory

18. LDPC Accumulate (LDPCA) codes Rate adaptivity D. Varodayan et al., "Rate-adaptive codes for distributed source coding," EURASIP Signal Processing Journal, Special Section on Distributed Source Coding, 2006

19. 65 LDPC codes 3

20. outline • Motivation and introduction • Mobile video trans • Traditional video codec • Distributed video codec • DVC architecture • Channel coding • Ldpca • The video communication system on Mobile device • Speed up DVC decoding time in feedback channel • Experiment Result • Conclusion

21. Distributed Video Coding LDPC Encoder Channel Decoder LDPC Decoder Channel Encoder D. Varodayan, A. Aaron, and B. Girod, “Rate-Adaptive Codes for Distributed Source Coding,”EURASIP Signal Processing Journal, Special Issue on Distributed Source Coding,,November 2006.

22. Distributed Video Coding 1’th bitstream&CRC WZ Bit stream Channel Decoder LDPC Decoder 63’th bitstream&CRC request Key frame D. Varodayan, A. Aaron, and B. Girod, “Rate-Adaptive Codes for Distributed Source Coding,”EURASIP Signal Processing Journal, Special Issue on Distributed Source Coding,,November 2006.

23. outline • Motivation and introduction • Mobile video trans • Traditional video codec • Distributed video codec • DVC architecture • Channel coding • Ldpca • The video communication system on Mobile device • Speed up DVC decoding time in feedback channel • Experiment Result • Conclusion

24. Amdahl's law 29% 66.6% • Maximum speedup can be reached by improving the most critical part of the system • Feedback channel in the DVC coding.

25. Amdahl's law 95.6% • Maximum speedup can be reached by improving the most critical part of the system • Feedback channel in the DVC coding.

26. Amdahl's law 1.7 X 92.8% • Maximum speedup can be reached by improving the most critical part of the system • Feedback channel in the DVC coding. • estimate syndromes size per bitplane

27. Amdahl's law 4.6X 82.8% • Maximum speedup can be reached by improving the most critical part of the system • Feedback channel in the DVC coding. • estimate syndromes size per WZ frame

28. Methods to speed up at feedback channel • So we propose two methods to decrease time consuming at feedback channel. • Estimate the syndromes size per WZ frame • Estimate the syndromes size per bitplane

29. Number of requests per bitplane per band AC11 AC13 AC9 AC10 AC12 AC14 AC6 AC7 AC8 AC1 AC2 AC4 AC5 DC AC3 Bitplane Number

30. Number of requests per bitplane per band AC11 AC13 AC9 AC10 AC12 AC14 AC6 AC7 AC8 AC1 AC2 AC4 AC5 DC AC3 Bitplane Number

31. Estimate by previous bands 1.formula 2.time ratio

32. Number of requests per bitplane per band AC11 AC13 AC9 AC10 AC12 AC14 AC6 AC7 AC8 AC1 AC2 AC4 AC5 DC AC3 Bitplane Number

33. Number of requests per bitplane per band AC11 AC13 AC9 AC10 AC12 AC14 AC6 AC7 AC8 AC1 AC2 AC4 AC5 DC AC3 Bitplane Number

34. Number of requests per bitplane per band AC11 AC13 AC9 AC10 AC12 AC14 AC6 AC7 AC8 AC1 AC2 AC4 AC5 DC AC3 Bitplane Number

35. formula (ESacnbt)WZn= ( Sdcbt-1 )WZn WZn means the n’th WZ frame. acn means the n’th AC band. bt means the bitplane index of this AC band. Which ESacnbtmean the estimated syndrome size for the bt’thbitplane in n’th AC band. dc means the DC band. Sdcbt-1means the syndromes bits for bt-1’th bitplane in DC band.

36. Amdahl's law 95.6% 406.39 s • Maximum speedup can be reached by improving the most critical part of the system • Feedback channel in the DVC coding.

37. Amdahl's law 1.7 X 92.8% 237.87 s • Maximum speedup can be reached by improving the most critical part of the system • Feedback channel in the DVC coding.

38. Estimate by reference frame 1.formula 2.time ratio

39. formula (ESbt)WZn=(Sbt)WZ(n-GOPsize) WZn means the WZ frame’s index n-GOPsize means the correspond WZ frame in the previous GOP. bt is the bitplane index. (ESbt)WZn means the estimated syndromes bit for the bt’thbitplane of WZn frame. (Sbt)WZ(n-GOPsize) means the syndromes

40. Amdahl's law 95.6% 406.39 s • Maximum speedup can be reached by improving the most critical part of the system • Feedback channel in the DVC coding.

41. Amdahl's law 4.6 X 82.8% 87.99 s • Maximum speedup can be reached by improving the most critical part of the system • Feedback channel in the DVC coding.

42. outline • Motivation and introduction • Mobile video trans • Traditional video codec • Distributed video codec • DVC architecture • Channel coding • Ldpca • The video communication system on Mobile device • Speed up DVC decoding time in feedback channel • Experiment Result • Conclusion

43. Test condition • 12 CPU, 24 processor • Intel(R) Xeon(R) CPU X5650 @ 2.67GHz • GPU: Tesla M2050 • Mobile device: HTC sensation

44. DVC encoder on mobile 14 s for foreman sequence, GOP8 , intra mode on. 4 s for foreman sequence, GOP8 , intra mode off.

45. Test materials Motion: High Low Soccer Foreman Coastguard Hall Monitor • Test sequences : • QCIF, 15Hz, all frames • GOP Size 2, 4 and 8 • Only luminance component is used

46. Speed up

47. Bitrate

48. RD curve Foreman with LRSS,GOP8

49. RD curve Hall monitor with LRSS,GOP8

50. outline • Motivation and introduction • Mobile video trans • Traditional video codec • Distributed video codec • DVC architecture • Channel coding • Ldpca • The video communication system on Mobile device • Speed up DVC decoding time in feedback channel • Experiment Result • Conclusion