1 / 60

Distributed Video System realized on mobile device with efficient Feedback channel

Distributed Video System realized on mobile device with efficient Feedback channel. 分散 式 影像編碼在手機上的實現與有效率的回饋通道. Chen, chun-yuan 陳群元 Advisor: Prof. Wu, Ja -Ling 吳家麟 教授 2012/6/28. Outline. Motivation and introduction DVC architecture overview DVC architecture with mobile device

qiana
Download Presentation

Distributed Video System realized on mobile device with efficient Feedback channel

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Distributed Video System realized on mobile device with efficient Feedback channel 分散式影像編碼在手機上的實現與有效率的回饋通道 Chen, chun-yuan陳群元 Advisor: Prof. Wu, Ja-Ling 吳家麟 教授 2012/6/28

  2. Outline • Motivation and introduction • DVC architecture overview • DVCarchitecture with mobiledevice • Propose an efficient feedback channel • Transcoding • Experiment Result • Conclusion • Future work

  3. Motivation • Video communication is essential in Mobile device • Decrease the coding time and battery consuming on mobile.

  4. Introduction • For Mobile device video codec • Conventional video coding (ex:MPEG-4 H.264) • Heavy weight encoder, light weight decoder. • Distributed Video Coding • Light weight encoder, heavy weight decoder.

  5. Conventional video codec Decoding Encoding

  6. DVC to H.264 Transcoder Clouding server: DVC to H.264 transcoder Feedback channel Parity bits Encoded sequence H.264 decoder DVC encoder

  7. DVC codec baseline: • DISPAC+DVC transcoder 1.Encoder no feedback channel ! 2.Decoder Tse-Chung Su. Yun-Chung Shen. and Ja-Ling Wu. 2011. Real-time Decoding for LDPC Based Distributed Video Coding. National Taiwan University

  8. Time complexity analysis without feedback channel • DVC Decoding without Feedback channel • Side-Information generation • LDPCA Total decoding time: 17.10 sec 31%

  9. Time complexity analysis with feedback channel • DVC Decoding • Side-Information generation • Send-Request at Feedback channel • LDPCA Total decoding time: 330.51 sec 1% 95% Bottle neck!

  10. Outline • Motivation and introduction • DVC architecture Overview • DVCarchitecture with mobiledevice • Propose an efficient feedback channel • Transcoding • Experiment Result • Conclusion • Future work

  11. Distributed Video Codec Decoded WZ Frames Key Frame Key Frame WZ Frames CRC-8 Mode selection CRC check CRC IDCT& Deblocking filter DCT Feedback channel LDPCA Encoder LDPCA Decoder WZ bitstream level Quantizer SI Reconstruction Buffer Bitplanes WZ Frame WZ Frame WZ Frame Correlation Noise Modeling DCT SI generation& refinement GOP size 4 Conventional Intraframe Encoder Conventional Intraframe Encoder Key Frames Decoded Key Frames

  12. Outline • Motivation and introduction • DVC architecture overview • DVCarchitecture with mobiledevice • Propose an efficient feedback channel • Transcoding • Experiment Result • Conclusion • Future work

  13. Distributed Video Codec with mobile Decoded WZ Frames CRC-8 Decoded Video sequence With specific form Transcode level Quantizer Bitplanes CRC check CRC Video deliver IDCT& Deblocking filter Feedback channel DCT LDPCA Encoder LDPCA Decoder WZ bitstream SI Reconstruction Buffer Mode selection Correlation Noise Modeling DCT WZ Frames SI generation& refinement Conventional Intraframe Encoder Conventional Intraframe Encoder Key Frames Decoded Key Frames

  14. DVC Encoder Realization • Implement DVC encoder on Mobile • Key frame encoding • WZ frame encoding • Efficient intra mode realization • Feedback channel transmission • Network connection between mobile and server • Determine payload of a packet with LDPCA protocal

  15. Key frame encoding on Mobile • Previous ver. use JM9.5 • Transcoder ref x.264 on PC Keyframe encoding Cross compiler x.264 lib

  16. DVC Encoder Realization • Implement DVC encoder on Mobile • Key frame encoding • WZ frame encoding • Efficient intra mode realization • Feedback channel transmission • Network connection between mobile and server • Determine payload of a packet with LDPCA protocal

  17. WZ coding with NDK Native code DVC encoder

  18. DVC Encoder Realization • Implement DVC encoder on Mobile • Key frame encoding • WZ frame encoding • Efficient intra mode realization • Feedback channel transmission • Network connection between mobile and server • Determine payload of a packet with LDPCA protocal

  19. Efficient intra block coding • Previous ver. Transcoder Use JM code. • Intra block encoding consumes lots of time in encoding. Clouding server: DVC to H.264 transcoder

  20. DVC Encoder Realization • Implement DVC encoder on Mobile • Key frame encoding • WZ frame encoding • Efficient intra mode realization • Feedback channel transmission • Network connection between mobile and server • Determine payload of a packet with LDPCA protocal

  21. Network connection • Create network connection to connect remote server. • TCP socketin C Clouding server: DVC to H.264 transcoder

  22. DVC Encoder Realization • Implement DVC encoder on Mobile • Key frame encoding • WZ frame encoding • Efficient intra mode realization • Feedback channel transmission • Network connection between mobile and server • Determine payload of a packet with LDPCA protocal

  23. ACK at Feedback channel TCP socket syndromes syndromes syndromes syndromes … Send syndrome Request

  24. Consuming time atFeedback channel • Packet header occupy large part of network bandwidth. • Network latency in communication at Feedback channel

  25. Outline • Motivation and introduction • DVCarchitecture with mobiledevice • DVC architecture overview • Propose an efficient feedback channel • Transcoding • Experiment Result • Conclusion • Future work

  26. Efficient feedback channel • we propose two methods to decrease time consuming at feedback channel. • Estimate the syndromes size per WZ frame • Estimate the syndromes size per bitplane

  27. Average number of requests per bitplane AC11 AC13 AC9 AC10 AC12 AC14 AC6 AC7 AC8 AC1 AC2 AC4 AC5 DC AC3 Bitplane Number

  28. Estimate the syndromes size per WZ frame • Estimate syndromes size by correspond WZ frame in previous GOP • Ex. GOP 4 WZ frame WZ frame WZ frame WZ frame Key frame Key frame WZ frame WZ frame Key frame Key frame Second GOP First GOP

  29. formula • (ESbt)WZn=(Sbt)WZ(n-GOPsize) • WZn means the WZ frame’s index • n-GOPsize means the correspond WZ frame in the previous GOP. • bt is the bitplane index. • (ESbt)WZn means the estimated syndromes bit for the bt’thbitplane of WZn frame. • (Sbt)WZ(n-GOPsize) means the syndromes bit for the bt’thbitplane of WZ(n-GOPsize)frame

  30. Efficient feedback channel • we propose two methods to decrease time consuming at feedback channel. • Estimate the syndromes size per WZ frame • Estimate the syndromes size per bitplane

  31. Number of requests per bitplane per band AC11 AC13 AC9 AC10 AC12 AC14 AC6 AC7 AC8 AC1 AC2 AC4 AC5 DC AC3 Bitplane Number

  32. Number of requests per bitplane per band AC11 AC13 AC9 AC10 AC12 AC14 AC6 AC7 AC8 AC1 AC2 AC4 AC5 DC AC3 Bitplane Number

  33. Estimate by previous bands • 1.formula • 2.time ratio

  34. Number of requests per bitplane per band AC11 AC13 AC9 AC10 AC12 AC14 AC6 AC7 AC8 AC1 AC2 AC4 AC5 DC AC3 Bitplane Number

  35. formula • (ESacnbt)WZn= ( Sdcbt-1 )WZn • WZn means the n’th WZ frame. • acn means the n’th AC band. • bt means the bitplane index of this AC band. • Which ESacnbtmean the estimated syndrome size for the bt’thbitplane in n’th AC band. • dc means the DC band. • Sdcbt-1means the syndromes bits for bt-1’th bitplane in DC band.

  36. Outline • Motivation and introduction • DVC architecture overview • DVCarchitecture with mobiledevice • Propose an efficient feedback channel • Transcoding • Experiment Result • Conclusion • Future work

  37. Transcode to mobile codec • Because DVC only encode luminance part in YUV, so we got only Y for output. • Our mobile device can only read mp4 and 3gp. Y G R B Frame 1 Frame 2 Frame 3 Frame4 Frame 5 Frame 6 Frame 7 U V .mp4 Frame n

  38. Outline • Motivation and introduction • DVC architecture overview • DVCarchitecture with mobiledevice • Propose an efficient feedback channel • Transcoding • Experiment Result • Conclusion • Future work

  39. Experiment Result • 規格 • 實驗數據: • 時間(&decode)、packet數量(傳幾次)+

  40. Test condition • 12 CPU, 24 processor • Intel(R) Xeon(R) CPU X5650 @ 2.67GHz • GPU: Tesla M2050 • Mobile device: HTC sensation

  41. Test materials Motion: High Low Soccer Foreman Coastguard Hall Monitor • Test sequences : • QCIF, 15Hz, all frames • GOP Size 2, 4 and 8 • Only luminance component is used

  42. Speed up

  43. Bitrate

  44. RD curve • Foreman with LRSS,GOP8

  45. RD curve • Hall monitor with LRSS,GOP8

  46. Outline • Motivation and introduction • DVC architecture overview • DVCarchitecture with mobiledevice • Propose an efficient feedback channel • Transcoding • Experiment Result • Conclusion • Future work

  47. Conclusion • We combine DISPAC and transcoder, then port it’s encoder on mobile. • Implement and propose Feedback channel. • Record video with mobile device’s camera. • Transcode the decoded sequence to mobile codec.

  48. Outline • Motivation and introduction • DVC architecture overview • DVCarchitecture with mobiledevice • Propose an efficient feedback channel • Transcoding • Experiment Result • Conclusion • Future work

  49. Future Work • Take a more precise estimation of syndrome size. • Use motion information to estimate the syndrome size. • Build a reliable DVC codec which is loss tolerant.

  50. Thank You

More Related