1 / 49

Distributed Video System realized on mobile device with efficient Feedback channel

Distributed Video System realized on mobile device with efficient Feedback channel. 分散 式 影像編碼在手機上的實現與有效率的回饋通道. Chen,chun-yuan 陳群元 Advisor:Prof . Wu,Ja -Ling 吳家麟 2012/6/6. Outline. Motivation and introduction DVC architecture with mobile device Propose an efficient feedback channel

shay-kim
Download Presentation

Distributed Video System realized on mobile device with efficient Feedback channel

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Distributed Video System realized on mobile device with efficient Feedback channel 分散式影像編碼在手機上的實現與有效率的回饋通道 Chen,chun-yuan陳群元 Advisor:Prof. Wu,Ja-Ling 吳家麟 2012/6/6

  2. Outline • Motivation and introduction • DVCarchitecture with mobiledevice • Propose an efficient feedback channel • Transcoding • Experiment Result • Conclusion • Future work

  3. Motivation • Video communication is essential in Mobile device • Decrease the coding time and battery consuming on mobile.

  4. Motivation

  5. Introduction • For Mobile device video codec • Conventional video coding (ex:MPEG-4 H.264) • Heavy weight encoder, light weight decoder. • Distributed Video Coding • Light weight encoder, heavy weight decoder.

  6. Conventional video codec Decoding Encoding

  7. DVC to H.264 Transcoder Clouding server: DVC to H.264 transcoder Feedback channel Parity bits Encoded sequence H.264 decoder DVC encoder

  8. Channel Coding • Slepian-Wolf Theorem(1973) Wyner-Ziv Theorem(1976) • Channel • decoder • Channel encoder • Source encoder X • RY =H (X|Y) • H (Y) • H (X|Y) • H (X, Y) • Correlation is exploited by motion estimation Virtual channel -------Distributed------- • Side Information creation • RY =H (Y) Y • Source encoder • Source decoder

  9. Distributed Video Codec Decoded WZ Frames WZ Frames CRC-8 Mode selection CRC check CRC IDCT& Deblocking filter DCT Feedback channel LDPCA Encoder LDPCA Decoder WZ bitstream level Quantizer SI Reconstruction Buffer Bitplanes Correlation Noise Modeling DCT SI generation& refinement Conventional Intraframe Encoder Conventional Intraframe Encoder Key Frames Decoded Key Frames

  10. Our DVC codec: • DVC transcoder 1.Encoder no feedback channel ! 2.Decoder

  11. DVC to H.264 Transcoder Clouding server: DVC to H.264 transcoder Feedback channel Parity bits Encoded sequence H.264 decoder DVC encoder

  12. DVC Decoding Process • DVC Decoding without Feedback channel • Side-Information generation • LDPCA 88%

  13. DVC Decoding Process • DVC Decoding • Side-Information generation • Send-Request at Feedback channel • LDPCA 29% 66.6%

  14. Outline • Motivation and introduction • DVCarchitecture with mobiledevice • Propose an efficient feedback channel • Transcoding • Experiment Result • Conclusion • Future work

  15. DVC Encoder Realization • Implement DVC encoder on Mobile • Key frame encoding • WZ frame encoding • Efficient intra mode realization • Feedback channel transmission • Network connection between mobile and server • Determine payload of a packet with LDPCA protocal

  16. Key frame encoding on Mobile • Previous ver. use JM9.5 • Transcoder ref x.264 on PC Keyframe encoding Cross compiler x.264 lib

  17. DVC Encoder Realization • Implement DVC encoder on Mobile • Key frame encoding • WZ frame encoding • Efficient intra mode realization • Feedback channel transmission • Network connection between mobile and server • Determine payload of a packet with LDPCA protocal

  18. WZ coding with NDK Native code DVC encoder

  19. DVC Encoder Realization • Implement DVC encoder on Mobile • Key frame encoding • WZ frame encoding • Efficient intra mode realization • Feedback channel transmission • Network connection between mobile and server • Determine payload of a packet with LDPCA protocal

  20. Efficient intra block coding • Previous ver. Transcoder Use JM code. • Intra block encoding consumes lots of time in encoding. Clouding server: DVC to H.264 transcoder

  21. DVC Encoder Realization • Implement DVC encoder on Mobile • Key frame encoding • WZ frame encoding • Efficient intra mode realization • Feedback channel transmission • Network connection between mobile and server • Determine payload of a packet with LDPCA protocal

  22. Network connection • Create network connection to connect remote server. • TCP socketin C Clouding server: DVC to H.264 transcoder

  23. DVC Encoder Realization • Implement DVC encoder on Mobile • Key frame encoding • WZ frame encoding • Efficient intra mode realization • Feedback channel transmission • Network connection between mobile and server • Determine payload of a packet with LDPCA protocal

  24. ACK at Feedback channel TCP socket syndromes syndromes syndromes syndromes … Send syndrome Request

  25. Time consuming atFeedback channel • Packet header occupy large part of network bandwidth. • Network latency in communication at Feedback channel

  26. Outline • Motivation and introduction • DVCarchitecture with mobiledevice • Propose an efficient feedback channel • Transcoding • Experiment Result • Conclusion • Future work

  27. Efficient feedback channel • we propose two methods to decrease time consuming at feedback channel. • Estimate the syndromes size per WZ frame • Estimate the syndromes size per bitplane

  28. Number of requests per bitplane per band AC11 AC13 AC9 AC10 AC12 AC14 AC6 AC7 AC8 AC1 AC2 AC4 AC5 DC AC3 Bitplane Number

  29. Estimate the syndromes size per WZ frame • Estimate syndromes size by correspond WZ frame in previous GOP • Ex. GOP 4 WZ frame WZ frame WZ frame WZ frame Key frame Key frame WZ frame WZ frame Key frame Key frame Second GOP First GOP

  30. formula • (ESbt)WZn=(Sbt)WZ(n-GOPsize) • WZn means the WZ frame’s index • n-GOPsize means the correspond WZ frame in the previous GOP. • bt is the bitplane index. • (ESbt)WZn means the estimated syndromes bit for the bt’thbitplane of WZn frame. • (Sbt)WZ(n-GOPsize) means the syndromes

  31. Efficient feedback channel • we propose two methods to decrease time consuming at feedback channel. • Estimate the syndromes size per WZ frame • Estimate the syndromes size per bitplane

  32. Number of requests per bitplane per band AC11 AC13 AC9 AC10 AC12 AC14 AC6 AC7 AC8 AC1 AC2 AC4 AC5 DC AC3 Bitplane Number

  33. Number of requests per bitplane per band AC11 AC13 AC9 AC10 AC12 AC14 AC6 AC7 AC8 AC1 AC2 AC4 AC5 DC AC3 Bitplane Number

  34. Estimate by previous bands • 1.formula • 2.time ratio

  35. Number of requests per bitplane per band AC11 AC13 AC9 AC10 AC12 AC14 AC6 AC7 AC8 AC1 AC2 AC4 AC5 DC AC3 Bitplane Number

  36. formula • (ESacnbt)WZn= ( Sdcbt-1 )WZn • WZn means the n’th WZ frame. • acn means the n’th AC band. • bt means the bitplane index of this AC band. • Which ESacnbtmean the estimated syndrome size for the bt’thbitplane in n’th AC band. • dc means the DC band. • Sdcbt-1means the syndromes bits for bt-1’th bitplane in DC band.

  37. Outline • Motivation and introduction • DVCarchitecture with mobiledevice • Propose an efficient feedback channel • Transcoding • Experiment Result • Conclusion • Future work

  38. Transcode to mobile codec • Because DVC only encode luminance part in YUV, so we got only Y for output. • Our mobile device can only read mp4 and 3gp. Y G R B Frame 1 Frame 2 Frame 3 Frame4 Frame 5 Frame 6 Frame 7 U V .mp4 Frame n

  39. Outline • Motivation and introduction • DVCarchitecture with mobiledevice • Propose an efficient feedback channel • Transcoding • Experiment Result • Conclusion • Future work

  40. Experiment Result • 規格 • 實驗數據: • 時間(encode&decode)、packet數量(傳幾次)+ • RD • 使用x264對quality的影響(intra改quality應該會變，但變快)

  41. Outline • Motivation and introduction • DVCarchitecture with mobiledevice • Propose an efficient feedback channel • Transcoding • Experiment Result • Conclusion • Future work

  42. Contribution • Realize DVC encoding part on Mobile device • Implement Feedback channel • Implement encoder’s component with x264 • Propose an efficient Feedback channel • Transcode the decoded sequence to mobile codec

  43. Outline • Motivation and introduction • DVCarchitecture with mobiledevice • Propose an efficient feedback channel • Transcoding • Experiment Result • Conclusion • Future work

  44. Future Work • Take a more precise estimation of syndrome size. • Use motion information to estimate the syndrome size. • Build a reliable DVC codec which is loss tolerate

  45. Thank You

  46. To do • Real time decoder • Efficient intra coding • Print MSG on Mobile

  47. DVC decoding time • Side Info Creation: 0.85 sec • Overcomplete Transform: 2.35 sec • Correlation Noise Modeling: 2.33 sec • Conditional Bit Prob Compute: 0.60 sec • Update Side Info: 7.14 sec • Ldpca Decode: 123.34 sec • Motion Learning: 1.79 sec • Others: 1.80 sec

  48. Distributed Video Codec Decoded WZ Frames WZ Frames CRC-8 Mode selection CRC check CRC IDCT& Deblocking filter DCT Feedback channel LDPCA Encoder LDPCA Decoder WZ bitstream level Quantizer SI Reconstruction Buffer Bitplanes Correlation Noise Modeling DCT SI generation& refinement Conventional Intraframe Encoder Conventional Intraframe Encoder Key Frames Decoded Key Frames

  49. Outline • Motivation and introduction • DVCarchitecture with mobiledevice • Propose an efficient feedback channel • Transcoding • Experiment Result • Conclusion • Future work

More Related