1 / 75

Distributed Video System realized on mobile device with efficient Feedback channel

Distributed Video System realized on mobile device with efficient Feedback channel. 分散 式 影像編碼在手機上的實現與有效率的回饋通道. Chen, chun-yuan 陳群元 Advisor: Prof. Wu, Ja -Ling 吳家麟 教授 2012/7/9. Outline. Motivation and introduction DVC architecture overview DVC architecture with mobile device

tasya
Download Presentation

Distributed Video System realized on mobile device with efficient Feedback channel

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Distributed Video System realized on mobile device with efficient Feedback channel 分散式影像編碼在手機上的實現與有效率的回饋通道 Chen, chun-yuan陳群元 Advisor: Prof. Wu, Ja-Ling 吳家麟 教授 2012/7/9

  2. Outline • Motivation and introduction • DVC architecture overview • DVCarchitecture with mobiledevice • Proposed algorithm for efficient feedback channel • Experimental Results • Conclusion • Future work

  3. Motivation • Video communication is essential in Mobile device • Decrease the coding time and battery consuming on mobile.

  4. Introduction • For Mobile device video codec • Conventional video coding (ex:MPEG-4 H.264) • Heavy weight encoder, light weight decoder. • Distributed Video Coding • Light weight encoder, heavy weight decoder.

  5. Conventional video codec Decoding Encoding

  6. DVC to H.264 Transcoder Clouding server: DVC to H.264 transcoder LDPCA decoding is the bottleneck of the overall codec system. Feedback channel Encoded sequence Parity bits DVC encoder H.264 decoder Tse-Chung Su, Yun-Chung Shen , and Ja-Ling Wu , “Real-time decoding for LDPC based distributed video coding ,” MM '11 Proceedings of the 19th ACM international conference on Multimedia,, 2011.

  7. DVC codec baseline: • DISPAC+DVC transcoder 1.Encoder without an actual feedback channel ! 2.Decoder Tse-Chung Su. Yun-Chung Shen. and Ja-Ling Wu. 2011. Real-time Decoding for LDPC Based Distributed Video Coding. National Taiwan University

  8. Time complexity analysis without feedback channel • DVC Decoding without Feedback channel • Side-Information generation • LDPCA Total decoding time: 17.10 sec 31% Tse-Chung Su, Yun-Chung Shen , and Ja-Ling Wu , “Real-time decoding for LDPC based distributed video coding ,” MM '11 Proceedings of the 19th ACM international conference on Multimedia,, 2011.

  9. Time complexity analysis with feedback channel • DVC Decoding • Side-Information generation • Send-Request at Feedback channel • LDPCA Total decoding time: 446.83 sec 1% 96% Bottle neck! T. V. Lakshman and U. Madhow, “The performance of TCP/IP for networks with high bandwidth-delay products and random loss, IEEE/ACM Trans. Networking, vol. 6, pp. 336–350, June 1997.

  10. Outline • Motivation and introduction • DVC architecture Overview • DVCarchitecture with mobiledevice • Proposed algorithm for efficient feedback channel • Experimental Results • Conclusion • Future work

  11. Distributed Video Codec Decoded WZ Frames Key Frame Key Frame WZ Frames CRC-8 Mode selection CRC check CRC IDCT& Deblocking filter DCT Feedback channel LDPCA Encoder LDPCA Decoder WZ bitstream level Quantizer SI Reconstruction Buffer Bitplanes WZ Frame WZ Frame WZ Frame Correlation Noise Modeling DCT SI generation& refinement GOP size 4 Conventional Intraframe Encoder Conventional Intraframe Decoder Key Frames Decoded Key Frames

  12. LDPC decoding Side Information (real number) +  0 -  1 decode output hard decision a25 b25 c25 d25 e25 f25 g25 a1 b1 c1 d1 e1 f1 g1 a b c d e f g 3 4 7 1 5 6 2 Horizontal processing Vertical processing b c a 1 0 1 syndrome bits Kschischang, F.R., Frey, B.J., and Loeliger, H.-A. 2001. Factor graphs and the sum-product algorithm.IEEE Trans. Inform. Theory

  13. LDPC Accumulate (LDPCA) codes Rate adaptivity D. Varodayan et al., "Rate-adaptive codes for distributed source coding," EURASIP Signal Processing Journal, Special Section on Distributed Source Coding, 2006

  14. 65 LDPC codes

  15. Outline • Motivation and introduction • DVC architecture overview • DVCarchitecture with mobiledevice • Proposed algorithm for efficient feedback channel • Experimental Results • Conclusion • Future work

  16. Distributed Video Codec with mobile Decoded WZ Frames CRC-8 Decoded Video sequence With specific form Transcode level Quantizer Bitplanes CRC check CRC Video deliver IDCT& Deblocking filter Feedback channel DCT LDPCA Encoder LDPCA Decoder WZ bitstream SI Reconstruction Buffer Mode selection Correlation Noise Modeling DCT WZ Frames SI generation& refinement Conventional Intraframe Encoder Conventional Intraframe Decoder Key Frames Decoded Key Frames Android java coding

  17. Distributed Video Codec with mobile Decoded WZ Frames CRC-8 Decoded Video sequence With specific form Transcode level Quantizer Bitplanes CRC check CRC Video deliver IDCT& Deblocking filter Feedback channel DCT LDPCA Encoder LDPCA Decoder WZ bitstream SI Reconstruction Buffer Mode selection Correlation Noise Modeling DCT WZ Frames SI generation& refinement Conventional Intraframe Encoder Conventional Intraframe Decoder Key Frames Decoded Key Frames

  18. Key frame encoding on Mobile • Previous ver. use JM9.5 • Transcoder ref x264 on PC Keyframe encoding Cross compiler x264 static library x264 lib

  19. Distributed Video Codec with mobile Decoded WZ Frames CRC-8 Decoded Video sequence With specific form Transcode level Quantizer Bitplanes CRC check CRC Video deliver IDCT& Deblocking filter Feedback channel DCT LDPCA Encoder LDPCA Decoder WZ bitstream SI Reconstruction Buffer Mode selection Correlation Noise Modeling DCT WZ Frames SI generation& refinement Conventional Intraframe Encoder Conventional Intraframe Decoder Key Frames Decoded Key Frames

  20. WZ coding with NDK Jni Makefile Native code DVC encoder C/C++ code with NDK rule

  21. Distributed Video Codec with mobile Decoded WZ Frames CRC-8 Decoded Video sequence With specific form Transcode level Quantizer Bitplanes CRC check CRC Video deliver IDCT& Deblocking filter Feedback channel DCT LDPCA Encoder LDPCA Decoder WZ bitstream SI Reconstruction Buffer Mode selection Correlation Noise Modeling DCT WZ Frames SI generation& refinement Conventional Intraframe Encoder Conventional Intraframe Decoder Key Frames Decoded Key Frames

  22. Network connection • Create network connection to connect remote server. • TCP socketin C Clouding server: DVC to H.264 transcoder

  23. ACK at Feedback channel TCP socket syndromes syndromes syndromes syndromes … Send syndrome Request

  24. Decoding complexity onFeedback channel • Packet header occupy large part of network bandwidth. • 48 bits syndromes vs. 24 bytes header • Network latency in communication at Feedback channel • About 20 thousands times requests for 10 sec sequence.

  25. Outline • Motivation and introduction • DVCarchitecture with mobiledevice • DVC architecture overview • Proposed algorithm for efficient feedback channel • Experimental Results • Conclusion • Future work

  26. Efficient feedback channel • we propose two methods to decrease time consuming at feedback channel. • Estimate the syndrome size per bitplane • Estimate the syndrome size per WZ frame

  27. Number of requests per bitplane per band AC11 AC13 AC9 AC10 AC12 AC14 AC6 AC7 AC8 AC1 AC2 AC4 AC5 DC AC3 Bitplane Number

  28. Number of requests per bitplane per band AC11 AC13 AC9 AC10 AC12 AC14 AC6 AC7 AC8 AC1 AC2 AC4 AC5 DC AC3 Bitplane Number

  29. Number of requests per bitplane per band AC11 AC13 AC9 AC10 AC12 AC14 AC6 AC7 AC8 AC1 AC2 AC4 AC5 DC AC3 Bitplane Number

  30. Efficient feedback channel • we propose two methods to decrease time consuming at feedback channel. • Estimate the syndrome size per bitplane • Estimate the syndrome size per WZ frame

  31. Average number of requests per bitplane AC11 AC13 AC9 AC10 AC12 AC14 AC6 AC7 AC8 AC1 AC2 AC4 AC5 DC AC3 Bitplane Number

  32. Estimate the syndromes size per WZ frame • Estimate syndroms size by correspond WZ frame in previous GOP • Ex. GOP 4 WZ frame WZ frame WZ frame WZ frame Key frame Key frame WZ frame WZ frame Key frame Second GOP First GOP

  33. Outline • Motivation and introduction • DVC architecture overview • DVCarchitecture with mobiledevice • Proposed algorithm for efficient feedback channel • Experimental Results • Conclusion • Future work

  34. Test condition • 12 CPU, 24 processor • Intel(R) Xeon(R) CPU X5650 @ 2.67GHz • GPU: Tesla M2050 • Mobile device: HTC sensation

  35. Test materials Motion: High Low Soccer Foreman Coastguard Hall Monitor • Test sequences : • QCIF, 15Hz, all frames • GOP Size 2, 4 and 8 • Only luminance component is used

  36. Decoding time

  37. Request amount

  38. Speed up ratio

  39. Bitrate

  40. RD(Foreman)

  41. RD(hall)

  42. Outline • Motivation and introduction • DVC architecture overview • DVCarchitecture with mobiledevice • Proposed algorithm for efficient feedback channel • Experimental Results • Conclusion • Future work

  43. Conclusion • We build a realistic DVC system based on mobile and cloud. • Propose algorithm for Feedback channel which the previous work doesn’t discuss or improve. • Transcode the decoded sequence to mobile codec.

  44. Outline • Motivation and introduction • DVC architecture overview • DVCarchitecture with mobiledevice • Proposed algorithm for efficient feedback channel • Experimental Results • Conclusion • Future work

  45. Future Work • Take a more precise estimation of syndrome size. • Use motion information to estimate the syndrome size. • Build a reliable DVC codec which is loss tolerant.

  46. Thank You

  47. streaming Key Frame Key Frame Key Frame Key Frame WZ Frame WZ Frame WZ Frame WZ Frame WZ Frame WZ Frame GOP size 4 GOP size 4

  48. Key Frame Key Frame Key Frame Key Frame WZ Frame WZ Frame WZ Frame WZ Frame WZ Frame WZ Frame GOP size 4 GOP size 4

  49. To do • Real time decoder • Efficient intra coding • Print MSG on Mobile

  50. DVC decoding time • Side Info Creation: 0.85 sec • Overcomplete Transform: 2.35 sec • Correlation Noise Modeling: 2.33 sec • Conditional Bit Prob Compute: 0.60 sec • Update Side Info: 7.14 sec • Ldpca Decode: 123.34 sec • Motion Learning: 1.79 sec • Others: 1.80 sec

More Related