1 / 48

Subrahmanya M V (Under the guidance of Dr. Rao and Dr.Jin-soo Kim)

Subrahmanya M V (Under the guidance of Dr. Rao and Dr.Jin-soo Kim). Contents. Traditional Video coding Wyner-Ziv video coding Applications Wyner-Ziv encoder Key frame encoding WZ frame encoding Wyner-Ziv decoder Key frame decoding WZ frame decoding. Contents( contd.).

Download Presentation

Subrahmanya M V (Under the guidance of Dr. Rao and Dr.Jin-soo Kim)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Subrahmanya M V (Under the guidance of Dr. Rao and Dr.Jin-soo Kim)

  2. Contents • Traditional Video coding • Wyner-Ziv video coding • Applications • Wyner-Ziv encoder • Key frame encoding • WZ frame encoding • Wyner-Ziv decoder • Key frame decoding • WZ frame decoding

  3. Contents( contd.) • Side Information Generation • Inter-prediction • Intra-prediction • Motion estimation (ME) • Full-pixel • Half-pixel • Quarter-pixel • Motion compensation (MC) • Luma • Chroma • Bi-directional interpolation • Intra-prediction

  4. Contents( contd.) • Implementation • Testing • Results • SI prediction schemes • ME block sizes • ME search ranges • Key frame distances • Bit rates • Conclusions • References

  5. Traditional Video coding • Source coding • Encoder removes temporal and spatial redundancies • Intra frame encoding • Inter frame encoding: Computationally intensive • Complex Encoder + Simple decoder • [Dr. Jin-soo Kim]

  6. Wyner-Ziv video coding • Source coding + Channel coding • Decoding with side-information(SI) • Slepian-Wolf coding: Lossless • Wyner-Ziv coding: Lossy • Simple encoder + Complex decoder • [Dr. Jin-soo Kim]

  7. Applications • Light encoder and light decoder [20] • Wireless low power video surveillance • Sensor network • Multi-view image acquisition • Networked camcoders • [Dr. Jin-soo Kim]

  8. Wyner-Ziv Encoder • Input YUV 420 • Output: Bitstream • Encoding Key frame + WZ frame

  9. Key frame encoding • H.264 Intra frame coding • Intra prediction • Integer Transform • Quantization • Entropy Coding • [M. Lee, A. Moore]

  10. Intra prediction • Predicted from top, left, top left and top right neighbors • 4x4 intra prediction • Nine directional modes [T. Wiegand and G.J Sullivan] [J. S. Park and H. J. Song]

  11. Key frame encoding • Integer Transform • Quantization • Element by element Multiplication • Entropy Coding using • Exp-Golomb codes

  12. WZ frame encoding • Residual signal Generation • [Dr. Jin-soo Kim] Residual Signal

  13. WZ frame encoding(contd.) • Residual bit plane extraction

  14. WZ frame encoding(contd.) • Low Density Parity Check(LDPC) encoding • Obtain parity bits for the message bits • Message bits(90%) + Parity bits(10%) • Only parity bits are sent to achieve compression

  15. Wyner-Ziv Decoder • Decoding Key frame + WZ frame Key Frame WZ Frame [E. Peixoto, R. L. de Queiroz, and D. Mukherjee]

  16. Wyner-Ziv Decoder( Contd. ) • Key frame decoding • Intra prediction • Entropy decoding • Inverse Quantization • Inverse Integer transform

  17. Key frame decoding • Inverse Integer Transform • Inverse Quantization • Element by Element Multiplication • Entropy decoding • Exp-Golomb codes are decoded using prefix tables

  18. Wyner-Ziv Decoder( Contd. ) • WZ frame decoding • SI generation: Prediction to obtain the message bits • LDPC decoding: Correct prediction errors using parity bits • Message contains a pair of values [qij(0), qij(1)] or [rij(0), rij(1)] – Signify the amount of belief that yiis a ”0” or a ”1”. • Initial belief is based on message bits predicted and parity bits sent • Belief is iteratively updated and new message bits are obtained • Iteration proceeds till fixed iteration count or message bits remain unchanged in an iteration

  19. Side Information Generation • Motion estimation • Estimate motion of a block between two key frames to obtain motion vectors( MVx, MVy ) for each block • Interpolate motion vectors to obtain motion between key frame and WZ frame • Motion Compensation • Interpolate key frame based on motion vectors to obtain MC prediction for WZ frame • Intra prediction • Prediction for blocks which are absent in key frames

  20. Motion Estimation • Forward ME between key frames – MVF • Backward ME between key frames - MVB • Derive MVs for WZ frame using Key frame MVS [E. Peixoto, R. L. de Queiroz, and D. Mukherjee 2008]

  21. Higher key frame distance • For larger key frame distance hierarchical structure can be used [A. Aaron, E. Setton and B. Girod 2003]

  22. SAD based ME • Obtain absolute difference between each pixel in the block to be searched and the reference block • Best match is a block with least sum of absolute differences (W+2SR) X (H+2SR)

  23. Sub-pixel ME • Motion vector is estimated up to quarter-pixel units • Interpolation of reference is done to obtain half-pixel and quarter-pixel units • Half-pixel positions are obtained using five tap filter • Quarter pixels are obtained using bilinear interpolation

  24. Half-pixel motion estimation • H33 = [F13 + -5 * F23 + 20 * F33 + 20 * F43 + -5 * F53 + F63 + 15] >> 5 • G33 = [F31 + -5 * F32 + 20 * F33 + 20 * F34 + -5 * F35 + F36 + 15] >> 5 • D33 = [H31 + -5 * H32 + 20 * H33 + 20 * H34 + -5 * H35 + H36 + 15] >> 5

  25. quarter-pixel motion estimation q1 = ( F33 + G33 + 1 ) >> 1 q2 = ( G33 + F34 + 1 ) >> 1 q3 = ( F33 + H33 + 1 ) >> 1 q4 = ( H33 + G33 + 1 ) >> 1 q5 = (G33 + D33 + 1 ) >> 1 q6 = ( G33 + H34 + 1 ) >> 1 q7 = ( H33 + D33 + 1 ) >> 1 q8 = ( D33 + H34 + 1 ) >> 1 q9 = ( H33 + F43 + 1 ) >> 1 q10 = ( H33 + G43 + 1 ) >> 1 q11 = ( D33 + G43 + 1 ) >> 1 q12 = ( G43 + H34 + 1 ) >> 1

  26. Motion Compensation (Luma) • Obtain a block from key frame with offset specified by motion vector

  27. Motion Compensation (Chroma) • Chroma MV = Luma MV / 2 • Obtain A, B, C, D based on integer part of MV • Floor(MV/8) • Predict based on fractional part of MV • (MV & 7)

  28. Bi-directional interpolation SI frame = (Forward prediction Frame + Backward Prediction frame + 1)/2

  29. Intra prediction • 4x4 prediction • Prediction mode of co-located block from previous key frame

  30. Implementation • Encoder • Key frame encoding is done using JM encoder • WZ frame decoding is implemented in ‘C’ • Residual generation • Quantization • Bit plane extraction • LDPC encoder • Decoder • Key frame is decoded using JM decoder • WZ frame decoding is implemented in ‘C’ • SI generation • LDPC decoder • De-quantization • Reconstruction • LDPC encoder and decoder are implemented by Dr. Jin-soo Kim [4]

  31. Testing • Comparison with traditional coding • H.264 is used for comparison • Generate H.264 reconstructed sequence • Encode using H.264 encoder • Decode using H.264 decoder to obtain reconstructed YUV sequence • Generate Wyner-Ziv reconstructed sequence • Encode using Wyner-Ziv encoder • Decode using Wyner-Ziv decoder to obtain reconstructed YUV sequence • Compare two sequences using PSNR and SSIM • Significance of different parameters in Wyner-Ziv coding • SI Prediction method • Key frame distance • ME search range • ME block size

  32. SI prediction schemes(Sample reconstructed images) • bus( CIF) [Average, Full pixel ME, Half-pixel ME, Quarter-pixel ME, Bi-prediction, Intra-prediction combined]

  33. SI prediction schemes( Plots)

  34. ME block sizes(Sample reconstructed images) • coastgaurd (CIF) [16x16, 8x8, 4x4]

  35. ME block sizes( Plots)

  36. ME search ranges (Sample reconstructed images) • stefan (CIF) [16, 8, 4]

  37. ME search ranges( Plots)

  38. Key frame distances (Sample reconstructed images) • garden (CIF) [1, 2, 3, 4, 5]

  39. Key frame distances( Plots)

  40. Comparison with H.264 (Sample reconstructed images)

  41. Comparison with H.264( Plots)

  42. Conclusions • H.264 based SI generation based on quarter-pixel ME performs better compared to previously done H.263which is based on half-pixel ME[1] . • ME block size 16x16 is ideal • Higher ME search range produces better results • At higher key frame distance quality drops sharply • Traditional video coding has better R-D performance

  43. Future work • Wyner-Ziv to H.264 transcoder • Light encoder + light decoder • Inter and intra prediction done in Wyner-Ziv decoder can be reused for the H.264 encoder

  44. Accepted conference paper SubrahmanyaVenkatrav and Dr. K. R. Rao, “Side information generation in Wyner-Ziv decoder”, The 7th International Joint Conference on Computer Science and Software Engineering May 13-14, 2010 Bangkok, Thailand

  45. References • E. Peixoto, R. L. de Queiroz, and D. Mukherjee, “Mobile video communications using a Wyner-Ziv transcoder,” Proc. SPIE 6822, VCIP, 68220R Jan. 2008. • A. Aaron, E. Setton and B. Girod, "Towards practical Wyner-Ziv coding of video," Proceedings. IEEE International Conference on Image Processing, 2003. ICIP 2003., vol.3, pp. III-869-872, 14-17 Sept. 2003. • K. R. Rao and J. J. Hwang, Techniques and standards for image, video, and audio coding, Prentice Hall PTR, 1996. • Jin-Soo Kim, "Brief overview of Wyner-Ziv CODEC" (Private Communication) • A. Aaron, D. Varodayan, and B. Girod, “Wyner-Ziv residual coding of video,” Proc. International Picture Coding Symposium, Beijing, P. R. China , April 2006. • T. Wiegand and G.J Sullivan, “The H.264/AVC video coding standard”, IEEE SP Magazine, vol. 24, pp. 148-153, March 2007. • G. J. Sullivan, P. Topiwala, and A. Luthra, "The H.264/AVC advanced video coding standard: Overview and introduction to the fidelity range extensions", SPIE Conf. on applications of digital image processing XXVII, vol. 5558, pp. 53-74, Aug. 2004. • S.K. Kwon, A. Tamhankar, and K.R. Rao “Overview of H.264/MPEG-4 Part 10” J. VCIR, Vol. 17, pp. 186-216, April 2006, Special Issue on “Emerging H.264/AVC Video Coding Standard,”.

  46. References(Contd) • A. Wyner and J. Ziv, "The rate-distortion function for source coding with side information at the decoder,"  IEEE Trans., Information Theory, vol.22, pp. 1-10, Jan 1976. • D. Slepian and J. Wolf, “Noiseless coding of correlated information sources,” IEEE Trans. on Information Theory Vol. 19, pp. 471–480, July 1973. • D. Varodayan, A. Aaron and B. Girod, "Rate-adaptive distributed source coding using low-density parity-check codes,"  Conference Record of the Thirty-Ninth Asilomar Conference on Signals, Systems and Computers, pp. 1203-1207, Oct. 28 – Nov. 1, 2005. • Z. Li and E.J. Delp, "Wyner-Ziv video side estimator: conventional motion search methods revisited," IEEE International Conference on Image Processing, 2005. ICIP 2005, vol.1, pp. I-825-828, 11-14 Sept. 2005. • L. Liu and E. J. Delp, "Wyner-Ziv video coding using LDPC codes," Proceedings of the 7th Nordic Signal Processing Symposium, 2006. NORSIG 2006. •  D. Kubasov, K. Lajnef and C. Guillemot, "A hybrid encoder/decoder rate control for Wyner-Ziv video coding with a feedback channel," IEEE 9th Workshop on Multimedia Signal Processing, 2007. MMSP 2007., pp.251-254, 1-3 Oct. 2007. • C. Brites and F. Pereira, "Encoder rate control for transform domain Wyner-Ziv video coding," IEEE International Conference on Image Processing, 2007. ICIP 2007., vol.2, pp.II -5-II -8, 16-19 Sept. 2007 • A. Roca, et al, "Rate control algorithm for pixel-domain Wyner-Ziv video coding ," Proc. SPIE, vol. 6822, 68221T (2008).

  47. References(Contd) • D. Mukherjee, “Optimal parameter choice for Wyner-Ziv coding of Laplacian sources with decoder side information,” HP Labs Technical Report HPL-2007-34, 2007. • Z. Wang, et al, "Image quality assessment: From error visibility to structural similarity," IEEE Trans., Image Processing, vol.13, pp. 600-612, April 2004 • I. Richardson, “The H.264 advanced video compression standard,” Hoboken, NJ: Wiley, 2010. • D. Rebollo-Monedero, S. Rane, A. Aaron and B. Girod, "High-rate quantization and transform coding with side information at the decoder," EURASIP Signal Processing Journal, Special Issue on Distributed Source Coding. • S.-K Kwon, A. Tamhankar and K.R. Rao, ‘Overview of H.264 / MPEG-4 Part 10” ISME, Hong Kong, Oct. 2004. • JVT documents ftp://standards.polycom.com http://ftp3.itu.ch/av-arch/jvt-site/ • M. Lee and A. Moore, “H.264 Encoder Design“, Group 3, May 17, 2006 • J. S. Park and H. J. Song, “Selective Intra Prediction Mode Decision for H.264 /AVC Encoders”, World Academy of Science, Engineering and Technology 13 2006. • B. M.J. Leiner, “LDPC Codes – a brief Tutorial”, April 8, 2005

  48. References(Contd) • G. Cote, B. Erol, M. Gallant, and F. Kossentini, “H.263+: Video coding at low bit rates,” IEEE Trans. Circuits and Systems for Video Technology ,Vol.8, pp. 849–866, Nov. 1998 • (2008, August) H.264/avc JM reference software. Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG. [Online]. Available: http://iphome.hhi.de/suehring/tml/

More Related