Download
slide1 n.
Skip this Video
Loading SlideShow in 5 Seconds..
Rate-Distortion Optimized Motion Estimation for Error Resilient Video Coding PowerPoint Presentation
Download Presentation
Rate-Distortion Optimized Motion Estimation for Error Resilient Video Coding

Rate-Distortion Optimized Motion Estimation for Error Resilient Video Coding

158 Views Download Presentation
Download Presentation

Rate-Distortion Optimized Motion Estimation for Error Resilient Video Coding

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Hua Yang and Kenneth Rose Signal Compression Lab ECE Department University of California Santa Barbara, USA Mar. 2005 Rate-Distortion Optimized Motion Estimation for Error Resilient Video Coding

  2. Outline • Motion estimation (ME) for coding efficiency • Conventional ME • Rate-constrained ME & rate-distortion (RD) optimized ME • Motion estimation for error resilience • Proposed end-to-end distortion based RDME • Intuition behind • End-to-end distortion analysis • Simulation results • Conclusions ICASSP 2005

  3. Coded frame n-1 Original frame n Motion Estimation for Coding Efficiency • Motion compensated prediction (MCP) • To remove inherent temporal redundancy of video signal • Both the motion vector and the prediction residue are encoded. ICASSP 2005

  4. Motion Estimation for Coding Efficiency • Conventional motion estimation • ME Criterion: minimize prediction residue • Ignoring the motion vector bit-rate cost ICASSP 2005

  5. Rate-constrained motion estimation : Lagrange multiplier • However, not yet the ultimate rate-distortion optimization for the best overall coding performance. Motion Estimation for Coding Efficiency • Motion estimation in low bit rate video coding • In low bit rate video coding, motion vectors may occupy a significant portion of total bit rate. • Efficient bit allocation between motion vector and prediction residue coding is necessary for better overall coding efficiency. ICASSP 2005

  6. Motion Estimation for Coding Efficiency • Motion estimation for low bit rate video coding (cont’d) • Rate-distortion optimized motion estimation (RDME) • Some references • [Girod `94] Theoretical analysis of rate-constrained ME • [Sullivan `98] Summary of rate-constrained ME • [Chung `96] Low complexity RDME for each MB using RD modeling • [Schuster `97] Joint RDME for multiple MB’s ICASSP 2005

  7. Error resilient video coding • RD optimization with end-to-end distortion • Coding mode selection: {Intra/Inter, QP} No mv for Inter-mode! Not comprehensively attack the RD optimization problem! Motion Estimation for Error Resilience • In the presence of packet loss: • Packet loss & error propagation • Internet – no QoS guarantee Wireless – inherent error-prone channel • Error propagation due to MCP • Error resilience via motion compensation • Multi-frame motion compensation (MFMC) [Budagavi `01] • Reference picture selection (RPS) [H.263+] • Error resilient rate-constrained ME [Wiegand `00] ICASSP 2005

  8. Motion Estimation for Error Resilience • We propose end-to-end distortion based RDME [accounting for packet loss] • The exact RD optimal ME solution for error resilience • Critical: accurate pixel-level end-to-end distortion estimation • Build on: recursive optimal per-pixel estimate (ROPE) [R. Zhang, S. Regunathan, and K. Rose `00] ICASSP 2005

  9. P1 I P2 P3 P1 P2 P4 P1 I For coding efficiency For error resilience Best trade-off • Conventional motion estimation completely ignores the error resilience information. • This error resilience information should be exactly considered for each pixel. Proposed RDME • Intuition for “error resilience via ME” ICASSP 2005

  10. Error concealment ROPE Error propagated distortion • DEP is explicitly affected by mv, whose minimization favors mv’s that point to reference areas with less encoder-decoder mismatch. Proposed RDME • ROPE-based end-to-end distortion analysis ICASSP 2005

  11. Packet loss impact Proposed RDME • The proposed RDME solution • Comparing with existent RDME • Source coding distortion  end-to-end distortion • mv affects not only the Rmv vs. Rres trade-off, but also more importantly, the coding efficiency vs. error resilience trade-off. • Comparing with existent RD optimized coding mode selection • Extended Inter mode with the mv parameter • Further optimize the Inter-mode performance ICASSP 2005

  12. Simulations • Objective: to check upper-bound performance • Joint {mv, QP} optimization • RD calculation via actual encoding • Simulation settings • UBC H.263+ • Encoding: I-P-P-…… • Transmission: independent packet loss, with a uniform p • Decoding: 50 different packet loss realizations for each p • Performance: average luminance PSNR ICASSP 2005

  13. Simulations • Simulation settings (cont’d) • Testing methods • Conventional ME (cME) • The proposed RDME (RDME) • Testing scenarios • Random Intra updating (rI): arbitrarily assigns MB’s to 1/p groups, and cycles through them updating one group per frame. • Optimal Intra updating (oI): RD optimized Intra/Inter mode selection. ICASSP 2005

  14. Simulation Results Random Intra Miss_am Foreman PSNR vs. Packet loss rate [QCIF, 10f/s, 48kb/s] ICASSP 2005

  15. Simulation Results Optimal Intra Miss_am Foreman PSNR vs. Packet loss rate [QCIF, 10f/s, 48kb/s] ICASSP 2005

  16. Simulation Results Random Intra Miss_am Foreman PSNR vs. Total bit rate [QCIF, 10f/s, p=10%] ICASSP 2005

  17. Simulation Results Optimal Intra Miss_am Foreman PSNR vs. Total bit rate [QCIF, 10f/s, p=10%] ICASSP 2005

  18. Simulation Results Conventional ME [29.58dB] RDME [33.83dB] Miss_am: QCIF, 10f/s, 48kb/s, p=10%, random Intra ICASSP 2005

  19. Simulation Results RDME [26.92dB] Conventional ME [23.92dB] Foreman: 1st 200f, QCIF, 10f/s, 112kb/s, p=10%, random Intra ICASSP 2005

  20. Besides Intra updating, RDME presents another good alternative for error resilience. Conclusions • Identify the new opportunity of achieving error resilience via motion estimation. • Propose an RD optimal ME solution, which further optimizes the Inter-mode performance. • Investigate the upper-bound performance. • With random Intra: substantial gain • With optimal Intra: significant gain at low bit rates. ICASSP 2005

  21. Conclusions • Originally, the power of Intra coded MB’s is only recognized as stopping past error propagation, while the proposed RDME reveals their new potential on reducing future error propagation. • Future work I: more comprehensive tests • Inaccurate p, bursty loss, or over actual networks, etc. • Future work II: complexity reduction • RD modeling, separate mv and QP optimization, sophisticated ME strategies, etc. ICASSP 2005

  22. References • [Girod `94] B. Girod, ``Rate-constrained motion estimation,'' Nov. 1994. • [Sullivan `98] G. J. Sullivan and T. Wiegand, ``Rate-distortion optimization for video compression,’’ Nov. 1998. • [Chung `96] W. C. Chung, F. Kossentini, and M. J. T. Smith, ``An efficient motion estimation technique based on a rate-distortion criterion,'' May 1996. • [Schuster `97] G. M. Schuster and A. K. Katsaggeslos, ``A theory for the optimal bit allocation between displacement vector field and displaced frame difference,'' Dec. 1997. • [Budagavi `01] M. Budagavi and J. D. Gibson, ``Multiframe video coding for improved performance over wireless channels,'' Feb. 2001. • [H.263+] ITU-T, Rec. H,263, ``Video codeing for low bitrate communications'', version 2 (H.263+), Jan. 1998. • [Wiegand `00] T. Wiegand, N. Farber, K. Stuhlmuller and B. Girod, ``Error-resilient video transmission using long-term memory motion-compensated prediction,'' June 2000. • [Zhang `00] R. Zhang, S. L. Regunathan, and K. Rose, ``Video coding with optimal intra/inter mode switching for packet loss resilience,'' June 2000. ICASSP 2005

  23. The End ICASSP 2005