Hua Yang and Kenneth Rose
Download
1 / 23

Rate-Distortion Optimized Motion Estimation for Error Resilient Video Coding - PowerPoint PPT Presentation


  • 148 Views
  • Uploaded on

Hua Yang and Kenneth Rose Signal Compression Lab ECE Department University of California Santa Barbara, USA Mar. 2005. Rate-Distortion Optimized Motion Estimation for Error Resilient Video Coding. Outline. Motion estimation (ME) for coding efficiency Conventional ME

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Rate-Distortion Optimized Motion Estimation for Error Resilient Video Coding' - zlata


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Hua Yang and Kenneth Rose

Signal Compression Lab

ECE Department

University of California Santa Barbara, USA

Mar. 2005

Rate-Distortion Optimized Motion Estimation for Error Resilient Video Coding


Outline
Outline

  • Motion estimation (ME) for coding efficiency

    • Conventional ME

    • Rate-constrained ME & rate-distortion (RD) optimized ME

  • Motion estimation for error resilience

  • Proposed end-to-end distortion based RDME

    • Intuition behind

    • End-to-end distortion analysis

  • Simulation results

  • Conclusions

ICASSP 2005


Motion estimation for coding efficiency

Coded frame n-1

Original frame n

Motion Estimation for Coding Efficiency

  • Motion compensated prediction (MCP)

    • To remove inherent temporal redundancy of video signal

    • Both the motion vector and the prediction residue are encoded.

ICASSP 2005


Motion estimation for coding efficiency1
Motion Estimation for Coding Efficiency

  • Conventional motion estimation

    • ME Criterion: minimize prediction residue

      • Ignoring the motion vector bit-rate cost

ICASSP 2005


Motion estimation for coding efficiency2

: Lagrange multiplier

  • However, not yet the ultimate rate-distortion optimization for the best overall coding performance.

Motion Estimation for Coding Efficiency

  • Motion estimation in low bit rate video coding

    • In low bit rate video coding, motion vectors may occupy a significant portion of total bit rate.

  • Efficient bit allocation between motion vector and prediction residue coding is necessary for better overall coding efficiency.

ICASSP 2005


Motion estimation for coding efficiency3
Motion Estimation for Coding Efficiency

  • Motion estimation for low bit rate video coding (cont’d)

    • Rate-distortion optimized motion estimation (RDME)

  • Some references

    • [Girod `94] Theoretical analysis of rate-constrained ME

    • [Sullivan `98] Summary of rate-constrained ME

    • [Chung `96] Low complexity RDME for each MB using RD modeling

    • [Schuster `97] Joint RDME for multiple MB’s

ICASSP 2005


Motion estimation for error resilience

No mv for Inter-mode!

Not comprehensively attack the RD optimization problem!

Motion Estimation for Error Resilience

  • In the presence of packet loss:

    • Packet loss & error propagation

      • Internet – no QoS guarantee

        Wireless – inherent error-prone channel

      • Error propagation due to MCP

  • Error resilience via motion compensation

    • Multi-frame motion compensation (MFMC) [Budagavi `01]

    • Reference picture selection (RPS) [H.263+]

    • Error resilient rate-constrained ME [Wiegand `00]

ICASSP 2005


Motion estimation for error resilience1
Motion Estimation for Error Resilience

  • We propose end-to-end distortion based RDME

    [accounting for packet loss]

    • The exact RD optimal ME solution for error resilience

    • Critical:

      accurate pixel-level end-to-end distortion estimation

      • Build on: recursive optimal per-pixel estimate (ROPE)

        [R. Zhang, S. Regunathan, and K. Rose `00]

ICASSP 2005


Proposed rdme

P1

I

P2

P3

P1

P2

P4

P1

I

For coding efficiency

For error resilience

Best trade-off

  • Conventional motion estimation completely ignores the error resilience information.

  • This error resilience information should be exactly considered for each pixel.

Proposed RDME

  • Intuition for “error resilience via ME”

ICASSP 2005


Proposed rdme1

Error concealment

ROPE

Error propagated distortion

  • DEP is explicitly affected by mv, whose minimization favors mv’s that point to reference areas with less encoder-decoder mismatch.

Proposed RDME

  • ROPE-based end-to-end distortion analysis

ICASSP 2005


Proposed rdme2

Packet loss impact

Proposed RDME

  • The proposed RDME solution

  • Comparing with existent RDME

    • Source coding distortion  end-to-end distortion

    • mv affects not only the Rmv vs. Rres trade-off, but also more importantly, the coding efficiency vs. error resilience trade-off.

  • Comparing with existent RD optimized coding mode selection

    • Extended Inter mode with the mv parameter

    • Further optimize the Inter-mode performance

ICASSP 2005


Simulations
Simulations

  • Objective: to check upper-bound performance

    • Joint {mv, QP} optimization

    • RD calculation via actual encoding

  • Simulation settings

    • UBC H.263+

    • Encoding: I-P-P-……

    • Transmission: independent packet loss, with a uniform p

    • Decoding: 50 different packet loss realizations for each p

    • Performance: average luminance PSNR

ICASSP 2005


Simulations1
Simulations

  • Simulation settings (cont’d)

    • Testing methods

      • Conventional ME (cME)

      • The proposed RDME (RDME)

    • Testing scenarios

      • Random Intra updating (rI):

        arbitrarily assigns MB’s to 1/p groups, and cycles through them updating one group per frame.

      • Optimal Intra updating (oI):

        RD optimized Intra/Inter mode selection.

ICASSP 2005


Simulation results random intra
Simulation Results Random Intra

Miss_am

Foreman

PSNR vs. Packet loss rate

[QCIF, 10f/s, 48kb/s]

ICASSP 2005


Simulation results optimal intra
Simulation Results Optimal Intra

Miss_am

Foreman

PSNR vs. Packet loss rate

[QCIF, 10f/s, 48kb/s]

ICASSP 2005


Simulation results random intra1
Simulation Results Random Intra

Miss_am

Foreman

PSNR vs. Total bit rate

[QCIF, 10f/s, p=10%]

ICASSP 2005


Simulation results optimal intra1
Simulation Results Optimal Intra

Miss_am

Foreman

PSNR vs. Total bit rate

[QCIF, 10f/s, p=10%]

ICASSP 2005


Simulation results
Simulation Results

Conventional ME [29.58dB]

RDME

[33.83dB]

Miss_am: QCIF, 10f/s, 48kb/s, p=10%, random Intra

ICASSP 2005


Simulation results1
Simulation Results

RDME

[26.92dB]

Conventional ME

[23.92dB]

Foreman: 1st 200f, QCIF, 10f/s, 112kb/s, p=10%, random Intra

ICASSP 2005


Conclusions

Conclusions

  • Identify the new opportunity of achieving error resilience via motion estimation.

  • Propose an RD optimal ME solution, which further optimizes the Inter-mode performance.

  • Investigate the upper-bound performance.

    • With random Intra: substantial gain

    • With optimal Intra: significant gain at low bit rates.

ICASSP 2005


Conclusions1
Conclusions alternative for error resilience.

  • Originally, the power of Intra coded MB’s is only recognized as stopping past error propagation, while the proposed RDME reveals their new potential on reducing future error propagation.

  • Future work I: more comprehensive tests

    • Inaccurate p, bursty loss, or over actual networks, etc.

  • Future work II: complexity reduction

    • RD modeling, separate mv and QP optimization, sophisticated ME strategies, etc.

ICASSP 2005


References
References alternative for error resilience.

  • [Girod `94] B. Girod, ``Rate-constrained motion estimation,'' Nov. 1994.

  • [Sullivan `98] G. J. Sullivan and T. Wiegand, ``Rate-distortion optimization for video compression,’’ Nov. 1998.

  • [Chung `96] W. C. Chung, F. Kossentini, and M. J. T. Smith, ``An efficient motion estimation technique based on a rate-distortion criterion,'' May 1996.

  • [Schuster `97] G. M. Schuster and A. K. Katsaggeslos, ``A theory for the optimal bit allocation between displacement vector field and displaced frame difference,'' Dec. 1997.

  • [Budagavi `01] M. Budagavi and J. D. Gibson, ``Multiframe video coding for improved performance over wireless channels,'' Feb. 2001.

  • [H.263+] ITU-T, Rec. H,263, ``Video codeing for low bitrate communications'', version 2 (H.263+), Jan. 1998.

  • [Wiegand `00] T. Wiegand, N. Farber, K. Stuhlmuller and B. Girod, ``Error-resilient video transmission using long-term memory motion-compensated prediction,'' June 2000.

  • [Zhang `00] R. Zhang, S. L. Regunathan, and K. Rose, ``Video coding with optimal intra/inter mode switching for packet loss resilience,'' June 2000.

ICASSP 2005


The End alternative for error resilience.

ICASSP 2005


ad