Create Presentation
Download Presentation

Download Presentation
## Source-Channel Prediction in Error Resilient Video Coding

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Source-Channel Prediction in Error Resilient Video Coding**Hua Yang and Kenneth Rose Signal Compression Laboratory ECE Department University of California, Santa Barbara**Outline**• Introduction • Source-channel prediction • Simulation results • Conclusions ICME 2003**Introduction**• Existent error resilient approaches on the prediction mechanism • Slice coding • limit prediction within certain non-overlapping spatial regions • Video redundancy coding • Multiple independently predicted “threads” • Multi-frame motion compensation • Multiple reference frames for prediction • Feature in common: assume the same underlying conventional prediction framework • Framework: separate source-channel coding • Prediction: past encoder reconstructed frames • Motion estimation criterion: minimum prediction error ICME 2003**Introduction**• Via considering packet loss effects during encoding, joint source-channel coding usually achieves better error resilience than that of separate coding. • Our proposed approach • Prediction is based on expected decoder reconstruction of the previous frames. • Novelty Unlike all the other existent error resilient prediction schemes and all the other existent source-channel coding schemes, our proposed method is actually a source-channel prediction scheme. ICME 2003**Introduction**p1 p2 p3 0 pi packet loss rate of video packet i. 1-p1 1-p2 1-p3 Encoder reconstruction, i.e. “best possible” decoder reconstruction: quantization loss only. p3 p1 p2 1-p3 p3 1-p2 Other possible decoder reconstructions: different transmission loss patterns. 1-p3 p2 p3 1-p3 p3 Expected decoder reconstruction: quantization loss & transmission loss. ICME 2003**unknown**Random variable Introduction • Expected decoder reconstruction • Encoder’s estimate of the decoder reconstruction. • Given the packet loss rate, it can be accurately computed with the ROPE method. • Recursive optimal per-pixel estimate (ROPE) • Basic idea: • ROPE accurately computes these unknown quantities in a recursive manner for all the pixels of every frame. • Accurate & Low complexity • Frequently used to estimate end-to-end distortion in various RD optimization scenarios. Now we use these expectations for source-channel prediction. ICME 2003**Source-channel Prediction**• Conventional prediction Source-channel prediction Prediction residue For pixel i in frame n: Original and predicted values Encoder and decoder reconstruction values of pixel j in frame n-1 to predict pixel i in frame n. Prediction error to be quantized ICME 2003**Plug in:**Constant value: Not the actual predictor of the decoder Criterion I Source-channel Prediction • Source-channel prediction is the optimal prediction in the sense of minimum MSE end-to-end distortion. • Pending problem: motion estimation criterion ? Criterion in the conventional scheme ICME 2003**Criterion II**Random variable: Actual predictor of the decoder Source-channel Prediction • Pending problem: motion estimation criterion? (cont.) • Criterion II is superior than Criterion I in that it explicitly accounts for the randomness of the decoder’s actual predictor. ICME 2003**Source-channel Prediction**• Another interpretation of Criterion II While Criterion II considers the properly weighted impacts of both DR and DD , in contrast, Criterion I only considers DR . In this sense, Criterion II is more “comprehensive”. ICME 2003**Simulation Results**• Simulation conditions • H.263+ video codec • System performance: average luminance PSNR • 50 different packet loss patterns • Testing scenarios • No INTRA Updating • Periodic INTRA Updating For packet loss rate p, coding a MB in INTRA mode once for every 1/p frames. • R-D optimized INTRA Updating For each MB, select its coding mode as INTER or INTRA with the R-D criterion. ICME 2003**(a) No INTRA updating ( p = 10%)**(b) Periodic INTRA updating. (c) RD optimal INTRA updating. ICME 2003**Simulation Results**• Observations • The proposed “SCP_CII” method consistently offers the best performance, which proves our previous analysis. • When INTRA updating is more effectively performed, smaller gains are achieved by “SCP_CII” over “EP”. Hence, the gain depends on how much damage of packet loss is not accounted for in the conventional scheme. • Similar results also hold for other testing sequences, e.g., carphone, miss_am, salesman, etc. ICME 2003**Demo**Conventional prediction based on encoder reconstruction (PSNR = 25.06dB) Source-channel prediction based on expected decoder reconstruction (PSNR = 26.72dB) Foreman, QCIF, 30f/s, 300kb/s, packet loss rate = 10%, periodic Intra update. ICME 2003**Conclusions**• Novelty: the proposal of further enhancement of error resilience via fundamental modification of the conventional prediction structure. • Source-channel prediction based on expected decoder reconstruction, which uses ROPE to get accurate estimate of decoder quantities. • In spite of the loss in source coding gain due to the lower source prediction quality, our scheme achieves better overall R-D tradeoff than the conventional scheme. • We identify the subtle points in selecting the motion estimation criterion, and shows that it is advantageous to use the criterion of minimizing the expected prediction error. ICME 2003**Thanks!**ICME 2003