Drift Management and Adaptive Bit Rate Allocation in Scalable Video Coding

1. Drift Management and Adaptive Bit Rate Allocation in Scalable Video Coding H. Yang, R. Zhang and K. Rose Signal Compression Lab ECE Department University of California, Santa Barbara ICIP2002

2. Outline • Introduction • ROPE for scalable coding • R-D optimized mode selection • Simulation results • Conclusions ICIP2002

3. Introduction • Scalable video coding • Drift problem • Multicast scenario & existent framework • Point-to-point scenario & proposed framework • Proposed coding approach ICIP2002

4. Scalable video coding and drift problem • Scalable video coding • Error resilience • Multiple QoS • Drift problem • Whether to use enhancement layer information for prediction • If used better prediction improve coding gain • If lost mismatch / error error propagation ICIP2002

5. Scalable video coding and drift problem • H.263 and MPEG4 favor no-drift system. • Drift management • Goal: Achieve a good trade-off. • Key : Accurately measure and thus effectively control the • amount of incurred error propagation. EI EP EP I P P ICIP2002

6. Multicast scenario & existent framework • Independent channels with different capacities • Some receivers have only access to the base layer, while others have access to both. • A coarse but acceptable base layer video quality is necessary. • Bit rates of different layers are determined by channel capacities. • Existent coding framework: EI EP EP I P P ICIP2002

7. Point-to-point scenario & proposed framework • Only one channel is considered. • Scalable coding only provides error resilience. • An acceptable base layer video quality is not necessary. • Bit rates of the layers don’t need to be specified before encoding. • Proposed coding framework: • Research Purposes: • How much we can gain by using the proposed framework. • Investigate the importance of accurate end-to-end distortion estimation in effective management of drift. EI EP EP I P P ICIP2002

8. Proposed coding approach • Macroblock(MB) based SNR scalable video coding • Objective: To minimize the expected end-to-end distortion given the packet loss rate and the total bit rate. • Drift management and adaptive bit rate allocation are fulfilled via R-D optimized coding mode selection for each MB. • To accurately estimate the end-to-end distortion, ROPE is adopted. ICIP2002

9. Proposed coding approach • Coding mode selection is an efficient means to optimize the tradeoff between coding efficiency and error resilience. Intra: Stop error propagation & most bits. Inter B B: No new error & less bits. Inter E B: New error & least bits. EI EP EP I P P ICIP2002

10. ROPE for Scalable Coding • Recursive Optimal per-Pixel Estimate (ROPE): • Take account of all the relevant factors as quatization, packet loss and error concealment. • Accurate & low complexity. • Adapt ROPE to scalable coding: • All the data of one frame is transmitted in one packet. • The channel is modeled as a Bernoulli process with packet loss only in the enhancement layer. ICIP2002

11. Overall expected decoder distortion of pixel i in frame n: Original value , Encoder reconstruction values , Decoder reconstruction values , Quantized prediction residues Packet loss rate of the enhancement layer ICIP2002

12. Calculation of and • Intra: • Inter B B: • Inter E B: ICIP2002

13. Calculation of and assuming upward error concealment • Intra: • Upward: • Inter E E: ICIP2002

14. RD Optimized Coding Mode Selection • Unconstrained minimization: • J can be independently minimized for each MB. • The coding mode and quantization step size of each MB are jointly selected. • Joint optimization: • Global optimal but with non-trivial complexity . . For simple retransmission, . ICIP2002

15. RD Optimized Coding Mode Selection • Sequential optimization: • Sub-optimal but with low complexity • For the base layer: • For the enhancement layer: ICIP2002

16. Simulation Results • UBC H.263+ codec with two-layer scalability. • Mean luminance PSNR: average first over the frames and then over the packet loss patterns. • QCIF sequences: Carphone and Salesman, first 150 frames, frame rate: 30 f/s, total bit rate: 300 kb/s. • 50 packet loss patterns. • Assuming simple retransmission: ICIP2002

17. (a) QCIF “Carphone” (b) QCIF “Salesman” Fig.1 PSNR Performance of different coding frameworks • Gain of “B&E drift” over “E drift” : 0.78dB~ 2.80 dB • Gain of “E drift” over “no drift”: 0.65 dB ~2.59dB • Sequential opt. captures much of the gain of the joint opt., while their complexity ratio is approximately 1:13. ICIP2002

18. (a) QCIF “Carphone” (b) QCIF “Salesman” Fig.2 Performance of different distortion estimation methods • “ROPE-RD” always largely outperforms “QDE-RD”. • At high packet loss rates, “QDE-RD” performs even worse than “no drift”. ICIP2002

19. Conclusions In the context of point-to-point video transmission over lossy networks: • Decoder drift due to prediction and packet loss should be controlled but not altogether disallowed. • Bit rates of different layers should be adaptively allocated for each frame. • Reaping the full benefits of drift management and adaptive bit rate allocation requires accurate estimation of end-to-end distortion. ICIP2002

20. Thank you! ICIP2002