Optimal Quality Adaptation for MPEG-4 Fine-Grained Scalable Video

1. Optimal Quality Adaptation for MPEG-4 Fine-Grained Scalable Video Taehyun Kim and Mostafa H. Ammar College of Computing, Georgia Institute of Technology

2. Problems • Rate smoothing is not useful for a best effort network, since the Internet does not provide any information about the bandwidth evolution in advance. • A smooth data rate does not always guarantee a smooth quality for VBR video.

3. Problems • Frequent adding and dropping of layers can incur significant quality variability. • Quality Adaptation for minimizing the perceptual video quality by using bidirectional optimum layer selection.

4. Problems • Small time scale variability • Receiver buffer • Large time scale variability • Scalable (Layered) video encoding

5. Goal • Trying to accommodate the mismatch caused by both the available bandwidth variability and the encoded video variability. • To develop an optimal algorithm that minimizes the quality variability while at the same time maximizing the utilization of the variable network bandwidth.

6. Rate variability in MPEG-4 FGS A river runs through it , GOP=12 Base Layer FGS Layer (SNR) .Fixed quantization step size .Max variation = 33.6 kBytes .Max variation = 7.4 kBytes

7. Rate variability in MPEG-4 FGS FGST Layer (SNR) .Max variation = 29.9 kBytes

8. Two hybrid temporal-SNR scalability structures FGS-FGST

9. Two hybrid temporal-SNR scalability structures FGST-FGS

10. Video Quality – Base Layer 100 VOPs, 45 dB

11. Video Quality – Base+FGS Layer 100 VOPs, improved more than 20 dB

12. Video Quality – Base+FGST Layer 300 VOPs, inconsistent quality

13. Video Quality – All Layer 300 VOPs, 67.3 dB

14. Quality Adaptation Algorithm • Quality adaptation is defined by a mechanism that adds and drops layers based on the available network bandwidth while maximizing the perceptual video quality. • Consistent “long runs” of the same quality video.

15. Composed Algorithm • The quality smoothing algorithm proposed in [13] accomplishes the maximum reduction of quality variability for layered CBR video using bidirectional layer selection. • Rate smoothing algorithm presented in [18] enables a sender to transmit a piecewise CBR sequence by using the work-ahead smoothing technique.

16. Composed Algorithm N: Number of VOPs L: Number of layers Sik : a feasible sequence of layer i xi[k]: size of VOP k

17. Optimal Quality Adaptation The receiver buffer size for storing unplayed i -th layer video The cumulative capacity The cumulative selected data defined by The VOP size of i -th layer, at time k The available bandwidth : the residual bandwidth after accommodating layers 1, 2, …, i-1. i -th layer, at time k .

18. Framework of quality adaptation No display video Display buffered video + prefetch

19. Framework of quality adaptation • The constraint of rate adaptation is determined by the receiver buffer size and the source video rate, whereas the main constraint of quality adaptation is transmission resources.

20. State transition diagram specifying the quality adaptation mechanism No capacity Available cumulative capacity <threshold Enough capacity Select Discard Available cumulative capacity ≥ threshold

21. Optimal Adaptation Available network bandwidth is known Stay as long as possible Residual bandwidth for higher layer

22. Theorem 1 • In the framework of the optimal quality adaptation, a threshold value equal to the receiver buffer size satisfies • 1) minimum video quality variability • 2) the necessary condition of maximum network utilization

23. Online Heuristic • The optimal quality adaptation algorithm assumes the available bandwidth information is known in advance. • An algorithm that minimizes quality variability without using future bandwidth information. • The differences between the online heuristic and the optimal adaptation • 1) the online heuristic makes a decision on which layer and which VOP to be transmitted in real time (lines 4-6) • 2) a sender makes a receiver prefetch the next selected VOPs when there is a transition from the select state to the discard state (line 15).

24. Online Heuristic Algorithm Make decision on which layer and which VOP to be transmitted in real time Receiver prefetch the next selected VOPs

25. How to determine the next prefetch point at the transition time ? • An MA (Moving Average) type estimator to determine the prefetch point. • simple and widely known for the usage of TCP retransmission timeout estimation in [7].

26. Experimental model

27. TFRC throughput Quality transition of the i th layer is defined by Rate smoothing Quality smoothing Receiver buffer TFRC/UDP (TCP-Friendly Rate Control)

28. Composed Performance over TFRC (1) QT=121 QT=13 Slow response time of TFRC Optimal adaptation QT=87 QT=9

29. Threshold based Performance over TFRC (2) Target on minimize loss probability Online heuristic QT=126 QT=16

30. TCP throughput Small time scale variability is significant as much as 3 Mbps

31. Composed Performance over TCP (1) Optimal adaptation

32. Threshold based Performance over TCP (2) Online heuristic

33. Two reasons contribute to superiority of TCP • TCP achieves more throughput than TFRC in dynamic condition. • Although TCP exhibits significant small time scale variability, it can be successfully accommodated by the receiver buffer.

34. Experiment results for 4 video streams Average Quality Transition Average Run Length [13]

35. Conclusion • Considering a problem of providing perceptually good quality for layered VBR streaming video. • An optimal adaptation algorithm that minimizes quality variability while increasing the usage of the available bandwidth. • Companion web site • http://www.cc.gatech.edu/computing/Telecomm/people/Phd/tkim/qa.html