Error Control in Multimedia Streaming

1. Error Control in Multimedia Streaming

2. What’s the difference • Traditional data: Text • Packet loss is intolerable while delay is acceptable • Multimedia data: Real-time video • Some visual quality degradation is often acceptable while delay must be bounded • This feature introduces many new error control mechanisms applicable to video application, but not applicable to traditional data.

3. Categories • FEC • Retransmission • Error-resilient encoding • Error concealment

4. FEC • Forward Error Correction • Add redundant information so that original information can be reconstructed in the presence of packet loss. • More effective when the average number of consecutively lost packets is small.

5. Retransmission • May miss the play-out time, if the retransmission needs too much time which exceeds the allowable delay • So we need to know if the retransmission will be available before the deadline, then decide retransmit or not.

6. Error-resilient • Objective • Enhance robustness of compressed video to packet loss • Executed by the source to enhance robustness • Difference with FEC • prevent error propagation or limit the scope of the damage(caused by packet losses)

7. Error-resilient: example • MDC—multiple description coding • Compressed into multiple streams (descriptions) • Each description provides acceptable visual quality • More combined descriptions provide a better visual quality • Advantage • Robustness to loss, enhanced quality • Disadvantage • Trade off between compression efficiency and the reconstruction quality from one description

8. Error concealment • Performed by the receiver when packet loss has already occurred • The receiver conceal the lost data and make the presentation less displeasing to human eyes • Example • Error resilient and conceal are more about compression and the nature of the original data.

9. An architecture for error control mechanisms

10. FEC--categories • Channel coding • Source coding • Joint source/channel coding

11. FEC—Channel coding • The channel encoder places the k packets into a group and then create additional packets from them so that the total number of packets in the group becomes n. • A user must receive K (K>=k) packets. • These K packets will be enough for user to reconstruct all the original k packets.

12. FEC--Channel coding(contd.)

13. FEC--Channel Coding(contd.) • Pro • Can scale to arbitrary number of receivers in a large multicast group. • Allows the network and receivers to discard some of the packets which cannot be handled due to limited source. • Con • Increase the transmission rate • Increase delay • Not adaptive to varying loss characteristics (can use feedback to improve it)

14. FEC—channel coding(contd.) • EEP(equal error protection) • All the bits of the compressed video stream are treated equally. • Not good for compressed video stream--MPEG • UEP(unequal error protection) • Example: Priority Encoding Transmission • Allow user to set different levels of error protection for different segments of the video stream. • Transport MPEG video which has an inherent priority hierarchy (I-,P-,and B-frames), give more redundancy to I-frame, but less to B-frames.

15. Source coding • Source coding-based FEC(SFEC) • The nth packet contains both nth GOB (group of blocks) and redundant information about the (n-1)th GOB ( a much more compressed version) • The we can reconstruct (n-1)th GOB if it is lost and we receive nth GOB. (will has coarser quality) • Channel coding can perfectly recover, while SFEC can only recover an acceptable one. • Lower delay (needn’t to accumulate k blocks)

16. FEC—joint source/channel coding • Shannon’s Separation Principle • Both source coding and channel coding can be done independently. • But it is only valid upon the assumption of allowing infinitely long codewords (infinitely long delay)

17. FEC—JSCC(contd.) • channel capacity-limited video commu. system: • spending more bits on the source means not enough channel protection, which leads to channel errors, received video quality is bad; • spending more bits on the channel means enough protection and no transmission errors, but then you have overcompressed the source material and received video quality is again bad. • There is a trade-off and balance point where the channel capacity is optimally allocated between source and channel to achieve the best received video quality.

18. FEC—JSCC(contd.) Source coding Joint source/channel coding

19. FEC—JSCC(contd.) • Task 1: finding an optimal rate allocation between source coding and channel coding for a given channel loss characteristic • Task 2: designing a source coding scheme to achieve its target rate • Task 3: designing channel codes to match the channel loss characteristic and achieve the required robustness

20. Delay-constrained retransmission • An approach in Transport layer • The conventional retransmission scheme • When packets are lost, the receiver sends feedback to notify the source, the the source retransmits the lost packets • So the retransmitted packet arrives at least 3 one-way trip time after the transmission of original packet. • It only works if the one-way trip time is short with respect to the maximum allowable delay.

21. Categories • Unicast • Receiver-based • Sender-based • Hybrid control • Multicast

22. Unicast—receiver-based • Receiver-based control • Objective: minimize the requests of retransmission that will not arrive timely for display • Algorithm When receiver detects the loss of packet N: Ask sender to retransmit packet N Current time Slack term, could include tolerance of error in estimating RTT. Scheduled display time for packet N

23. Receiver-based(contd.)

24. Unicast—sender-based • Sender-based control • Objective: suppress retransmission of packets that will miss their display time at the receiver. • Algorithm: When the sender receives request of retrans. for packet N: retransmit packet N to receiver Estimated one-way trip time Estimate of Td(N)

25. Sender-based(contd.)

26. Unicast—hybrid control • Receiver-based control • Objective: both of the objectives of receiver-based and sender-based • Receiver makes decisions on whether to send retransmission requests while the sender makes decisions on whether to discard retransmission requests. • Higher performance at the cost of higher complexity.

27. Multicast • Retransmission has to be restricted within closely located multicast members. Because one-way trip times between these members tend to be small, and the neighbor members have the packet which is lost. • Basic idea: this closely located multicast member will be the designated receiver (DR) in a tree-based hierarchical framework.

28. Logical tree with DR

29. Local recovery for multicast

30. LVMR • Layered Video Multicast with Retransmission • Use a statically configured logical tree • Improve efficiency by not asking for retransmissions of those packets which cannot be recovered in time. • Improve response time by sending immediate NAKs and multicasting retransmissions. • Use buffers to not only absorb the jitter but also to increase the likelihood of getting retransmitted packets before deadline.

31. LVMR—contd. • Adaptive Playback Points with Extended Control time • playback point of frame i is • In LVMR, playback point p0 is extended by an additional amount which is referred to as control time. • We have , is the maximum jitter in the network. • So there is at least a budget of time units after the arrival of a frame to recover any missing fragments. • Trade-off between latency and reliability. P0:the playback point of frame 0 R: frame rate (frame/second)

32. STORM • STructure-Oriented Resilient Multicast • Use a dynamically changing logical tree • When a receiver first joins a multicast group, it uses an expanding ring search to look for parent nodes(DR). • Selects its parent based on the packet loss rate of the candidate parent nodes • Receiver periodically computes the ratio of the number of successful repairs from a parent to the number of NACKs sent to the parent. If this ratio falls below a threshold, start to find a new parent. • Let the receivers decide on their own the trade-off between latency and reliability.

33. Conclusion • FEC,retransmission, error-resilience are performed at both the source and the receiver side, while error concealment is carried out only at the receiver side. So it is used at the last stage. • They are combined to get a better performance. • Transport perspective: • Retransmission, FEC • Compression perspective: • Error-resilience, error concealment, FEC • Use feedback to improve the performance

34. Main Reference • A. Albanese, J.Blomer, J.Edmonds, M. Luby, and M. Sudna, “Priority encoding transmission,” IEEE trans. On information Theory, vol.42,no.6,pp.1737-1744, Nov.1996 • J-C. Bolot and T.Turletti, “Adaptive error control for packet video in the Internet,” in Proc. IEEE Int. Conf. On Image Processing, pp.25-28, Lausane, Switzerland, Sept. 1996 • Dapeng Wu,Yiwei Thomas Hou,etc, Streaming Video over the Internet: Approaches and Directions IEEE Transaction on Circuits and System for Video Technology, Vol11,No1,February 2001 • Dapeng Wu, Yiwei Thomas Hou,etc, Transporting real-time video over the Internet:challenges and approaches, Proceedings of the IEEE, vl.88,no. 12, Dec.2000 • X. Li, S. Paul, P.Pancha, and M.H. Ammar, “Layered video multicast with retransmissions (LVMR): evaluation of error recovery schemes, “ in Proc. IEEE Int. Workshop on Network and Operating System Support for Digital Audio and Video, pp.161-172, May 1997.