Video Streaming Concepts

1. Video Streaming Concepts Reading: John G. Apostolopoulos, Wai- tian Tan, Susie J. Wee, “Video Streaming: Concepts, Algorithms, and Systems”, HP Laboratories Palo Alto, 2002.

2. Classification of video apps • Point2point vs multicast vs broadcast • Is there a “reverse channel”? • Pros and cons of reverse channel • Real-time vs pre-encorded(stored) video • What about “almost real-time”? • Interactive vs non-interactive video • Static vs dynamic channels • Bandwidth, delay, and loss are static or dynamic • CBR vs VBR channels • Do not confuse with CBR/VBR encoding • Packet-switched vs circuit-switched channels • QoS support?

3. Video compression standards • What do the standards specify? • Encoder and decoder implementations? • Bit stream syntax? • Decoding process?

4. Video streaming challenges • Video delivery via file download vs streaming ? • Characteristics of Internet are unknown or time-varying • Bandwidth  rate control • Delay jitter  playout buffer • Loss error control

5. Transport and Rate Control for overcoming Time-varying BW • Objectives • How to estimate the appropriate transmission rates dynamically at the time of streaming ? – available BW estimation • How media coding has evolved to support such dynamic changes in transmission rates ? – coding rate control • Rate control for video streaming over TCP • TCP is a window (not rate) based transport protocol • TCP adjusts the window size based on AIMD congestion-control algorithm  varying throughput (saw-tooth pattern) • Retransmission  enlarge delay jitter • Nevertheless, TCP is often used in video streaming, especially when receiver window is appropriately sized • Stable and scalable TCP rate control • TCP guarantees delivery • How would you calculate the receiver window size to set the streaming rate at a certain value?

6. Transport and Rate Control for overcoming Time-varying BW • Rate control for video streaming over UDP • Separation of error control and rate control • TCP-friendly rate control • Mimic average throughput of TCP = f(RTT, p) • Smooth the instantaneous fluctuations of TCP’s AIMD algorithm • Rate  1/RTT : video streaming에 적합한가? • Some media streaming system do not perform rate control • Multicasting: Identical stream is transmitted to all recipients via channels of different levels of congestion • No feedback channel • Receiver-driven rate control • Layered multicast of scalable or layered compressed video

7. Meeting Transmission BW Constraints • If channel transmission rate < media bit rate, then ?? • Transcoding • Decode and re-encode to the desired bit rate • Include bit rate reduction, spatial downsampling, frame rate reduction, changing compression formats • Requires application-layer gateways • Multiple file switching • Multiple copies of the same content at different bit-rates. Client chooses the appropriate media rate. • Multi-rate switching: enables dynamic switching between different rates within a session • No recompression penalty, but BW waste for multiple copies of the same media • Scalable compression (layered coding)

8. Evolving Approaches in Internet • IntServ model • QoS guarantees BW, packet loss rate, delay on per flow basis • Explicit resource reservation via RSVP • DiffServ model • Classify and differentiate among classes based on a tag(code-point) in each packet

9. Playout Buffer for Overcoming Jitter • Solution: Playback buffering • Key question: how long should the playback buffer (or playback delay) be? • Playback buffers have additional advantages: • Error recovery through ReTx • Error resilience through interleaving • Smooth throughput variations (e.g., due to TCP) • Adaptive Media Playout

10. Error Control for Overcoming Channel Losses • Bit error vs packet loss • In wired network, almost packet loss due to congestion • In wireless network, bit errors or burst errors may cause • To be passed up to the appl. Layer • Or, discarded (packet loss) • Solutions? • Retransmissions • Commonly used, but constrained by delay budget and playback delay • Requires back-channel • Forward Error Correction • Send N packets (K data packets + K-N redundant packets). If received packets >= K correctlry, data packets are recovered. • Introduce bandwidth overhead (and potentially delay) • Or, cannot recover. • Loss/error concealment • Loss/error resilient video coding

11. Loss/Error Concealment • Pixels, MBs, slices, and entire frame may be lost. • Exploits the correlation along • Spatial interpolation • Temporal extrapolation (freeze frame) • Motion-compensated temporal extrapolation • Motion-compensated block + estimation of lost MV from ?? • Error concealment is performed at the decoder

12. Error Resilience Video Coding: Overcoming Loss of Bitstream Synch. • Resynch Marker • Put the Resync markers (placed where?) • Reversible VLCs (MPEG-4) • Less efficient than VLS • Data Partitioning (MPEG-4) • Place most important data just after markers • Application Level Framing (ALF) • Design the packet(frame) payload to minimize the effect of loss

13. Error Resilience Video Coding: Overcoming Incorrect State and Error Propagation • Overcome error propagation • Use periodic I-pictures (GOP) • Use periodic intra-coding of MBs • In point-to-point comm. with back channel (short RTT) • Both the encoder and decoder store multiple previously coded frames • Decoder notifies the packet error to the encoder • The encode tells which picture should be used as the reference for the next prediction(Reference Picture Selection in H.263)

14. Scalable (Layered) Video Coding for Lossy Networks • SVC implicitly assumes broadcast or multicast of video steams (including overlay multicasting) • Base layer and several enhancement layers need different treatments • unequal error protection (UEP), • prioritized transmission • Useful if network provides several classes of service or priorities (DiffServ, IntServ) • Under the best effort Internet ?

15. Multiple Description Video Coding • Multiple description coding (MDC) video • Several “descriptions” of the same video • The more descriptions you receive, the better • Significant redundancy among descriptions • What if all descriptions are subject to simultaneous losses? • MDC video with path diversity

16. Media Streaming Protocols and Standards • Protocols for video streaming over the Internet • Media delivery: RTP/RCTP • RTP does not provide real-time delivery, neither support any QoS • Only provides time stamps, sequence numbering, video source identification, payload specification • RTCP provides QoS feedback in terms of # of los packets, delay, inter-arrival jitter, etc. • Media Session Control • RTSP (Realtime Streaming Protocol) • SIP (Session Initiation Protocol) • Media Description and Announcement • SDP (Session Description Protocol) • SAP (Session Announcement Protocol)

17. Additional Video Streaming Topics • Multicast • Should solve heterogeneity problem • Network heterogeneity: different channel conditions (BW, error protection) • Receiver heterogeneity • Approaches • Different multicasts for different ranges of intended bit-rates: e.g) simulcast • Different multicasts can contain incremental information (SVC): e.g) layered multicast • End-to-end security and Transcoding • Streaming over wired and wireless Links • Loss differentialtion: • Loss due to congestion or noise ? • Streaming media CDN

18. Streaming Media CDN • Conventional CDN vs streaming media CDN • Issues • Application-layer multicast (overlay multicast) • Seamless hand-off • MD-CDN • Achieve path diversity from the infrastruture of CDN