1 / 17

Real-time smoothing for network adaptive video streaming

Real-time smoothing for network adaptive video streaming. Kui Gao, Wen Gao, Simin He, Yuan Zhang J. Vis. Commun. Image R. 16 (2005). Outline. Motivation The architecture of network-adaptive streaming system A Real-time dynamic smoothing algorithm How to select and schedule packets

noble-parker
Download Presentation

Real-time smoothing for network adaptive video streaming

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Real-time smoothing for network adaptive video streaming Kui Gao, Wen Gao, Simin He, Yuan Zhang J. Vis. Commun. Image R. 16 (2005)

  2. Outline • Motivation • The architecture of network-adaptive streaming system • A Real-time dynamic smoothing algorithm • How to select and schedule packets • How to retransmit the lost packet • Simulation results • Conclusion

  3. Motivation • Problems: • Variation of network bandwidth • Packet may lost • The current Internet • Provide best-effort services • Do not provide QoS • Goals: • Provide a smoothing quality at a client end (real-time smoothing) • Maximizing the utilization of the variable network bandwidth (adaptive video streaming) • Propose a dynamic real-time smoothing algorithm to select and scheduling the packet • Use FGS streaming to maintaining a very flexible and simple video-coding structure

  4. The architecture of network-adaptive streaming system Server Client Use a real-time smoothing algorithm Feedback information includes: Packet loss rate, RTT(round-trip time)

  5. How to select and schedule the packet • Considerations • Playback distortion (lost, damaged) • Network bandwidth • Packet Deadline (arrive before the playback time) • Packet dependency • Summary • Different portions of video bitstream have different contributions to the video quality. • The important packets (with large distortions) can be transmitted earlier. • The packets of base layer are the most important. Feedback information

  6. When packet loss, how to recover • Use retransmission scheme to recover lost packet • But not all lost packet can be recovered • Due to the available network bandwidth and the delivery deadline constraint • Summary: • The lower layer has more chances to retransmit its packets.

  7. The characteristics of the transmission buffer (1/2) • The packet in transmission buffer has two states: • In the ready states: • Can be scheduled for transmission • In the blocked states: • Can not be scheduled until packet loss or its timeout RTO≦tout • How to select a packet to transmit at tcur from the transmission buffer • Sbuf is the set of all the ready packets Sbuf = {pi,j|ai,j≦tcur and tcur+ci,j≦di and in ready state} pi,j: the packet of the jth layer in frame Fi ai,j: release time (the earliest time the packet in the buffer) ci,j: process time of pi,j di:deadline (the latest time pi,j should be sent to client)

  8. The characteristics of the transmission buffer (2/2) • How to select • First step • Server selects the ready packets of base layer with earliest deadline firstly (EDF) • Second step • If no base layer packets in ready state, the serer select and schedule the ready packets of enhancement layers • In enhancement layer • According to rate-distortion decision • If the packets have same distortion, packet with earliest deadline are served first

  9. client The transmission flow ai: the earliest time the packetin the buffer Feedback includes: packet loss rate & RTT(round-trip time) ACK: packet unloss NAK: packet loss Layer 3 Layer 3 Layer 3 Layer 3 Layer 2 Layer 2 Layer 2 Layer 2 Layer 1 Layer 1 Layer 1 Layer 1 Base 0 Base 0 Base 0 Base 0 (1) According to feedbackto decide ai (release time) …4 3 2 1 (Frames) ai=4 ai=3 ai=2 ai=1 (3) Base layer transmit first Enhancement layer transmit according to the distortion (2) Put packets into buffer according to ai Ready state Layer 3 Layer 3 Layer 2 Layer 2 Layer 1 Layer 1 Base 0 Base 0 Block state Feedback information & (ACK or NAK) Transmit buffer

  10. A Real-time smoothing algorithm • n frames, a scalable media stream F = {F0,F1,…,Fn-1} • m: the number of layers • pi,j (0≦i≦n-1, 0≦j≦m): the packet of the jth layer in frame Fi • ai: release-time (the earliest time pi,j in the transmission buffer) • di: deadline (the latest time pi,j should be sent to client) • assume the packets in the same frame have the same ai and di • ploss(t): packet loss rate • RTT(t): round-trip-time • RTO(t): retransmission timeout • The processing time of pi,j: ci,j = bi,j/X(t) • bi,j: the size of the packet pi,j • X(t): the available network bandwidth at time t • The fulfill-time of pi,j : fi,j = si,j + ci,j • si,j: the scheduler time of pi,j (fulfill-time = scheduler time + processing time) feedback information

  11. How to decide the release time (ai) • MPEG-4 video can accept an error rate of 10-5 or lower • How to decide the release time (the earliest time pi,j into the transmission buffer) • If the release time is too early • The transmission buffer must be very large • If the release time is too late • There is no chance to retransmission • It is important to select the retransmission times K for lossy packets of the base layer • Let be the total loss probability of the pi,0 through K times retransmission, we get the smallest K that ≦10-5 • ai = di–K * RTO (K & RTO decided by feedback) • di: deadline (the latest time pi,j should be sent to client)

  12. Constraint of playback (In enhancement layer) (1/2) • It can be • Arbitrarily truncated (decided by the available channel bandwidth) • Tolerate the channel errors • A packet pi,j can be selected and scheduled if satisfied • The current time is later than release time(ai ≦ tcur ) • The fulfill-time is earlier than deadline (ai≦ tcur andtcur +ci,j≦ di ) p.s: ci,j = process time of pi,j • According to rate-distortion optimized decision

  13. Constraint of playback (In enhancement layer) (2/2) • Assume the quantization of a frame is Q • if the maximum number bit-plane of a frame is m, and the last bit-plane is z, then Q=2m-z • A distortion model built for uniform quantizer (UQ) is • The distortion of packet pi,j is • The total quality is when J is maximize, the total quality is the best

  14. Simulation results (1/3) • Use TFRC protocol to decide the sending rate • s: the packet size, R is RTT (round-trip time), tRTO is retransmission timeout value, and p is the pack loss ratio. • The sequences include Foreman, Coastguard and Akiyo • Encoded with 30 frames per second • There are 7 enhancement layer • The playback frame rate is 30Hz

  15. Simulation results (2/3)

  16. Simulation results (3/3)

  17. Conclusion • Propose a real-time smoothing algorithm • According to feedback information • to select and schedule the packet • under a rate-distortion optimized • The algorithm improves • the utility of the bandwidth • smoothes the playback quality

More Related