1 / 53

MobiUS: Enable Together-Viewing Video Experience across Two Mobile Devices

MobiUS: Enable Together-Viewing Video Experience across Two Mobile Devices. Guobin Shen, Yanlin Li, Yongguang Zhang Microsoft Research Asia. Contents. Introduction Collaborative Half-frame Decoding System Architecture and Implementation Experimental Results and Evaluation

kordell
Download Presentation

MobiUS: Enable Together-Viewing Video Experience across Two Mobile Devices

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. MobiUS: Enable Together-Viewing Video Experience across Two Mobile Devices Guobin Shen, Yanlin Li, Yongguang Zhang Microsoft Research Asia

  2. Contents • Introduction • Collaborative Half-frame Decoding • System Architecture and Implementation • Experimental Results and Evaluation • Discussion and conclusion

  3. Introduction Motivation: A new better-together mobile application paradigm when multiple mobile devices are placed together

  4. A specific together-viewing video application A higher resolution video is played back across screens of two mobile devices placed side by side

  5. Assumptions • One device has a higher resolution video whose size is about twice of its screen size while the other not • Two devices can communicate effectively and directly via high-speed local wireless networks • Two devices are homogeneous: same/similar software and hardware capabilities

  6. Requirements • Real-time synchronous playback • At least 15 frames per second (fps) • Same frame rendered at two screens simultaneously • Energy efficiency • Work in resource-constrained environment • Limited processing power, memory, battery life … • Dynamic adaption • Expand the video on to two devices with another coming • Shrink it on to one screen with another leaving

  7. Possible Solutions • Full-frame Decoding-based Approaches: • Thin client model • Thick client model • Half-frame Decoding-based Approaches: • Whole-bitstream transmission (WTHD) • Partial-bitstream transmission (PTHD)

  8. Thin Client Model • Computation of Mb not utilized • Huge bandwidth demand • Unbalanced energy consumption • Short operating lifetime Decoded right half-frame Ma: Decode whole frame Mb Display left half-frame Display right half-frame

  9. Thick Client Model • Computation power of both devices utilized • Less bandwidth requirement • Balanced energy consumption • Abuse more computation power than necessary Whole bitstream Ma Mb Decode whole frame Display the left half-frame Decode whole frame Display the right half-frame

  10. Whole-bitstream transmission (WTHD) • Computation power of both devices utilized • Less bandwidth requirement • Balanced energy consumption • Abuse more bandwidth than necessary Whole bitstream Ma Mb Decode the left half-frame Display the left half-frame Decode the right half-frame Display the right half-frame

  11. Partial-bitstream transmission (PTHD) • Computation power of both devices utilized • Less bandwidth requirement • Balanced energy consumption • Implementation complexity Right half-bitstream Ma Mb Decode the left half-frame Display the left half-frame Decode the right half-frame Display the right half-frame

  12. Comparison Which method is the best?

  13. However, there is no free lunch.

  14. Background on Video Coding • Properties of video sequences: • Strong spatial correlation: each frame is an image • Strong temporal correlation: capturing instant of neighboring frames close to each other • Basic logic of video coding: Maximally strip off spatial and temporal correlations

  15. Motion Compensated Prediction MCP creates recursive temporal frame dependency Challenges arise from motion, but is worsened by recursive temporal dependency Ma Mb Ma Mb Cross-boundary reference effect

  16. How to perform efficient half-frame decoding? Cross-device collaboration (CDC) transmit the missing reference to each other

  17. Fundamental Facts • Markovian effect of MCP a later frame only depends on a previous reference frame, no matter how the reference frame is obtained • Highly skewed MV distribution the motion vector is centered at the origin (0,0) more than 80% of motion vectors are smaller than 8

  18. Push-based Cross-device Delivery Scheme

  19. Can it be better in energy efficiency? Percentage of boundary blocks that require cross-device collaboration and their corresponding bandwidth requirement

  20. Cumulative distribution functions of horizontal component of motion vectors for the whole frames and the boundary columns

  21. The bandwidth requirement of the helping traffic is relatively high, reaching half of the bandwidth required for sending the half bitstream • To make best use of multiple radio interfaces, the streaming data should be low enough for the Bluetooth’s throughput to be capable of • More than 90% motion vectors are smaller than 16, the width of a macroblock

  22. Guardband-based collaborative half-frame decoding scheme Each device decodes one more column of macroblocks

  23. Is it a good idea?

  24. How about larger extended half-frame?

  25. Larger guardband is not so beneficial

  26. Argument Shall we need CDC traffic for decoding the boundary blocks in the guardband? Yes, only if we need to decode the whole guardband correctly. However, we do not have to ensure the guardband to be correctly and completely decoded.

  27. Different decoding schemes for guardband blocks

  28. System Archtecture

  29. automatically set up a network between two mobile devices

  30. a simple radio signal strength based strategy Ensure a close proximity setting

  31. Check capability of a newly added device and inform the content host about the arrival or departure of the other device

  32. Application level synchronization strategy RTT-based synchronization procedure

  33. RTT-based Synchronization Scheme Ma Estimate RTT Wait half RTT Display the next frame Display next frame Mb

  34. Decoded frames Half-bitstreams for local device Half-bitstreams for the other device Hold and send/receive the cross-device collaboration data to the other device

  35. Independent full-frame based fast DCT-domain down-scaling decoding module The guardband-based collaborative half-frame decoding module Parse the original bitsream into two half bitstreams and extract the motion vectors

  36. Configuration of Two Devices

  37. Experimental Results

  38. Benchmark of Mobile Devices Mobile devices are cost-effectively designed, Just able to meet the real-time playback requirement for videos at the same resolution of the screen

  39. Decoding Speed

  40. Decoding Speed Both collaborative half-frame decoding schemes significantly improve the decoding speed. The guardband-based scheme is only slightly slower than the half-frame decoding case.

  41. Synchronization

  42. Synchronization Due to periodical synchronization

  43. Synchronization Due to a large scene change with very high motion. It is tolerable because the human visual system is far less sensitive for such slight asynchronism, especially when the motion is large.

  44. Energy Efficiency Collaborative half-frame decoding scheme leads to significant energy savings.

  45. Discussions • Further optimization opportunities • Service provisioning • User study • Assumption on homogeneity

  46. Further Optimization Opportunities • Computing saving the color space conversion consumes 30% of the overall time • Collaborative traffic reduction simple compression; error concealment technique • Energy consumption reduction save screen backlight energy consumption through the gamma adjustment make use of dynamic voltage scaling capability

  47. Service Provisioning • New encoder profiles to generate completely self-constrained substreams each substream corresponds to half-frame • Efficient arbitrary resizing transcoding to generate video content with suitable resolution

  48. User Study

More Related