Presented by Zuo Yinbo ( 左银波 ) June 8 , Wed., 2011

1. GestureFlow: Streaming Gestures to an AudienceYuan Feng, Zimu Liu, Baochun LiDepartment of Electrical and Computer Engineering, University of Toronto Presented by ZuoYinbo (左银波) June 8, Wed., 2011

2. Background Use multi-touch gestures or body motion gestures to interact with computers

3. Background, cont. • Collaborations among users in real time need streaming gestures in a broadcast fashion from one user to all participating users. • Implement a gesture broadcast protocol to accomplish this aim.

4. Contents • Introduction & Related work • Design & Implement • Coding gesture broadcast sessions • Coding across multiple broadcast sessions • Experiences with GestureFlow • Overall performance • Network coding performance • Mitigating linear dependence • Conclusion

5. Introduction & Related work • In this paper, the author presents the design of GuestFlow, a gesture broadcast protocol designed for concurrent gesture-intensive streams in multiple broadcast sessions. • It achieves effectiveness and practicality by using inter-session network coding, and solve the challenges introduced by linear dependence.

6. Introduction & Related work, cont. • Random network coding • maximize information flow rates in multicast sessions • reduce download time of P2P systems • achieve high playback rate and system throughput • In this work, however, network coding is used for ensuring reliable delivery with low delays in low bit-rate traffic from gesture events.

7. Introduction & Related work, cont. • Network coding(NC) in transport protocols • improve unicast throughput (by intra-session NC) • achieve low multicast latency in streaming • provide better support to a unicast TCP session with sliding window • In this paper, however, the transport protocol (GestureFlow) is designed specifically for multiple broadcast sessions, and it specifies how the coding window advances itself.

8. Design & Implement • Gesture streams’ transporting features • low bit rate • bursty traffic • sensitive to packets loss and delay • continuous playing back of a gesture stream • TCP protocol is not suitable • varying available bandwidth over time • high delay jitters with congested links

9. Design & Implement, cont. • The transport solution to gestures streams • a protocol called GestureFlow • “all-to-all” broadcast ( each node broadcast to all other nodes and multiple broadcast sessions exist concurrently) • low-delay streaming with guaranteed reliability • random network coding • stream coded packets using UDP flows rather than TCP • either a direct UDP link between source and the receiver or a two-hop path which uses a single relay node • allow relay nodes to recode packets

10. Design & Implement, cont. Fig. 1. Streaming coded blocks along multiple paths in GestureFlow. Transmit Data packets from Node 1 to Node 4 in coded form, either with a direct link ,or relayed by Node 2 and Node 3 after being recoded with their own packets.

11. Design & Implement, cont. • Coding gesture broadcast sessions coding coefficients: choose from C is the received coefficient Receiver decodes as soon as it has received k linearly independent coded block. Loss of particular blocks will not cause failure of decoding and thus guarantee reliable delivery.

12. Design & Implement, cont. • Problem with coding of fixed k data blocks • all the previous k-1 blocks will be waiting for coding until the kthblock arrives (delay) • The solution in GestureFlow • use a sliding coding window with a maximum size • sequence number of the earliest original block in the window is embedded within the UDP packet • after receiving ACKs of the earliest original block from all the receivers in the broadcast session, the coding window advances itself by removing the earliest original block

13. Design & Implement, cont. The coding window advances and adapts itself over time according to the traffic.

14. Design & Implement, cont. • How to acknowledge the sender? • do not acknowledge a newly decoded data block immediately for there will be additional overhead and complexity • use cumulative acknowledgements of decoded data blocks • receiver sends an acknowledgement for a block only if all earlier blocks with smaller sequence numbers have been decoded • the earliest sequence number is embeddedin each packet (block)

15. Design & Implement, cont. Receiver sends a cumulative acknowledgement (for b3) only when all earlier blocks (b1, b2, b3) have been decoded

16. Design & Implement, cont. • Use single-hop and two-hop path together to reduce delay jitters • relay nodes are receivers themselves, no additional bandwidth is consumed • data block will arrive via a path with the lowest delay • three-hop will unlikely offer a lower delay • Use inter-session coding • relay node receive blocks belonging to different sessions • all incoming coded blocks are mixed together for recoding and then transmitted to other nodes

17. Design & Implement, cont. Relay node (Node 2) mixes all incoming coded blocks ( S1, S3, S4 ) together in its recoding process, and sends coded inter-session blocks out.

18. Design & Implement, cont. • How do relay nodes produce coded blocks ? • relay node also maintains a sliding recoding window and remove the decoded original blocks • after the source of a session advances its coding window by removing its earliest original block, all relay nodes will easily detect it as the sequence number of the earliest original block is embedded within a coded block • a relay node removes original data blocks whose sequence number is smaller than the starting sequence number in a newly received coded block from the source

19. Design & Implement, cont. removed from the recoding window newly received coded block a relay node removes blocks from its recoding window

20. Experiences with GestureFlow • MusicScore: a collaborative music composition application on iPad • users interact with it to compose music using only multi-touch gestures • it uses GestureFlow to stream gesture events among multiple participating users • Experiments scenarios • four nodes in “all-to-all” broadcast sessions • two iPadsconnecting to the Internet through Wi-Fi, and two iPhone 3GS devices connecting to the Internet through 3G and EDGE(2.75G), respectively.

21. Experiences with GestureFlow, cont.

22. Experiences with GestureFlow, cont. Percentage of recoded blocks from relay nodes over all received blocks Streaming delays with different numbers of participating nodes

23. Experiences with GestureFlow, cont. Bandwidth overhead per user The CPU load and memory usage of network coding in an iPhone 3GS device

24. Experiences with GestureFlow, cont. Average delays along with 95% confidence intervals in different experiment settings Streaming delay get minimum value when coding window size is 8

25. Experiences with GestureFlow, cont. 90% of the coding windows have a size of no more than 5 blocks, which indicates coding and decoding process add little delay, as blocks do not have to wait too long to be transmitted or relayed. 90% of the recoding windows have a size of no more than 11 blocks, with an average around 4 blocks. this indicates that there is only one block or two required to be recoded at the relay node.

26. Experiences with GestureFlow, cont. CDF of linear dependence in different settings of maximum coding window size (W). 90% of the coded blocks contain around 15% blocks that are linearly dependent with each other. Waste of bandwidth!!! Received coded blocks are considered useful only when they are linear independent with each other, or else they are considered as redundant blocks. The linear dependence ratio is higher when the coding window size is getting smaller, 18% when W = 8.

27. Mitigating linear dependence • Mitigate the bandwidth overhead due to linearly dependent blocks • apply Reed-Solomon codec based on the Vandermonde matrix on the source node in a GestureFlow broadcast session • a k×ksubmatrix of a Vandermonde matrix has a nonzero determinant and is nonsingular • use k×ksubmatrix to code original blocks to guarantee linear independence among all coded blocks • relay nodes still use random linear networking coding to recode received blocks

28. Mitigating linear dependence, cont. A Vandermonde matrix with n-k rows and k columns Any k×ksubmatrix of G has a nonzero determinant and is nonsingular, and as a result every subset of k rows of G is guaranteed to be linearly independent. As such, linear independence among all original and coded blocks transmitted from the source guaranteed with the use of Vandermonde matrix to generate coded blocks.

29. Mitigating linear dependence, cont. 90th percentile of linear dependence is significantly reduced by using the new design, from 18% to 6%. The remaining dependence is caused by the recoding process in relay nodes

30. Mitigating linear dependence, cont. Streaming delays and their standard deviations from Wi-Fi 1 User to other three users by using the new and original GestureFlow. By using RS codec at the source, average streaming delays through different kinds of connections have been evidently reduced. Since the linear dependence is mitigated, a received block can be used to decode with a higher probability, reducing the decoding delay.

31. Conclusion • Propose the new paradigm for users to interact with computers--Gesture • Design and implement a transportation protocol (GestureFlow) for streaming gesture events with low but bursty bit rates • Use random linear network coding and RS codecto solve the challenges brought by low streaming delays and multiple concurrent broadcast sessions • Explain thoroughly the design and improvement of the streaming protocol • Evaluate the proposed protocol with an iPad application Music Score

32. Thanks! Q & A