High Performance Recording and Manipulation of Distributed Streams

1. High Performance Recording and Manipulation of Distributed Streams Hasan Bulut Advisor: Prof. Geoffrey Fox Ph.D. Defense Exam April 24, 2007

2. Motivation • A collaboration environment contains multiple real-time stream sources and multiple sinks. • Streams are generated at different locations with different timestamps. • Streams can be from any collaboration system. • Collaboration environments, such as WebEx, may utilize messaging middleware systems. • We would like to replay them instantly or later at any time with control over the stream/streams, such as play, pause, forward and rewind.

3. Motivation • We also would like to replay any collection of streams. • Annotate streams and attach objects such as text. • The architecture should be generic so that we can archive and replay any type of streaming data format. • Independent of data format of the stream, i.e. video, audio, text, images. • Design and implement a system to demonstrate this generic framework; eSportsSystem.

4. GlobalMMCS Prototype System

5. Extending GlobalMMCS to Mobile Clients

6. eSports System Interface

7. eSports System - Collaborative Multimedia Streaming and Annotation System • A use case • eCoaching – educating athletes/players at remote locations by capturing streams from sports events or games and annotating them. • We have setup a prototype with machines in HPER at IU and Ithaca in NY. • We will extend it with machines in China to educateplayers in China remotely from US.

8. Research Issues - Stream Specific Issues • Data formatindependence – streams can be video, audio, text, images, etc. • Services required within messaging middleware to • archive and replay of streams • achieve instant replay of streams • achieve synchronization among streams generated at geographically large area • Increase fault tolerance • A scalable architecture to overcome performance issues • Jitter introduced by the archiving and replay service • Stream delay introduced for LAN and WAN clients

9. Research Issues - Session and Stream Management • Developing a XML-based control framework where XML messages are also used for information exchange. • Managing session information • Describing collaboration session metadata in XML format • Managing stream information • Streams are in different types and formats • They are archived at different repositories • Investigating the impact of using a distributed context management service on the design and architecture of the streaming service

10. Services/Components Built Within This Dissertation

11. Time Services • We need global time ordering of events. • NB-NTP implementation synchronizes with several different time servers instead of one in NTP case. • NB-NTP is incorporated within messaging middleware so that it can be used by entities. User does not need administrator privileges to use NTP. • Entities generating events in the system should utilize Time Service to timestamp the events. • High Resolution Timing Service • Gives 2-3 microsec resolution

12. NTP Benchmark Result • The first offset value is -139895 ms, which shows how much the clock in that machine is ahead of the real time. • The change of offsets is between (-3) - (2) ms. • Total change demonstrates clock skews

13. Jitter Reduction Services • Jitter value of a stream shows the quality of the stream. • The lower the jitter the better the stream quality. • Components are • Buffering Service • Time Differential Service

14. Jitter Reduction Service Benchmark Results Trans-Atlantic benchmark result (between UK and IN)

15. Distributed Repository Scheme • NaradaBrokering reliable delivery mechanism utilizes a repository to ensure that events are received by subscribers. • For repository failures, repository redundancy scheme extends the reliable delivery scheme. • Introduces modest latency for message delivery. • A repository gossips with other repositories to exchange information in order to retrieve missed events and to recover from failures. Fault-Tolerant Reliable Delivery of Messages in Distributed Publish/Subscribe Systems. (To appear) Proceedings of the 4th IEEE International Conference on Autonomic Computing.

16. Repository Redundancy Benchmark Results P1 Publisher S1 Subscriber (S1: Measuring client) Topology C Topology E Topology D Topology F Links are UDP

17. Repository Redundancy Benchmark Results 3 brokers, 3 repositories 3 brokers, 2 repositories 3 brokers, 1 repository 3 brokers

18. Session and Stream Information Management • We have used “Distributed Context Management” – developed at CGL. • Two levels of management • Sessions level management: to keep track of sessions • Intra-session level management: to keep track of stream information in a given session

19. Generic Streaming Service • Recording and replay of any type of streaming data. • Instant replay: replaying, pausing and rewinding real-time (live) streams while being recorded. • Linking multiple streams together to construct a session. • Uniform event type for generic streaming

20. Session Recorders Control message

21. Session Players Control message

22. eSports System – Key Features • Archive and replay of NaradaBrokering native events • Archive and replay of GlobalMMCS sessions • Instant replay • Utilizing Distributed Context Management service

23. eSports System and Streaming Services

24. eSports System Interface (Recording)

25. eSports System Interface (Replay)

26. Performance Results (Benchmark Setup) LAN Setup : gf4.ucs.indiana.edu WAN Setup (FSU): vlab2.scs.fsu.edu

27. Performance Tests (LAN Results)

28. Performance Tests (WAN –FSU Results)

29. Contribution • A generic fault tolerant distributed streaming framework • “Instant” replay of real-time streams. • High performance streaming with low latency (with delay 200ms). • Following services are introduced to messaging middleware; • Replication framework • Jitter reduction service • Time service • System contributions • eSports System prototype • Annotation capability • Distributed Context Management capability • XML Based General Session Protocol (XGSP) • Real-time videoconferencing streams for cellular clients.

30. Selected Publications • Implementing a NTP-Based Time Service within a Distributed Brokering System;Hasan Bulut, Shrideep Pallickara and Geoffrey Fox; Proceedings of the ACM International Conference on the Principles and Practice of Programming in Java. pp 126-134. • Worldwide Messaging Support for High Performance Real-time Collaboration; Pete Burnap, Hasan Bulut, Shrideep Pallickara, Geoffrey C Fox, David W. Walker, Beytullah Yildiz, Ali Kaplan; Proceedings of the UK e-Science Programme All Hands Meeting 2005 (AHM2005). Nottingham, UK • Fault-Tolerant Reliable Delivery of Messages in Distributed Publish/Subscribe Systems; Shrideep Pallickara, Hasan Bulut, Geoffrey Fox; (To appear) Proceedings of the 4th IEEE International Conference on Autonomic Computing. • Management of Real-Time Streaming Data Grid Services; Geoffrey Fox, Galip Aydin, Hasan Bulut, Harshawardhan Gadgil, Shrideep Pallickara, Marlon Pierce, and Wenjun Wu; Concurrency and Computation: Practice and Experience, March 12 2006. Special Issue from Grid and Cooperative Computing • Real Time Streaming Data Grid Applications; Geoffrey C. Fox, Mehmet S. Aktas, Galip Aydin, Hasan Bulut, Shrideep Pallickara, Marlon Pierce, Ahmet Sayar, Wenjun Wu, and Gang Zhai; Invited talk at TIWDC 2005 CNIT Tyrrhenian International Workshop on Digital Communications July 4-6 2005 • A Web Services Based Streaming Gateway for Heterogeneous A/V Collaboration; Hasan Bulut, Geoffrey Fox, Wenjun Wu, Ahmet Uyar, Shrideep Pallickara and Harun Altay; Proceedings of the 2004 International Conference on Internet Computing. Volume II, pp 493-499.

31. Selected Publications • Optimized Communication using the SOAP Infoset For Mobile Multimedia Collaboration Applications; Sangyoon Oh, Hasan Bulut, Ahmet Uyar, Wenjun Wu, Geoffrey Fox; Proceedings of the International Symposium on Collaborative Technologies and Systems CTS05 May 2005, St. Louis Missouri, USA • eSports: Collaborative and Synchronous Video Annotation System in Grid Computing Environment; Gang Zhai, Geoffrey Fox, Marlon Pierce, Wenjun Wu, Hasan Bulut; IEEE International Symposium on Multimedia (ISM2005) December 12-14, 2005 Irvine, California, USA • A Web-Services Based Conference Control Framework For Heterogenous A/V Collaboration; Wenjun Wu, Hasan Bulut, Ahmet Uyar, Geoffrey C. Fox; 7th IASTED International Conference on INTERNET AND MULTIMEDIA SYSTEMS AND APPLICATIONS ~IMSA 2003~ August 13-15, 2003 Honolulu, Hawaii, USA • Service Oriented Architecture for VoIP Conferencing; Wenjun Wu, Geoffrey Fox, Hasan Bulut, Ahmet Uyar, Tao Huang, Theory and Practice of the International Journal of Communication Systems - Special Issue on Voice over IP. • Design and Implementation of a Collaboration Web-services System; Wenjun Wu, Geoffrey Fox, Hasan Bulut, Ahmet Uyar, Harun Altay; Special Issue on on Grid computing in Journal of Neural Parallel and Scientific Computations (NPSC), Volume 12, pages 391-408 (2004) • Global Multimedia Collaboration System; Geoffrey Fox, Wenjun Wu, Ahmet Uyar, Hasan Bulut and Shrideep Pallickara; Journal of Concurrency and Computation: Practice & Experience. Middleware for Grid Computing Workshop Special Issue. Volume 16, Issue 5. pp 441-447. 2004.

32. Other Publications • Building and Applying Geographical Information System Grids; Galip Aydin, Ahmet Sayar, Harshawardhan Gadgil, Mehmet S. Aktas, Geoffrey C. Fox, Sunghoon Ko, Hasan Bulut,and Marlon E. Pierce; Concurrency and Computation: Practice and Experience, 12 January 2006. Special Issue on Geographical information Systems and Grids • Adapting H.323 Terminals in a Service-Oriented Collaboration System; Wenjun Wu, Hasan Bulut, Ahmet Uyar, Geoffrey Fox; Special Internet Media issue of IEEE Internet Computing July-August 2005, Vol 9 number 4 pages 43-50 • Grid Services for Earthquake Science; Geoffrey Fox, Sung Hoon Ko, Marlon E. Pierce, Ozgur Balsoy, Jake Kim, Sangmi Lee, Kang-Seok Kim, Sangyoon Oh, Xi Rao, Mustafa Varank, Hasan Bulut, Gurhan Gunduz, Xiaohong Qiu, Shrideep Pallickara, Ahmet Uyar, Choon-Han Youn; Journal of Concurrency and Computation: Practice & Experience. ACES Computational Environments for Earthquake Science Special Issue 2002. Volume 14(6-7):371-393 • Grids for Real Time Data Applications; Geoffrey C. Fox, Mehmet S. Aktas, Galip Aydin, Hasan Bulut, Harshawardhan Gadgil, Sangyoon Oh, Shrideep Pallickara, Marlon E. Pierce, Ahmet Sayar, and Gang Zhai; Invited talk in proceedings of PPAM 2005 – 6th International Conference on Parallel Processing and Applied Mathematics Poznan Poland September 11-14 2005 • Service-Oriented Architecture for Building a Scalable Videoconferencing System; Ahmet Uyar, Wenjun Wu, Hasan Bulut, Geoffrey Fox; IEEE International Conference on Pervasive Services 2005 (ICPS'05),11-14 July 2005, Santorini, Greece • Design and Implementation of Audio/Video Collaboration System Based on Publish/Subscribe Event Middleware; Geoffrey C. Fox, Wenjun Wu, Ahmet Uyar Hasan Bulut; Proceedings of CTS04 San Diego January 2004

33. Other Publications • An Integrated Videoconferencing System for Heterogeneous Multimedia Collaboration; Ahmet Uyar, Wenjun Wu, Hasan Bulut, Geoffrey Fox; 7th IASTED International Conference on INTERNET AND MULTIMEDIA SYSTEMS AND APPLICATIONS ~IMSA 2003~ August 13-15, 2003 Honolulu, Hawaii, USA • A Web Services Framework for Collaboration and Videoconferencing; Geoffrey Fox, Wenjun Wu, Ahmet Uyar, Hasan Bulut, Shrideep Pallickara; Proceedings of the 3rd Annual Workshop on Advanced Collaborative Environments • Integration of SIP VoIP and Messaging with the AccessGrid and H.323 Systems; Wenjun Wu, Ahmet Uyar, Hasan Bulut, Geoffrey Fox. Proceedings of 1st International Conference on Web Services Las Vegas June 2003 • Global Multimedia Collaboration System; Geoffrey Fox, Wenjun Wu, Ahmet Uyar, Hasan Bulut and Shrideep Pallickara; Proceedings of the 1st International Workshop on Middleware for Grid Computing. Co-located with ACM/IFIP Middleware Conference. pp 245-250. • Collaborative Web Services and Peer-to-Peer Grids; Geoffrey Fox, Hasan Bulut, Kangseok Kim, Sung-Hoon Ko, Sangmi Lee, Sangyoon Oh, Shrideep Pallickara, Xiaohong Qiu, Ahmet Uyar, Minjun Wang, Wenjun Wu; Proceedings of the 2003 Collaborative Technologies Symposium. • Integration of NaradaBrokering and Audio/Video Conferencing as a Web Service; Hasan Bulut, Geoffrey Fox, Shrideep Pallickara,Ahmet Uyar and Wenjun Wu; IASTED International Conference on Communications, Internet, and Information Technology, November 18 to November 20, 2002, in St.Thomas, US Virgin Islands

34. Other Publications • A Web Services Framework for Collaboration and Audio/Videoconferencing; Geoffrey Fox, Wenjun Wu, Ahmet Uyar, and Hasan Bulut • An Architecture for e-Science and its Implications; Hasan Bulut, Geoffrey Fox, Dennis Gannon, Kangseok Kim, Sung-Hoon Ko, Sangmi Lee, Sangyoon Oh, Xi Rao, Shrideep Pallickara, Qinglin Pei, Marlon Pierce, Aleksander Slominski, Ahmet Uyar, Wenjun Wu, Choonhan Youn; Proceedings of the 2002 International Symposium on Performance Evaluation of Computer and Telecommunication Systems (SPECTS). Society for Modeling and Simulation International, pp 14-24 (2002). • Service-Oriented Architecture for Building a Scalable Videoconferencing System; Ahmet Uyar, Wenjun Wu, Hasan Bulut, Geoffrey Fox; March 25 2006 to appear in book "Service-Oriented Architecture - Concepts & Cases" published by Institute of Chartered Financial Analysts of India (ICFAI) University • Grid Service Architecture for Videoconferencing; Wenjun Wu, Ahmet Uyar, Hasan Bulut, Sangyoon Oh, Geoffrey Fox; to appear as chapter in book "Grid Computational Methods" Edited by M.P. Bekakos, G.A. Gravvanis and H.R. Arabnia