The development of communication without spatial boundaries

1. The developmentofcommunication without spatial boundaries Carsten Griwodz griff@ifi.uio.no

2. In use Broadcast television Distributed games SMS PayTV Web P2P file sharing Video-on-demand MMS Audio and video conferencing … In labs 3D worlds Virtual reality Distributed classrooms Shopping malls 3D cameras Augmented reality P2P name resolution … Distributed interactive applications: 10 years ago

3. Prevalence of interactive applications in entertainment • Market shift in entertainment • 10 years ago • Broadcast media with roughly 50% of all content live • Sport shows • News shows • Reality shows • Game shows • Stand-alone gaming • Bought and rented videos • Today • Broadcast media with nearly 99% of all content live • Game shows • Reality shows • Game and sport reports • Live reports • Interactive shows • Interaction fiction (personalized, live generation of 3D worlds) • Interaction games

4. Prevalence of interactive applications in entertainment Interactive games Mrd Euro Broadcast media Interactive entertainment

5. Prevalence of interactive applications in entertainment • Reasons • Content theft in passive on-demand applications requires protection measures • Adequate protection technology • Required for all digital recording devices to be always connected • Interfered with device usage • Was not accepted by users • Solutions • Lease content and make it available only when connections are available • Provide distributed interactive applications such as games • Provide unstructured, mostly meaningless content • Artistic part added through interaction with content providers

6. Resource issues in home entertainment • Home entertainment systems • Integration of many independent devices into a single experience • TV screen (non-interactive) • Projectors and HDTV widescreen (non-interactive) • Video walls (interactive) • Distributed devices with session migration (interactive) • Input devices for interactivity • Low end single base devices • Remote control, keyboard, mouse, joystick, gamepad, .. • High end multiple base devices • PDA, mobile phone, … • gesture trackers, voice control, … • Input to one application shared among several consumers • From multiple independent end systems • Personal distributed computing • High synchronization requirements • Synchronize audio streams from several devices 2006 2012 2006 2016

7. Distributed devices and computing Number of cores in a single processor Azul Cell ia32, ia64 T1

8. Distributed devices and computing • Session migration • Move running applications between end systems • E.g. from a mobile phone to a home entertainment system • Distribution of computation and view • High level description for audiovisual objects • Enables distributed rendering on several output devices • E.g. picture-in-picture of summarized news and measurements • Personal grid • Development environments create distributable applications transparently • Jobs migrate automatically to machines with appropriate services

9. Resource issues in home entertainment • Most applications are interactive today • Interactive fiction requires very high bandwidth and very low latencies at the same time • Development steps • Latency requirements are addressed by program design • Latency requirements are addressed by architecture support • Bandwidth increase becomes possible • Mobility of devices improves • Team play within massive multiplayer games becomes popular • Mesh networks are used to increase scale • Required multi-homing, load balancing, multipath support • Mesh networks require charging and billing among end-users • QOS support 2006 2016

10. Latency issues • You can’t hide latency in some applications • Applications where prediction of the remote state will inevitably lead to wrong decisions • Remote operations • Mobile exploration, construction and rescue robots • Multiplayer games without physical models • Military command and control • Distributed production streets • … • Applications where the latency is so high that prediction can not be applied any more • Space exploration, construction and rescue • Result • New comeback of AI technology • Achieve independent decisions in many more cases • Based on general directions instead of direct control • New kind of critical operator jobs • Requires a high level of education: knowledge of processes and there interaction • Possibly working under high risk or in adverse conditions • QoS to push the limits

11. Mesh networks • Make it affordable to use higher bandwidth • Enable resource sharing • Required to achieve data rates required for interactive fiction • Higher aggregate bandwidth • Remain always on • Resilient against failures of individual connections • Incentive to share resources is required • Billing developed • Limited sharing is necessary • Wireless multihop routing extended with QoS mechanisms • QoS mechanisms for uplink resource sharing introduced • QoS for downlink resource sharing introduced

12. Mesh networks • Mesh networking implies multiplexed connections through several Internet service providers at the same time • Multi-homing and multi-path support • Local network • Market of local resources • Share other local resources as well • Mesh networks are implemented on top of wired and wireless devices • Operated under private ownership, as public service or commercial service, or any mix of this • One mesh is an administrative domain • Defined by a contract for resource sharing and the tools required to implement the contract • Nodes can participate in several meshes • Nodes do not have to route traffic

13. Two requirements for scalability in mesh networks Multi-homing Addressable presence of the same transport connection of an application in several networks at the same time Multi-path Transmission of a single data stream to a single application using several networks concurrently Required a new addressing abstraction Alternative supported approaches Anycast to multi-homed interfaces, and path choice at individual routers Transport address layer translation to several network layer addresses to perform congestion control for the separate homes and paths Connection-oriented network layer with end-to-end negotiation of several paths Transparent multi-homing and multi-path

14. Commercial content hosting development • Centralized web hosting • Ease of use, load leveling • Content distribution in static overlay networks • Scalability • Hosting of applications • Applications become more interactive • Offload central servers, keep traffic local, aggregate and filter at the application level • Not quite active networks

15. Commercial content hosting development • Proxies as a dynamic software solutions • Success of P2P generalized, exploit available end user resources, but need incentive model, which is here financial • Traffic can be kept local • Reduce load • Reduce latencies • Co-development with mesh network availability • Private business from private proxies • Small-scale revenue model that can grow and shrink dynamically • Resources can be traded among consumers • Power-nodes liker eBay power sellers • Goes hand-in-hand with QoS support in mesh networks

16. PGFS: Personal global file system • Data appear to be stored in a single global file system • Accessible from all connected end devices • Independent of • Current location • Current end system • Administrative domain (home or business) • Fully transparent access to data • Personal data • Publicly available content (e.g. web pages, incl. dynamically created web page) • Content shared with other users

17. PGFS: Personal global file system • The personal global file system • Is virtual • Every user has one personal global namespace • Can span several actual local namespaces • Lookup service • Subscription to one of several global namespace management services • P2P systems using aggressive caching and replication • Dynamic growth through hosting • Locking to a physical location is possible • Contribution to the storage pool is possible • Reserved storage space is traded on-line • Very secure

18. Importance of conferencing • 10 years ago • Hardly in use aside from two-party AV communication • Problems • Low video quality • High latency and jitter • No immersion • Cooperation mostly in a shared computer use paradigm • Now • Important because of high fuel prices • Available in public transport systems (including road trains, railcabs) • Features • 3D conferencing • Augmented reality conference settings • Gesture and voice recognition • Quality guarantees

19. Interactive distributed 3D design • 10 years ago • Go to IKEA with the measurements from your house • Now • Loan an ultrasound meter from IKEA • Set up it in your room • Design interactively with an online salesperson in a 3D presentation • Use gesture tracking to manipulate the room • Other applications • Distributed construction of houses, bridges, etc. • Design session is integrated with the architects' home system • E.g. perform stability automatically like spell-checking in a word processor 10 years ago • Owner can be included in the process • In general more interaction in production

20. Medical services • Hierarchically organized • From local first aid station to rare specialized clinics • Reason is the cost of specialization • Local health station need support to handle difficult cases • Approach • Central remote treatment centers • Mobile operation theaters • Requirement • Top quality data network to every local health station • Extremely resilient networks • QoS support in networks • Availability of dynamic short-term QoS contracts • Simple reconfiguration

21. QoS • 10 years ago • Service planning • Bandwidth-oriented • Generic SLAs among ISPs • Now • Latency-oriented • Local area QoS • Static distribution defined by mesh contracts • Policy-conforming distribution of resources • Resource brokerage • … all of the above • Long-distance QoS • Unified labeling and switching for low-latency connections • On-demand QoS offered by independent traffic aggregators (brokers)

22. Supervision cameras • Until 5 years ago • Massive growth in number of CCTV cameras • Decrease in police in the street • Increase in petty crime • Increase in private security service business • 4 years ago • Improvement in image detection and tracking improves solution rate for petty crime to nearly 100% • Increasing frequency of misuses makes people suspicious • 3 years ago • All CCTV cameras are made publicly accessible • Development of 3D scene creation from multiple camera angles • Object recognition allows automatic tracking of people • Coordinated actions through cities by private security companies based on Jedermannsrecht • Decrease in petty crime

23. Supervision cameras • Leads to business • Object tracking across several cameras • Suffers from inaccurate timing and different latencies and picture quality • Can be overcome by exact timecodes • Offered as commercial add-on service • Commercial redistributors negotiate for additional services • Exact timecodes (using Galileo), object recognition, and 3D reconstruction make it possible to create 3D scenes from several cameras, thereby following everybody around • Allows people to track others • One service: Nanny-TV • People track their children around • Leads to persecution complex amongchildren • School uniforms become fashionable