Distributed VR

1. Distributed VR Mark Green School of Creative Media

2. Distributed VR • Two or more users share the same virtual environment • aware of each other • can interact with each other • all users can interact with and modify the environment • usually at different locations

3. Issues: Technical • Communications bandwidth and latency • where is the environment stored? • What happens when there are no users? • How do we synchronize users? • What happens if two users want to modify the same object? • How do we detect this in a timely manner?

4. Issues: Non-Technical • Representation of users • users interfering with each other • do all users see the same environment? • private information, what should be shared and when? • Social structure and behavior

5. Architectures • If several users share the same environment they must be able to communicate • how do we do this efficiently? • One of the main problems is distance: • electrons only move so fast, takes a finite amount of time to travel any distance • long distances mean multiple time zones, everyone isn’t awake at the same time

6. Architectures • What needs to be communicated? • The environment (not all of the time) • user state • changes to the environment • ownership and environment state information • some of this information is exchanged frequently (several times per second), others are occasional events

7. Communications: User State • Need accurate up to date representations of other users • a time sequence of their actions • requirements: • low latency • correct order • timely information, missed updates are okay, but can’t have updates out of order

8. Architectures • Main concern is efficient communications • two extremes: • centralized: a central server maintains all the information on the environment, coordinates all user communications • decentralized: each user has their own copy of the environment, users communicate directly with each other

9. Centralized Architecture Server VE Client Client Client

10. Decentralized Architecture VE User VE User VE User User VE

11. Centralized Architecture • One copy of the environment on the server, all changes to the environment must go through the server • all user actions sent to the server, server then sends them to the other users • this is the simplest architecture to develop, all communications is controlled and centralized

12. Centralized Architecture • When client joins the session the current version of the environment is sent • list of all active users, and their representations is also sent • each user sends position and actions to the server frequently • maybe not every update, but 5 or 10 times per second

13. Centralized Architecture • Server collects user information, then sends all of this information to other users • again this happens 5 or 10 times per second • what happens when a user wants to interact with an object in the environment • user must request “ownership” of the object • by default the server owns all of the objects

14. Centralized Architecture • If server currently owns the object, ownership is transferred to the user • if another user owns the object: • user denied access to the object • other user may be requested to release the object, particularly if it hasn’t used it recently • once the user has ownership, it is free to manipulate it

15. Central Architecture • Each manipulation is sent to the central server, central server sends manipulations to other users • when user is finished manipulating the object it releases ownership • Note: in this model only one user can own an object at a time, no collaboration

16. Central Architecture • This approach has numerous advantages, besides simplicity • one central copy of the environment, we have a consistent representation of the environment • users must register, so we have a charging mechanism, important for commercial applications

17. Central Architecture • Server can package user updates, send as one packet to all users, this can save network bandwidth • need to understand a little bit about how networks work • information isn’t sent as a stream of characters (except in the case of modems), instead packets of bytes are used

18. Central Architecture • All networks have a minimum and maximum packet size – like boxes • minimum is rarely smaller than 64 bytes, maximum rarely more than 1500 bytes • if the information is smaller than the minimum packet size it is padded with extra garbage bytes to reach the minimum size • makes sense to send larger packets

19. Central Architecture • Observation: not just the amount of information that impacts network performance, also how it is packaged • ownership mechanism controls how objects in environment are manipulated • fair sharing of the objects, occurs in a controlled manner • in theory no surprises

20. Central Architecture • The central architecture also has a number of problems • one of the key problems is scalability • as we add more users the central server becomes a bottleneck, it must process an increasing number of messages • at some point it can’t keep up and all users suffer

21. Central Architecture • Another problem is with interaction • user must request ownership, this involves a message transfer to and from the server • this may take several seconds if the server is some distance away, or is busy • users don’t request ownership, the client program does, users aren’t aware of this

22. Central Architecture • User starts manipulating object, at this point client requests ownership • what happens while client waits for a response from server? • Do we let the user continue? • This is okay if we get ownership, but what happens if ownership is refused? • Do we reverse all the user’s actions?

23. Central Architecture • We could stop the user until ownership is received • user doesn’t know why program has stopped, thinks the program has a bad user interface • this is a no win situation, no matter what we do the user will be upset • problems when two users grab the same object at close to the same time

24. Central Architecture • The central architecture works well under the following conditions: • the server is close • small number of users, below the point where the server gets overloaded • relatively simple environments, no complex objects that users might want to share

25. Decentralized Architecture • Each user has their own copy of the environment • users communicate with each other directly • we remove the server bottleneck, no need to go to the server • this architecture is more complex, since their isn’t a single official copy of the environment

26. Decentralized Architecture • What happens when a new user joins? • New user has a copy of the initial state of the environment, all the objects in their initial positions • this probably isn’t the current state, other users have moved objects • send a “new user” message to one of the existing users

27. Decentralized Architecture • Response is the list of current users and the current state of the environment • new user informs all of the other users that he exists, and they start sending updates to him • not as fast as centralized architecture, may be some time before the new user has up to date copy of the environment

28. Decentralized Architecture • Each user sends updates of its state to all of the other users, again this happens 5 to 10 times per second • receives back the updated information from the other users • all the updates may not arrive at the same time, depends on network and processor speed of other users

29. Decentralized Architecture • Advantage: don’t need to wait for slower client computers • fast computers can exchange information more frequently, smoother motion and interaction • what happens when a user manipulates an object? • There are several possibilities

30. Decentralized Architecture • Can still use an ownership scheme to keep all copies consistent • each object will have an owner, one of the users • need to find the user who owns the object, request ownership from that user • object can store its owner, so we don’t need to search for it

31. Decentralized Architecture • This could be faster than centralized approach, depends on how close the other user is, how busy their computer is • can claim ownership, and see if anyone complains, this is much faster, but could lead to problems • what happens if several users claim ownership, modify object in different ways

32. Decentralized Architecture • End up with multiple versions of the environment, inconsistent in state of this object • Is this a problem? • Depends upon the object and application • if we have a game and the object is important for winning then object ownership is very important

33. Decentralized Architecture • If it is a minor object, this may not be a problem • Example: notepad in a meeting environment • each user can claim their own copy of the notepad and write their own notes • in this case inconsistency in the environment isn’t a big issue, its more of a feature

34. Decentralized Architecture • When a user leaves the environment, he gives up ownership of his objects • what happens when the last person leaves? • Is there a record of the final state of the environment? • If this is important there should be a non-human user whose main responsibility is to save the environment

35. Decentralized Architecture • In many cases we don’t need to save the environment • in an meeting environment, once the meeting is over there is nothing to save • same thing for training simulators, once the training session is over we don’t need to save anything, each session starts with a new scenario

36. Decentralized Architectures • Decentralized architectures have some major advantages • they can scale better, don’t depend on centralized server that can be a bottleneck • more flexible ways of sharing objects, don’t need to have complete consistency, since there are multiple copies of the environment

37. Decentralized Architectures • May or may not have higher bandwidth requirements • this is hard to evaluate, most models are too simplistic • if there are n users, each user will send n-1 update message and receive n-1 updates • this gives 2n(n-1) messages in total in each update

38. Decentralized Architecture • With a centralized server, each user sends one message to server, and gets one back, the return message is quite large so can be counted as n messages • this gives a total of n+1 messages • its appears that bandwidth requirements grow faster for a decentralized architecture • if using one network segment this is the case

39. Decentralized Architecture • In centralized architecture, all packets must go through the server, so there is a single network connection that carries all of the traffic • in a decentralized architecture there are multiple network connections carrying all of the traffic, possibly one for each user • since all users don’t update at the same rate, the bandwidth required could be lower

40. Decentralized Architecture • On more closer examination, there could be a separate path for each pair of users • each user will still have a packet coming in from each user and one going out to each user, a total of 2(n-1) • the central server receives a message from every user, and sends a message to every user, sent message is n times as large

41. Decentralized Architecture • Central server total traffic is n + n(n-1) • thus the central server has a much higher bandwidth requirement than any user in a decentralized architecture • as long as the users are evenly distributed over network segments (probably one per segment) decentralized will have better network performance

42. Decentralized Architectures • There are a number of disadvantages • one of the main ones is complexity, the user programs are more complex, they have more things to do • the network protocol is more complex • this requires more programming and there are more things that can go wrong

43. Decentralized Architectures • There is less control • harder to keep all versions of the environment the same • can’t easily update the environment • no easy way to charge people to use the system • harder to enforce the rules

44. Architectures • Rarely use pure centralized or decentralized architectures due to scaling problems • in the case of centralized architectures use multiple servers, each one with part of the users • servers must still communicate, but fewer messages in and out of each server

45. Architectures • If there are a large number of users they will not be able to see each other and communicate with each other • not likely that more than 10 or 20 users will communicate as a group • this means that we don’t need to send all updates to all users, only the ones that we are close to

46. Architectures • This benefits the decentralized architectures the most • need to know the users to send to, this may require some central coordination • some network techniques can be used • for centralized, servers still need to collect all user information, then decide which users get which information

47. User Representation • If we have multiple users in the same VE we need some way of representing them • you need to see where the other users are and what they are doing • this is both a technical and social issue • depends on the application and people involved • Avatar: term used for user representation

48. Avatars • What do we need to represent and how should it be represented? • This partly depends on the application and the available technology • for conference and meeting applications an accurate representation of the users is important • need eye contact and facial expressions

49. Avatars • For other applications a realistic representation isn’t desirable • realistic representations require resources: • space: can hide important parts of the application • display: takes considerable amount of time to display, display resources should be devoted to more important parts of the application

50. Avatars • What is the minimal amount of information we need? • Where the other user is located • the direction they are looking • manipulations they are performing • minimal representation can use coordinate axis to represent position and direction of sight