Using SIP to build Context-Aware VoIP Support for Multiplayer Networked Games

1. Using SIP to build Context-Aware VoIP Support for Multiplayer Networked Games • Arup Acharya • Network Server System Software • IBM T. J. Watson Research Center • arup@us.ibm.com Aameek Singh College of Computing Georgia Institute of Technology aameek@cc.gatech.edu

2. Introduction • Project on “Client and Infrastructure support for SIP-enabled applications”, 2003 • One of the applications for “SIP enablement”: multi-player networked games • Joint work with Aameek Singh starting with his summer internship at IBM Watson, summer 2003 • Motivation : explore how flexibility of SIP-based VoIP can be used in gaming Collaborative Apps Web Browser SameTime VoIP infrastructure Enhancements (conferencing, context) Notes Calendar/email VoIP enabled portals Distributed application scenarios, e.g. networked games Corporate VoIP/data network (Watson VoIP trial) (SIP-enabled) Services/ Infrastructure Client-side SIP service In-house conferencing service (IP/ PSTN phones) Game servers, IM servers

3. What is SIP (Session Initiation Protocol) ? • A signaling protocol for setting up multimedia sessions between endpoints • Fundamental shift from PSTN : infrastructure consists of software on standard servers • SIP designed in line with other Internet protocols by the IETF • Uses overlay control network consisting of SIP Proxies to route SIP messages : Media path (RTP/UDP) decoupled from signaling • name@domain addressing; message syntax similar to HTTP • SIP provides • Session setup/modification/handoff/tear-down : Voice/ Video over IP • Mobility control • Presence & Instant Messaging : Signaling message carries the IM as payload (SIMPLE) • Publish/subscribe mechanism : SUBSCRIBE/ NOTIFY to events • Supports calls to/from PSTN • Service/personal mobility • Examples of SIP adoption • VoIP : Vonage, CableVision,… • IM : Lotus Sametime • Push-to-talk : Sprint PCS, Verizon Wireless • Collaboration software : Microsoft Live Office • 3G SIP proxy SIP proxy SIP signaling RTP/UDP packets

4. 180 (Ringing) 180 (Ringing) 180 (Ringing) 200 (OK) 200 (OK) 200 (OK) 200 (OK) 200 (OK) 200 (OK) INVITE INVITE ACK ACK BYE BYE ACK BYE RTP MEDIA PATH Proxy Proxy INVITE Call Setup User Agent User Agent MediaPath Call Teardown SIP Messages – Methods and Responses VoIP Call setup : SIP message flow NETGAMES 2004

5. Wrapper 1 Wrapper 2 Wrapper 3 Enabling applications with SIP: client-side SIP service • A client-side utility to enable existing (and new) applications to invoke SIP features natively (IM, VoIP, pub/sub,…) • SIP unbundled from applications • SIP service exposes a well-defined API • Allows user choice of end-device (softphone, PSTN / IP Phone) Web Browser IP Phone SIP Stack with pub/sub IM Agent Application Interface SoftPhone Email Client SIP Service Other Apps (Multi-player Games, …) PSTN/ cell phone Client Laptop

6. CONVEDIA conferencing hardware SIP control SIP ConferenceManager Application Server (IM, email, web,..) RTP/UDP Voice packets (multi-party connection) Application protocol SIP Client Laptop Application transfer SIP service SIP+App integration Watson VoIP Pilot + new components (application integration, conferencing service) Cisco 3640 SIP gateway SIP Proxy Server Siemens PBX SIP RTP/UDP Voice packets (point-point connection) PSTN SIP phone POTS phone PSTN phone

7. A new Converged applicationSIP Gaming – Context-aware Conferencing Each square represents a conference (game room). At startup, each player is associated with an IP/ PSTN phone. Arup and Edie can talk to each other. If Arup drags his icon into Aameek’s region, then his current conference with Edie ends and he is seamlessly switched to a conference with Aameek. Illustrates multiple simultaneous conferencing not possible with PSTN. Integrates SIP service with a application scenario (multi-player gaming) Drag with mouse NETGAMES 2004

8. C1 C3 C2 G G G G G S S S S S U1 U2 U4 U5 U3 General Architecture : Service Composition Gaming service SIP (Conferencing) service • Overall goal is to offer SIP client/network service as a building block • Enable composition with other services such as gaming • Context-aware conferencing result of composing SIP and gaming services Server-side association S2 S1 S3 S1, S2, S3 : Concurrent states of the game server C1, C2, C3 : Corresponding Conferences U1, U2,, : Clients G : Gaming client S : SIP Service

9. SIP Based Conferencing Architecture SIP Workflow Back-to-Back UA Control Data 2INVITE Conference Server Mixer 3OK 6ACK 1INVITE 4OK RTP 5ACK UA NETGAMES 2004

10. Integration with Gaming Infrastructure Decentralized Gaming Centralized Gaming Mapping Game state to conferencing state B2B UA B2B UA Conference Server Mixer Conference Server Game Server SIP SIP Signaling SIP Game State Media Path SIP + SIP Signaling Mixer UA UA Conferencing media streams UA UA RTP/ UDP Game State Media Path NETGAMES 2004

11. Mixer SIP UA Game Client Client Prototype Implementation • Centralized Gaming Architecture • High level SIP Interface to control SIP communication Game Server Conference Server Control Agent Control Logic Conference Manager Session Sip Interface SIP Interface SIP stack Sip Stack NETGAMES 2004

12. SIP Workflows – Game Join • User joins the game with UA as its SIP device 0User Joins Game 4INVITE 3INVITE SIP Enabled Game Server Mixer Conference Server 6OK 5OK 9ACK 8ACK 1INVITE 2OK 7ACK 1INVITE contains NO SDP. UA sends 2OK with its SDP. GS gets media path info from that SDP and sends that as media info in 3INVITE. The CS completes transaction with the mixer and sends 6OK. The GS extracts the media info from it and sends it as part of SDP in 7ACK. Also ACKs the CS which ACKs the mixer. Voice pkts SIP UA GAME CLIENT NETGAMES 2004

13. SIP Workflow – Context Switch 1BYE 0User moves to a new room 2BYE SIP Enabled Game Server Conference Server Mixer 4OK 5INVITE 3OK 6INVITE 8OK 7OK 12ACK 13ACK 9RE-INVITE 10OK 11ACK UA 1BYE removes the user from its previous conference. The 5RE-INVITE initiates the process of bringing the user into its new conference, i.e. media flows to different port# of mixer. User sees no (perceptible) interruption. GAME CLIENT NETGAMES 2004

14. Other Features • Sub-Interactions : On-the-fly conferencing amongst ad-hoc groups • No pre-registration of conference name • Configure SIP Proxies to route based on domain names • Conference server creates conference when it sees a name for the first time • Share conference name with other intended members (of ad-hoc group) out-of-band, e.g. IM INVITE SIP Proxy Conference Server conf-server.com INVITE sip:myurl@ conf-server.com SIP Session UA NETGAMES 2004

15. Other Features • Enhanced Audio-Mixing • Use “distance” of player to calibrate “loudness” of voice while mixing multiple voice streams • Requires support from the media mixing hardware • SIP signaling can be used to carry “distance” information via the SIP INFO method

16. Conclusions and Future Work • Idea of context-aware multi-conferencing has broad applicability beyond gaming • Gaming is a metaphor for multi-party collaboration • Examples : a) drag-and-drop an “expert” b) attend >1 conference with a GUI indicator when needed c) coupled with presence • Presented an infrastructure for context-aware communication that is an add-on to a gaming infrastructure • Based on light-weight SIP protocol • Using a generic client-side SIP service • Future Work • Can SIP be used as the gaming protocol itself?

17. Thank you! Questions?Further information: Arup Acharya http://www.research.ibm.com/people/a/arup/ sip:arup@research.ibm.com