Henning Schulzrinne Xiaotao Wu & CINEMA crew Columbia University

1. From multimedia conferencing to context-aware communications Henning Schulzrinne Xiaotao Wu & CINEMA crew Columbia University Internet2 RTC Forum

2. Overview • Old challenge: any media, anywhere, anytime • New challenge: appropriate and context-sensitive communications • not just telephony • not just videoconferencing • on-demand, not special equipment, setup, arrangements • Status of multimedia communications • filling in the protocol matrix • On-going work: • presence-enabled multimedia communications • mobility  terminal, personal, session, service • creating new services in the web model, not the COBOL model • Location-based services • Challenges ahead

3. Internet services – the missing entry

4. Filling in the protocol gap

5. Rendezvous protocol lets users find each other by only knowing a permanent identifier Mobility enabler: personal mobility one person, multiple terminals terminal mobility one terminal, multiple IP addresses session mobility one user, multiple terminals in sequence or in parallel service mobility services move with user SIP as service enabler

6. Example SIP phones about $85

7. Ubiquitous computing aspects • Also related to pervasive computing • Mobility, but not just cell phones • Computation and communications • Integration of devices • “borrow” capabilities found in the environment  composition into logical devices • seamless mobility  session mobility • adaptation to local capabilities • environment senses instead of explicit user interaction • from small dumb devices to PCs • light switches and smart wallpaper

8. Context-aware communications • Traditional emphasis: communicate anywhere, anytime, any media  largely possible today • New challenge: tailor reachability • Context-aware communications • modify when, how, where to be reached •  machine: context-dependent call routing •  human: convey as part of call for human usage • context-aware services • leveraging local resources • awareness of other users • sources of location information • voluntary and automatic • location-based services  privacy concerns • applies to other personal information • activity, reachability, capabilities, bio sensor data, … • emergency services as a location-based service

9. Context • context = “the interrelated conditions in which something exists or occurs” • anything known about the participants in the (potential) communication relationship • both at caller and callee

10. “Legacy” IM & presence systems  SIP-based systems • centralized systems (single name space) • federated systems, similar to email • mostly instant text messages • media-agnostic – transmit any media object • separate from session-based services (VoIP, video conferencing) • integrated: • use IM as part of media sessions • use presence to facilitate session setup • limited presence status, mostly manually set • rich presence, with time information • imported from sensors, calendars, backend systems, … • proprietary systems (AOL, Yahoo!, MSN, ICQ, …) • standards-based systems

11. Presence = special case of event notification “user Alice is available for communication” Human users: multiple contacts per presentity device (cell, PDA, phone, …) service (“audio”) activities, current and planned surroundings (noise, privacy, vehicle, …) contact information composing (typing, recording audio/video IM, …) Multimedia systems: REFER (call transfer) message waiting indication conference floor control conference membership push-to-talk system configuration General events: emergency alert (“reverse 911”) industrial sensors (“boiler pressure too high”) business events (“more than 20 people waiting for service”) Presence and event notification

12. IETF efforts • SIP, SIPPING and SIMPLE working groups • but also XCON (conferencing) • Define SIP methods PUBLISH, SUBSCRIBE, NOTIFY • GEOPRIV: • geospatial privacy • location determination via DHCP • information delivery via SIP, HTTP, … • privacy policies • SIMPLE: • architecture for events and presence • configuration (XCAP) • session-oriented IM (↔ page mode) • filtering, rate limiting and authorization

13. RPID: rich presence • Provide watchers with better information about the what, where, how of presentities • facilitate appropriate communications: • “wait until end of meeting” • “use text messaging instead of phone call” • “make quick call before flight takes off” • designed to be derivable from calendar information • or provided by sensors in the environment • allow filtering by “sphere” – the parts of our life • don’t show recreation details to colleagues

14. Classification: contact-type device, in-person, service, presentity class for labeling sphere “work”, “home”, … relationship “family”, “associate”, “assistant”, “supervisor” Activities: activity “on-the-phone”, “away”, “appointment”, … idle last usage of device Surroundings: placetype “home”, “office”, “industrial”, … privacy “public”, “private” RPID: rich presence

15. CIPID: Contact Information • More long-term identification of contacts • Elements: • card – contact Information • home page • icon – to represent user • map – pointer to map for user • sound – presentity is available

16. Presence is about here & now but often only have (recent) past – e.g., calendar or future “will be traveling in two hours” “will be back shortly” allows watcher to plan communication loose synchronization of calendars <tuple id="7c8dqui"> <contact> sip:bob@example.com </contact> <status> <basic>open</basic> </status> <fs:timed-status from="2003-08-15T10:20:00.000-05:00“ until="2003-08-22T19:30:00.000-05:00"> <basic>closed</basic> </fs:timed-status> </tuple> <note>I'll be in Tokyo next week</note> Timed Status

17. GEOPRIV and SIMPLE architectures rule maker DHCP XCAP (rules) target location server location recipient notification interface publication interface GEOPRIV SUBSCRIBE presentity presence agent watcher SIP presence PUBLISH NOTIFY caller callee SIP call INVITE INVITE

18. Location-based services • Finding services based on location • physical services (stores, restaurants, ATMs, …) • electronic services (media I/O, printer, display, …) • not covered here • Using location to improve (network) services • communication • incoming communications changes based on where I am • configuration • devices in room adapt to their current users • awareness • others are (selectively) made aware of my location • security • proximity grants temporary access to local resources

19. Location-based SIP services • Location-aware inbound routing • do not forward call if time at callee location is [11 pm, 8 am] • only forward time-for-lunch if destination is on campus • do not ring phone if I’m in a theater • outbound call routing • contact nearest emergency call center • send delivery@pizza.com to nearest branch • location-based events • subscribe to locations, not people • Alice has entered the meeting room • subscriber may be device in room  our lab stereo changes CDs for each person that enters the room

20. a@foo.com: 128.59.16.1 SIP URIs for locations location beacon • Identify confined locations by a SIP URI, e.g., sip:rm815@cs.columbia.edu • Register all users or devices in room • Allows geographic anycast: reach any party in the room sip:rm815 Contact: bob Contact: alice Room 815

21. 802.11 Location Tracking • Standard access points • No client software • “Skiff” monitors • SA110 single board computer running Linux • Report signal strength, MAC address of all packets seen by Jamey from HP

22. Privacy  Presence policy SUBSCRIBE subscription policy subscriber (watcher) for each watcher event generator policy subscriber filter rate limiter change to previous notification? NOTIFY

23. Policy relationships common policy geopriv-specific presence-specific future RPID CIPID

24. Conditions identity, sphere, validity time of day current location identity as <uri> or <domain> + <except> Actions watcher confirmation Transformations include information reduced accuracy User gets maximum of permissions across all matching rules Extendable to new presence data rich presence biological sensors mood sensors Privacy rules

25. Example: user-adaptive device configuration “all devices that are in the building” RFC 3082? 802.11 signal strength  location SLP device controller HTTP PA REGISTER To: 815cepsr Contact: alice@cs tftp SUBSCRIBE to each room • discover room URI • REGISTER as contact for room URI SIP room 815 SUBSCRIBE to configuration for users currently in rooms

26. Location-based IM & presence

27. Location-based call routing – UA knows its location GPS INVITE sips:sos@ 40.86N 73.98E CN=us A1=NJ A2=Bergen leonia.nj.us.sos.arpa POLY 40.85 73.97 40.86 73.99 NAPTR … firedept@leoniaboro.org outbound proxy server provided by local ISP? 40.86N 73.98E: Leonia, NJ fire dept. DHCP

28. Service creation • Tailor a shared infrastructure to individual users • traditionally, only vendors (and sometimes carriers) • learn from web models

29. Service creation environment for CPL and LESS

30. location-switch for CPL

31. Challenges • Systems are still too hard to use without wizard assistance: • lack of interoperability (improving) • NAT and other configuration • volume mismatch, echo, … • audio problems not much changed since 1992 • network/system fault diagnosis • Closed wireless systems – would be very nice presence sensors • Threat of “spim” and nuisance calls • Provider platforms remain largely closed • promise of open service creation remains to be fulfilled

32. Conclusion • Standardization mostly complete • even if drafts don’t have RFC numbers yet • Many commercial-grade, second-generation products emerging • both open-source and commercial • emphasis on interoperability • Increasingly hostile network • multi-layer NATs, random port blocking, “transparent” proxies • Usability and reliability remain too low • dial-in audio conference still common • LCD problem (cf. MIME for email)