The Vision and Reality of Ubiquitous Computing - PowerPoint PPT Presentation

The vision and reality of ubiquitous computing l.jpg
Download
1 / 57

The Vision and Reality of Ubiquitous Computing. Prof. Henning Schulzrinne Dept. of Computer Science Columbia University (with Arezu Moghadam, Ron Shacham, Suman Srinivasan, Xiaotao Wu and other IRT members; parts in cooperation with DoCoMo Eurolabs). Overview.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

The Vision and Reality of Ubiquitous Computing

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


The vision and reality of ubiquitous computing l.jpg

The Vision and Reality of Ubiquitous Computing

Prof. Henning Schulzrinne

Dept. of Computer Science

Columbia University

(with Arezu Moghadam, Ron Shacham, Suman Srinivasan,Xiaotao Wu and other IRT members; parts in cooperation with DoCoMo Eurolabs)

Mobiquitous 2007


Overview l.jpg

Overview

  • The original vision of ubiquitous computing

  • User challenges

  • Beyond terminal mobility

  • Location as new core service

  • Universality: 7DS

Mobiquitous 2007


Ubiquitous computing ubiquitous communications l.jpg

Ubiquitous computing  ubiquitous communications

  • “It is invisible, everywhere computing that does not live on a personal device of any sort, but is in the woodwork everywhere.”

  • Weiser’s original vision (“Nomadic Issues in Ubiquitous Computing”, 1996)

    • “one person, many computers”

    • many computers embedded in environment

    • dynamic ownership

    • PC phonebooth

    • “IR use will grow rapidly”

  • Updated version, 2007

    • not physically invisible, but transparent

    • emphasis on communications, not computing

    • most devices are mobile (or nomadic)

    • cheap electronics personal devices

    • radio (channelized and UWB)

Mobiquitous 2007


Overview4 l.jpg

Overview

  • The original vision of ubiquitous computing

  • User challenges

  • Beyond terminal mobility

  • Location as new core service

  • Universality: 7DS

Mobiquitous 2007


User challenges vs research challenges l.jpg

User challenges vs. research challenges

  • Are we addressing real user needs?

    • Engineering vs. sports

  • My guesses

ease of use

no manual

reliability

no re-entry

no duplication

integration

phishing

data loss

cost

limited risk

Mobiquitous 2007


Example email configuration l.jpg

Example: Email configuration

  • Application configuration for (mobile) devices painful

  • SMTP port 25 vs. 587

  • IMAP vs. POP

  • TLS vs. SSL vs. “secure authentication”

  • Worse for SIP...

Mobiquitous 2007


Example sip configuration l.jpg

Example: SIP configuration

partially explains

  • highly technical parameters, with differing names

  • inconsistent conventions for user and realm

  • made worse by limited end systems (configure by multi-tap)

  • usually fails with some cryptic error message and no indication which parameter

  • out-of-box experience not good

Mobiquitous 2007


Mobile why s l.jpg

Mobile why’s

  • Not research, but examples of real annoyances

  • Why does each mobile device need its own power supply?

  • Why do I have to adjust the clock on my camera each time I travel?

  • Why do I have to know what my IMAP server is and whether it uses TLS or SSL?

  • Why do I have to type in my address book?

  • Why do I have to “synchronize” my PDA?

  • Why do I have to manually update software?

  • Why is connecting a laptop to a projector a gamble?

  • Why do we use USB memory sticks when all laptops have 802.11b?

Mobiquitous 2007


Consumer wireless mobile devices l.jpg

Consumer wireless & mobile devices

Prius key

Garage door opener

TAN display

Water leak alarm

wireless door bell

MSN Direct weather

Mobiquitous 2007


Mobile systems reality l.jpg

Mobile systems - reality

GPS

  • idea: special purpose (phone) --> universal communicator

    • idea is easy...

  • mobile equipment: laptop + phone

    • sufficiently different UI and capabilities

  • we all know the ideal (converged) cell phone

  • difficulty is not technology, but integration and programmability

    • (almost) each phone has a different flavor of OS

    • doesn’t implement all functionality in Java APIs

    • no dominant vendor (see UNIX/Linux vs. Microsoft)

    • external interfaces crippled or unavailable

      • e.g., phone book access

location data

Mobiquitous 2007


Example displays and speakers l.jpg

Example: displays and speakers

Mobiquitous 2007


The mobile ubiquitous challenge l.jpg

The mobile ubiquitous challenge

Mobile phone

Mobile Internet access

Interconnected devices

“Internet of things”

Mobiquitous 2007


What do we need l.jpg

What do we need?

  • Standards, not new technology

  • Radio connectivity

    • 802.11a/b/g/n, 802.15.4 

    • better discovery of networks

  • Location information everywhere

  • Discovery: devices & services

    • network-local discovery via Bonjour (mDNS) 

    • missing: location-based discovery

  • Advanced mobility: session, personal, service

  • Event notification

  • Data formats

    • location, sensor events, ...

Mobiquitous 2007


Examples of invisible behavior l.jpg

Examples of “invisible” behavior

  • MP3 player in car automatically picks up new files in home server

  • A new email with vcard attachment automatically updates my cell phone address book

  • The display of my laptop appears on the local projector

    • without cable or configuration

  • I can call people I just met at Mobiquitous

    • without exchanging business cards

  • My car key opens my front door

  • My cell phone serves as a TAN (one-time password) generator

  • My cell phone automatically turns itself off during a lecture

  • My camera knows where the picture was taken

Mobiquitous 2007


An interconnected system l.jpg

An interconnected system

opens doors

generates TAN

incoming call

updates location

time, location

address book

alert, events

any weather service

school closings

acoustic alerts

Mobiquitous 2007


Thinking beyond 802 11 and umts l.jpg

Thinking beyond 802.11 and UMTS

  • Many interesting networks beyond those covered in conferences

    • ease of access by researchers vs. importance

    • 90% of papers on 802.11b and maybe GPRS, BlueTooth

  • New wireless networks

    • broadcast instead of unicast -- useful for many ubiquitous applications

    • S5 for low-rate sensors (city scale)

    • Zigbee (802.15.4) for local sensors (20 - 250 kb/s)

    • FM subcarrier (not really new) -- MSN Direct

    • FM Radio Data System -- TMC

    • Sirius / XM

    • HD radio

    • paging

Mobiquitous 2007


Overview17 l.jpg

Overview

  • The original vision of ubiquitous computing

  • User challenges

  • Beyond terminal mobility

  • Location as new core service

  • Universality: 7DS

Mobiquitous 2007


Application layer mobility l.jpg

Application-layer mobility

  • terminal mobility

    • one terminal, multiple network addresses

  • Personal mobility

    • one person, multiple terminals

    • e.g., Grandcentral

  • session mobility

    • one user, multiple terminals in sequence or in parallel

  • service mobility

    • services move with user

Mobiquitous 2007


Session mobility l.jpg

Session mobility

  • Walk into office, switch from cell phone to desk phone

    • call transfer problem  SIP REFER

  • related problem: split session across end devices

    • e.g., wall display + desk phone + PC for collaborative application

    • assume devices (or stand-ins) are SIP-enabled

    • third-party call control

R. Shacham, H. Schulzrinne, S. Thakolsri, W. Kellerer, “Ubiquitous device personalization and use:The next generation of IP multimedia communications”, ACM TOMCCAP, May 2007

Mobiquitous 2007


How to find services l.jpg

How to find services?

  • Two complementary developments:

    • smaller devices carried on user instead of stationary devices

    • devices that can be time-shared

      • large plasma displays

      • projector

      • hi-res cameras

      • echo-canceling speaker systems

      • wide-area network access

  • Need to discover services in local environment

    • SLP (Service Location Protocol) allows querying for services

      • “find all color displays with at least XGA resolution”

      • slp://example.com/SrvRqst?public?type=printer

    • SLP in multicast mode

    • SLP in DA mode

    • Apple Bonjour

  • Need to discover services before getting to environment

    • “is there a camera in the meeting room?”

    • SLP extension: find remote DA via DNS SRV

    • LoST to find services by geographic location

Mobiquitous 2007


Session mobility21 l.jpg

Session mobility

Local Devices

Transcoder

Internet

SLP DA

SLP UA

SLP SA

SIP SM

SIP UA

SIP UA

Correspondent

Node (CN)

SLP

SIP

RTP

SIP SM

SIP UA

SLP UA

Mobiquitous 2007

Mobile Node (MN)


Overview22 l.jpg

Overview

  • The original vision of ubiquitous computing

  • User challenges

  • Beyond terminal mobility

  • Location as new core service

  • Universality: 7DS

Mobiquitous 2007


Context aware communication l.jpg

Context-aware communication

  • context = “the interrelated conditions in which something exists or occurs”

  • anything known about the participants in the (potential) communication relationship

  • both at caller and callee

Mobiquitous 2007


Geopriv and simple architectures l.jpg

GEOPRIV and SIMPLE architectures

rule

maker

DHCP

XCAP

(rules)

target

location

server

location

recipient

notification

interface

publication

interface

GEOPRIV

SUBSCRIBE

presentity

presence

agent

watcher

SIP

presence

PUBLISH

NOTIFY

caller

callee

SIP

call

INVITE

INVITE

Mobiquitous 2007


Presence data architecture l.jpg

Presence data architecture

presence sources

PUBLISH

raw

presence

document

privacy

filtering

create

view

(compose)

depends on watcher

XCAP

select best source

resolve contradictions

XCAP

privacy

policy

composition

policy

(not defined yet)

draft-ietf-simple-presence-data-model

Mobiquitous 2007


Presence data architecture26 l.jpg

Presence data architecture

candidate

presence

document

raw

presence

document

post-processing

composition

(merging)

watcher

filter

remove data not of

interest

SUBSCRIBE

difference

to previous notification

final

presence

document

watcher

NOTIFY

Mobiquitous 2007


Presence data model l.jpg

Presence data model

“calendar”

“cell”

“manual”

person

(presentity)

(views)

alice@example.com

audio, video, text

r42@example.com

video

services

devices

Mobiquitous 2007


Rpid rich presence l.jpg

RPID = rich presence

  • Provide watchers with better information about the what, where, how of presentities

  • facilitate appropriate communications:

    • “wait until end of meeting”

    • “use text messaging instead of phone call”

    • “make quick call before flight takes off”

  • designed to be derivable from calendar information

    • or provided by sensors in the environment

  • allow filtering by “sphere” – the parts of our life

    • don’t show recreation details to colleagues

Mobiquitous 2007


Rich presence l.jpg

Rich presence

  • More information: for (authorized) people and applications

  • automatically derived from

    • sensors: physical presence, movement

    • electronic activity: calendars

  • Rich information:

    • multiple contacts per presentity

      • device (cell, PDA, phone, …)

      • service (“audio”)

    • activities, current and planned

    • sphere (home vs. work)

    • current user mood

    • surroundings (noise, privacy, vehicle, …)

    • contact information

    • composing (typing, recording audio/video IM, …)

Mobiquitous 2007


Presence and privacy l.jpg

Presence and privacy

  • All presence data, particularly location, is highly sensitive

  • Basic location object (PIDF-LO) describes

    • distribution (binary)

    • retention duration

  • Policy rules for more detailed access control

    • who can subscribe to my presence

    • who can see what when

<tuple id="sg89ae">

<status>

<gp:geopriv>

<gp:location-info>

<gml:location>

<gml:Point gml:id="point1“

srsName="epsg:4326">

<gml:coordinates>37:46:30N 122:25:10W

</gml:coordinates>

</gml:Point>

</gml:location>

</gp:location-info>

<gp:usage-rules>

<gp:retransmission-allowed>no

</gp:retransmission-allowed>

<gp:retention-expiry>2003-06-23T04:57:29Z

</gp:retention-expiry>

</gp:usage-rules>

</gp:geopriv>

</status>

<timestamp>2003-06-22T20:57:29Z</timestamp>

</tuple>

Mobiquitous 2007


Events as missing internet capability l.jpg

Events as missing Internet capability

  • aka PUB/SUB

  • Used across applications, e.g.,

    • email and voicemail notification

    • presence

    • replace RSS (= poll!)

    • web service completion

    • emergency alerts (“reverse 9-1-1”)

    • network management

    • home control

    • data synchronization

  • Rich research history

    • but too complex, optimize the wrong thing

  • XMPP and SIP as likely transport candidates

Mobiquitous 2007


Slide32 l.jpg

Local Switch

Automatic

Number

Identification

Automatic

Location

Identification

Collaboration between

local phone providers and

local public safety agencies

Mobiquitous 2007


911 technology failures l.jpg

911 technology failures

  • NY Times (“An S O S for 911 Systems in Age of High-Tech”), 4/6/07:

    • “40% of … counties, most of them rural or small-town …, cannot yet pinpoint the location of the cellphone callers, though the technology to do so has been available for at least five years.”“In … Okmulgee, Okla., last November, 4-year-old Graciella Mathews-Tiger died in a house fire after a 911 operator who lacked the technology to pinpoint the call misheard the address.”

      • Phase II wireless; billions of dollars spent

      • In Mississippi, only 1 of out 5 counties

    • “As it ages, it is cracking, with problems like system overload, understaffing, misrouted calls and bug-ridden databases leading to unanswered calls and dangerous errors.”

      • operator (CAMA) trunks, with 8-digit number delivery

      • MSAG and ALI databases

Mobiquitous 2007


Location delivery l.jpg

Location delivery

GPS

HELD

HTTP

wire

map

DHCP

LLDP-MED

Mobiquitous 2007


Location location location l.jpg

Location, location, location, ...

Voice Service Provider (VSP)

sees emergency call

but does not know caller location

ISP/IAP knows user location

but does not handle call

Mobiquitous 2007


Options for location delivery l.jpg

Options for location delivery

  • Wireless

    • GPS

    • S5 wireless (active sensors + triangulation)

    • Skyhook (802.11 in urban areas)

    • HDTV

  • L2: LLDP-MED (standardized version of CDP + location data)

    • periodic per-port broadcast of configuration information

  • L3: DHCP for

    • geospatial (RFC 3825)

    • civic (RFC 4676)

  • L7: proposals for retrievals: HELD, SIP, …

    • for own IP address or by third party (e.g., ISP to infrastructure provider)

    • by IP address

    • by MAC address

    • by identifier (conveyed by DHCP or PPP)

    • HTTP-based

Mobiquitous 2007


Locating caller using lldp med l.jpg

Locating Caller using LLDP-MED

LLDP-MED stands for: *

Link Layer Discovery Protocol

“a vendor-neutral Layer 2 protocol that allows a network device to advertise its identity and capabilities on the local network.”

Media Endpoint Discovery

“an enhancement to the LLDP that allows discovery of other things including location“

* From Wikipedia

“I am LLDP-MED Capable.

I can process location information.”

“Your location is:

500 W 120TH st. New York NY 10027”

Mobiquitous 2007


Location determination options l.jpg

Location determination options

Mobiquitous 2007


Sip message for location info l.jpg

SIP message for Location Info.

------- =_ZGY1NTFlZDJkMDkxY2FkMTIxMWI2MzIzNjE1M2U0OTY=

MIME-Version: 1.0

Content-Type: application/pidf+xml

Content-Transfer-Encoding: 8bit

<?xml version="1.0" encoding="ISO-8859-1"?>

<presence xmlns="urn:ietf:params:xml:ns:pidf"

xmlns:gp="urn:ietf:params:xml:ns:pidf:geopriv10"

xmlns:cl=" urn:ietf:params:xml:ns:pidf:geopriv10:civilLoc"

xmlns:gml="urn:opengis:specification:gml:schema-xsd:feature:v3.0"

entity="sip:calltaker_ny2@irt.cs.columbia.edu">

<tuple id="28185">

<status>

<gp:geopriv>

<gp:location-info>

<cl:civilAddress>

<cl:country>us</cl:country>

<cl:A1>ny</cl:A1>

<cl:A2>new york</cl:A2>

<cl:A3>new york</cl:A3>

<cl:A6>amsterdam</cl:A6>

<cl:HNO>1214</cl:HNO>

</cl:civilAddress>

</gp:location-info>

<gp:method>Manual</gp:method>

</gp:geopriv>

</status>

<contact priority="0.8">sip:eddie@160.39.54.70:5060</contact>

<timestamp>2005-09-26T15:57:34-04:00</timestamp>

</tuple>

</presence>

------- =_ZGY1NTFlZDJkMDkxY2FkMTIxMWI2MzIzNjE1M2U0OTY=--

INVITE urn:service:sos SIP/2.0

request line

To: urn:service:sos

Call-ID: 763782461@192.168.1.106

Via: SIP/2.0/TCP 192.168.1.106:4064;rport

Content-Type: multipart/mixed; boundary

From: sip:caller@irt.cs.columbia.edu

Contact: <sip:eddie@160.39.54.70:5060>

CSeq: 1 INVITE

Content-Length: 1379

header fields

------ =_ZGY1NTFlZDJkMDkxY2FkMTIxMWI2MzIzNjE1M2U0OTY=

MIME-Version: 1.0

content-Type: application/sdp

Content-Transfer-Encoding: 8bit

v=0

o=eddie 1127764654 1127764654 IN IP4 192.168.1.106

s=SIPC Call

c=IN IP4 160.39.54.70

t=0 0

m=audio 10000 RTP/AVP 0 3

m=video 20000 RTP 31

SDP

PIDF-LO

Mobiquitous 2007


Slide40 l.jpg

Mobiquitous 2007


Tracking l.jpg

Tracking

Mobiquitous 2007


Data formats location l.jpg

Data formats: location

  • Civic (street)

    • jurisdictional & postal

  • Geo (longitude & latitude)

    • point, polygon, circle, …

  • see GeoRSS for simple example

Mobiquitous 2007


Ecrit lost functionality l.jpg

Civic as well as geospatial queries

civic address validation

Recursive and iterative resolution

Fully distributed and hierarchical deployment

can be split by any geographic or civic boundary

same civic region can span multiple LoST servers

Indicates errors in civic location data  debugging

but provides best-effort resolution

Can be used for non-emergency services:

directory and information services

pizza delivery services, towing companies, …

ECRIT: LoST Functionality

<findService xmlns="urn:…:lost1">

<location profile="basic-civic">

<civicAddress>

<country>Germany</country>

<A1>Bavaria</A1>

<A3>Munich</A3>

<A6>Neu Perlach</A6>

<HNO>96</HNO>

</civicAddress>

</location>

<service>urn:service:sos.police</service>

</findService>

Mobiquitous 2007


Lost location to url mapping l.jpg

LoST: Location-to-URL Mapping

VSP1

cluster serving VSP1

replicate

root information

cluster

serves VSP2

123 Broad Ave

Leonia

Bergen County

NJ US

LoST

root

nodes

NJ

US

NY

US

sip:psap@leonianj.gov

search

referral

Bergen County

NJ US

Leonia

NJ US

Mobiquitous 2007


Lost architecture l.jpg

LoST Architecture

G

tree guide

G

G

G

broadcast (gossip)

T1: .us

T2: .de

G

resolver

T2

(.de)

seeker

313 Westview

Leonia, NJ US

T3

(.dk)

T1

(.us)

Leonia, NJ  sip:psap@leonianj.gov

Mobiquitous 2007


Left to do event notification l.jpg

Left to do: event notification

  • notify (small) group of users when something of interest happens

    • presence = change of communications state

    • email, voicemail alerts

    • environmental conditions

    • vehicle status

    • emergency alerts

  • kludges

    • HTTP with pending response

    • inverse HTTP --> doesn’t work with NATs

  • Lots of research (e.g., SIENA)

  • IETF efforts starting

    • SIP-based

    • XMPP

Mobiquitous 2007


Overview47 l.jpg

Overview

  • The original vision of ubiquitous computing

  • User challenges

  • Beyond terminal mobility

  • Location as new core service

  • Universality: 7DS

Mobiquitous 2007


Problems with wide area wireless l.jpg

Problems with Wide Area Wireless

  • 802.11 currently hard to deploy across city or large area

  • Problem: How can mobile devices / gadgets get information while on the move?

  • Use local peer-to-peer wireless networks to exchange information

    • Peers can get information they do not have from another peer

Solution: 7DS!

Mobiquitous 2007

S. Srinivasan, A. Moghadam, S.G. Hong, H. Schulzrinne,

“7DS - Node Cooperation and Information Exchange in Mostly Disconnected Networks”, ICC '07. June 2007.


How 7ds works l.jpg

How 7DS Works

  • When devices are in the same BSS (Basic service set) of 802.11 ad-hoc network, they discover each other using service discovery of Zeroconf

zeroconf

Mobiquitous 2007


How 7ds works50 l.jpg

How 7DS Works

  • If there is no Internet connection, the devices can communicatewith each other to exchange information

Internet

Mobiquitous 2007


Web delivery model l.jpg

Web Delivery Model

  • 7DS core functionality: Emulation of web content access and e-mail delivery

Mobiquitous 2007


Design l.jpg

Design

  • Peer-to-peer network set up using zeroconf

    • Protocol enables devices to communicate with each other without a DHCP server, a DNS server and a Directory server

  • Proxy server serves content

  • Search engine searches for local data

  • MTA store and forward

  • In progress: File synchronization, BBS

Mobiquitous 2007


Store and forward l.jpg

Store and Forward

  • Forwarding e-mail in the ad-hoc network

  • Acts as an MTA

Mobiquitous 2007


Search engine l.jpg

Search Engine

  • Provides ability to query self for results

  • Searches the cache index using Swish-e library

  • Presents results in any of three formats: HTML, XML and plain text

  • Similar in concept to Google Desktop

Mobiquitous 2007


Query multicast engine l.jpg

Query Multicast Engine

  • Used to actually exchange information among peers

  • Requesting peer broadcasts a query to the network

  • Responding peers reply if they have information

    • Send encoded string with list of matching items

  • Requesting peer retrieves suitable information

Mobiquitous 2007


File synchronization l.jpg

File Synchronization

SERVICE ANNOUNCEMENT

SERVICE RESOLUTION

FILE SYNCHRONIZATION USING RSYNC PROTOCOL

SRV : 7ds-fs1.filesync._7ds._udp.local.

7ds-device1.local:2525

TXT : file1.xml

TXT : file2.xml

“I want

Word.doc and presentation.ppt”

Word.doc

Presentation.ppt

“I want

File1.xml and file2.xml”

SRV : 7ds-fs2.filesync._7ds._udp.local.

7ds-device2.local:2525

TXT : word.doc

TXT : presentation.ppt

File1.xml

File2.xml

File1.xml

File2.xml

Word.doc

Presentation.ppt

Mobiquitous 2007


Conclusion l.jpg

Conclusion

  • Motivate mobile ubiquitous research by user problems

  • From stovepipe mobile & wireless systems to personal and shareable wireless networks

  • Thinking beyond single applications

    • presence event notification

    • 9-1-1 location  time & location as infrastructure

  • Need new models of creating services

    • domain-specific languages, not Java APIs

Mobiquitous 2007


  • Login