BlindSight:
Sponsored Links
This presentation is the property of its rightful owner.
1 / 77

blindSight: Eyes-free mobile phone interaction PowerPoint PPT Presentation


  • 101 Views
  • Uploaded on
  • Presentation posted in: General

blindSight: Eyes-free mobile phone interaction. Kevin Li , University of California, San Diego Patrick Baudisch , Microsoft Research Ken Hinckley , Microsoft Research. blindSight. “How ab out Monday morning?”. calendar. “Monday 9am”. preview. “tic, tic, sssssh”.

Download Presentation

blindSight: Eyes-free mobile phone interaction

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


blindSight:

Eyes-free

mobile phone interaction

Kevin Li, University of California, San Diego

Patrick Baudisch, Microsoft Research

Ken Hinckley, Microsoft Research


blindSight

“How about Monday morning?”

calendar

“Monday 9am”

preview

“tic, tic, sssssh”

“Yeah, looks likeI’m free after 10”


blindSight

is an application running on Microsoft Windows Smartphone

is launched when user places or receive a call. It then replaces the in-call menu

unlike the in-call menu, blindSight uses auditory feedback


why?


PCs…

PC screens have the users’ undivided attention

 design for the visual channel


eyePhone


environment


visual impairment


screen-less device


can’t seescreen


Lots of information is stored on

mobile phones…


… the interfaces are visual


survey

“I need to access as part of a phone conversation:”

# of participants


  • Ok, so let’s just translate all text from visual to auditory

    • “Menu: Press 1 to search contacts; press 2 to add a contact; press 3 to access your calendar…”

  • Wait, that sounds familiar


Please listen carefully as ouroptions have changed…


related work


interactive voice

response

  • User’s should be able to “dial ahead” [Perugini et al.,CHI 2007]

  • Zap and Zoom allows users to jump to locations using shortcuts [Hornstein, UBILAB Rep 1994]

  • Use visual channel to inform users about options [Yin and Zhai, CHI 2006]


phone interactionmid-conversation

  • Time compress audio[Dietz and Yerazunis, UIST 2001]

  • Integrate speech commands into the conversation [Lyons et al., CHI 2004]


blindSight’s

auditory feedback


audio is heard only by the user,not by the person at the other end


rationale

human-human conversation contains redundancy

 people can recover from audio interruptions

as long as interruption is short

can we use this redundancy to injectauditory feedback from the device?


how do we make sure device feedback fits into these time windows of low information content?


rules

1. feedback only on-demand

home

recordvoice

addcontact

calendar

hear

voice note

findcontact

hearemails

heartask list

hear textmessage

speaker phone

mute


rules

type 6

“200 hits”

type 2

“12 hits”

type 7

“Marion”

2. brevity

find contact

2

1

3

abc

def

5

4

6

jkl

ghi

mno

8

7

9

pqrs

tuv

wxyz

play

delete

next


rules

(what if the content is a long list,such as appointments for a day?)

3. non-speech previews of composites

calendar

+

whereAmIgo today

week

week

+

previewday

day

day

preview3 hours

3 hours

3 hours

_

block½h

½ hour

½ hour

++


rules

(what if the content is a long list,such as appointments for a day?)

4. decomposition

+

whereAmIgo today

week

week

+

previewday

day

day

preview3 hours

3 hours

3 hours

_

next

½ hour

½ hour


rules

5. interruptability

user interface runs as a separate thread


rules

modes

2

2

1

1

3

3

5

5

4

4

6

6

8

8

7

7

9

9

action

action

delete

delete

save

save

6. minimizemodes

pick day

start time

end time

tue

mon

wed

fri

thu

sat

sun

action

delete

save


rules

6. minimizemodes (avoid wizards)

whereAmIgo today

+

week

week

+

previewday

day

day

preview3 hours

+

3 hours

3 hours

_

+

block½h

½ hour

½ hour

++


home

recordvoice

addcontact

calendar

hear

voice note

findcontact

hearemails

heartask list

hear textmessage

speaker phone

mute


add contact

2

1

3

5

4

6

8

7

9

0

delete

save


patterns

iterator

menu

2

1

3

5

4

6

8

action

7

9

action

delete

save


calendar

whereAmIgo today

+

week

week

+

previewday

day

day

preview3 hours

+

3 hours

3 hours

_

block½h

+

½ hour

½ hour

++


demo video

(shows fast usage by an experienced user)


hardware


space


epoxy dots

enlarged spaces


error

Flip

Ear

Visual


.

2

1

3

5

4

6

8

7

9

*

0

#


error

Flip

Ear

Visual


blindSight

evaluation


interfaces

vs.

BlindSight (eyes-free)

Smartphone 2003 (sighted)


task

(1) schedule appointments and (2) add contacts

idle

while “driving”


results

Prefer Overall

Prefer if driving and talking

Better for setting meeting times

Felt in control of the conversation

Knew what day/time I was at

Knew position in the menu

Was not missing information

blindSight

Smartphone

0

1

2

3

4

5

6

7

8

Overall preference


lessons

  • 1. brevity is good, but use in moderationclarification of navigation overrides brevity

  • 2. predictable/modeless user interface is key

  • 3. auditory feedback goes a long way even during phone call(disclaimer: need to study how it interferes with activities… driving)


next:

can’t seescreen

environment

screen-lessdevice

visual impairment


eyePhone


?

eyePhone

eyesFreePhone


blindSight:

Eyes-free

mobile phone interaction

Kevin Li, University of California, San Diego

Patrick Baudisch, Microsoft Research

Ken Hinckley, Microsoft Research


extra slides


contributions

  • built a system

  • a set of eyes-free design rules

  • keypad modifications enabling eyes-free

  • user study comparing with a product (Smartphone 2003)


rules

1. feedback only on-demand

2. brevity

3. non-speech previews of composites

4. decomposition

5. interruptability

6. minimize modes


patterns

iterator

menu

2

1

3

5

4

6

8

action

7

9

action

delete

save


home

recordvoice

addcontact

calendar

hear

voice note

findcontact

hearemails

heartask list

hear textmessage

speaker phone

mute


add contact

2

1

3

5

4

6

8

7

9

0

delete

save


patterns

iterator

menu

2

1

3

5

4

6

8

action

7

9

action

delete

save


calendar

whereAmIgo today

+

week

week

+

previewday

day

day

preview3 hours

+

3 hours

3 hours

_

block½h

+

½ hour

½ hour

++


menu

home

find contact

add contact

recordvoice

addcontact

calendar

hear

voice note

findcontact

hearemails

2

2

1

1

3

3

abc

def

heartask list

hear textmessage

5

5

4

4

6

6

jkl

ghi

mno

speaker phone

8

mute

7

8

9

7

9

pqrs

tuv

wxyz

play

0

delete

save

delete

next

hold bottom left for

home

email, tasks, voice, SMS

calendar

hold bottom right for

help

+

+

whereAmIgo today

type

type

week

week

+

+

previewday

folder

folder

day

day

+

preview3 hours

preview

n items

n items

3 hours

3 hours

_

_

play

+

block½h

item

item

½ hour

½ hour

++


blindSight...

…is a phenomenon in which people who are perceptually blind in a certain area of their visual field demonstrate some visual awareness, without any qualitative experience

[wikipedia]


blind

sight

don’t mode me in

tactile features

10 design rulesto allow eyes-free use and flow


phones…

…are in in a mobile situation If they requires visual attention,users will fail at their current activity

interference with social activitiesdrive off the road…


interfaces

vs.

baseline


  • Login