BlindSight:
Download
1 / 77

blindSight: Eyes-free mobile phone interaction - PowerPoint PPT Presentation


  • 133 Views
  • Uploaded on

blindSight: Eyes-free mobile phone interaction. Kevin Li , University of California, San Diego Patrick Baudisch , Microsoft Research Ken Hinckley , Microsoft Research. blindSight. “How ab out Monday morning?”. calendar. “Monday 9am”. preview. “tic, tic, sssssh”.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' blindSight: Eyes-free mobile phone interaction' - brittany-graves


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

blindSight:

Eyes-free

mobile phone interaction

Kevin Li, University of California, San Diego

Patrick Baudisch, Microsoft Research

Ken Hinckley, Microsoft Research


blindSight

“How about Monday morning?”

calendar

“Monday 9am”

preview

“tic, tic, sssssh”

“Yeah, looks likeI’m free after 10”


blindSight

is an application running on Microsoft Windows Smartphone

is launched when user places or receive a call. It then replaces the in-call menu

unlike the in-call menu, blindSight uses auditory feedback



PCs…

PC screens have the users’ undivided attention

 design for the visual channel









survey

“I need to access as part of a phone conversation:”

# of participants



Please listen carefully as our auditoryoptions have changed…


related work auditory


interactive voice auditory

response

  • User’s should be able to “dial ahead” [Perugini et al.,CHI 2007]

  • Zap and Zoom allows users to jump to locations using shortcuts [Hornstein, UBILAB Rep 1994]

  • Use visual channel to inform users about options [Yin and Zhai, CHI 2006]


phone interaction auditorymid-conversation

  • Time compress audio[Dietz and Yerazunis, UIST 2001]

  • Integrate speech commands into the conversation [Lyons et al., CHI 2004]


blindSight’s auditory

auditory feedback


audio is heard only by the user, auditorynot by the person at the other end


rationale auditory

human-human conversation contains redundancy

 people can recover from audio interruptions

as long as interruption is short

can we use this redundancy to injectauditory feedback from the device?



rules windows of low information content?

1. feedback only on-demand

home

recordvoice

addcontact

calendar

hear

voice note

findcontact

hearemails

heartask list

hear textmessage

speaker phone

mute


rules windows of low information content?

type 6

“200 hits”

type 2

“12 hits”

type 7

“Marion”

2. brevity

find contact

2

1

3

abc

def

5

4

6

jkl

ghi

mno

8

7

9

pqrs

tuv

wxyz

play

delete

next


rules windows of low information content?

(what if the content is a long list,such as appointments for a day?)

3. non-speech previews of composites

calendar

+

whereAmIgo today

week

week

+

previewday

day

day

preview3 hours

3 hours

3 hours

_

block½h

½ hour

½ hour

++


rules windows of low information content?

(what if the content is a long list,such as appointments for a day?)

4. decomposition

+

whereAmIgo today

week

week

+

previewday

day

day

preview3 hours

3 hours

3 hours

_

next

½ hour

½ hour


rules windows of low information content?

5. interruptability

user interface runs as a separate thread


rules windows of low information content?

modes

2

2

1

1

3

3

5

5

4

4

6

6

8

8

7

7

9

9

action

action

delete

delete

save

save

6. minimizemodes

pick day

start time

end time

tue

mon

wed

fri

thu

sat

sun

action

delete

save


rules windows of low information content?

6. minimizemodes (avoid wizards)

whereAmIgo today

+

week

week

+

previewday

day

day

preview3 hours

+

3 hours

3 hours

_

+

block½h

½ hour

½ hour

++


home windows of low information content?

recordvoice

addcontact

calendar

hear

voice note

findcontact

hearemails

heartask list

hear textmessage

speaker phone

mute


add contact windows of low information content?

2

1

3

5

4

6

8

7

9

0

delete

save


patterns windows of low information content?

iterator

menu

2

1

3

5

4

6

8

action

7

9

action

delete

save


calendar windows of low information content?

whereAmIgo today

+

week

week

+

previewday

day

day

preview3 hours

+

3 hours

3 hours

_

block½h

+

½ hour

½ hour

++


demo video windows of low information content?

(shows fast usage by an experienced user)


hardware windows of low information content?


space windows of low information content?


epoxy dots windows of low information content?

enlarged spaces


error windows of low information content?

Flip

Ear

Visual


. windows of low information content?

2

1

3

5

4

6

8

7

9

*

0

#


error windows of low information content?

Flip

Ear

Visual


blindSight windows of low information content?

evaluation


interfaces windows of low information content?

vs.

BlindSight (eyes-free)

Smartphone 2003 (sighted)


task windows of low information content?

(1) schedule appointments and (2) add contacts

idle

while “driving”


results windows of low information content?

Prefer Overall

Prefer if driving and talking

Better for setting meeting times

Felt in control of the conversation

Knew what day/time I was at

Knew position in the menu

Was not missing information

blindSight

Smartphone

0

1

2

3

4

5

6

7

8

Overall preference


lessons windows of low information content?

  • 1. brevity is good, but use in moderationclarification of navigation overrides brevity

  • 2. predictable/modeless user interface is key

  • 3. auditory feedback goes a long way even during phone call(disclaimer: need to study how it interferes with activities… driving)


next: windows of low information content?

can’t seescreen

environment

screen-lessdevice

visual impairment


eyePhone windows of low information content?


? windows of low information content?

eyePhone

eyesFreePhone


blindSight: windows of low information content?

Eyes-free

mobile phone interaction

Kevin Li, University of California, San Diego

Patrick Baudisch, Microsoft Research

Ken Hinckley, Microsoft Research


extra slides windows of low information content?


contributions windows of low information content?

  • built a system

  • a set of eyes-free design rules

  • keypad modifications enabling eyes-free

  • user study comparing with a product (Smartphone 2003)


rules windows of low information content?

1. feedback only on-demand

2. brevity

3. non-speech previews of composites

4. decomposition

5. interruptability

6. minimize modes


patterns windows of low information content?

iterator

menu

2

1

3

5

4

6

8

action

7

9

action

delete

save


home windows of low information content?

recordvoice

addcontact

calendar

hear

voice note

findcontact

hearemails

heartask list

hear textmessage

speaker phone

mute


add contact windows of low information content?

2

1

3

5

4

6

8

7

9

0

delete

save


patterns windows of low information content?

iterator

menu

2

1

3

5

4

6

8

action

7

9

action

delete

save


calendar windows of low information content?

whereAmIgo today

+

week

week

+

previewday

day

day

preview3 hours

+

3 hours

3 hours

_

block½h

+

½ hour

½ hour

++


menu windows of low information content?

home

find contact

add contact

recordvoice

addcontact

calendar

hear

voice note

findcontact

hearemails

2

2

1

1

3

3

abc

def

heartask list

hear textmessage

5

5

4

4

6

6

jkl

ghi

mno

speaker phone

8

mute

7

8

9

7

9

pqrs

tuv

wxyz

play

0

delete

save

delete

next

hold bottom left for

home

email, tasks, voice, SMS

calendar

hold bottom right for

help

+

+

whereAmIgo today

type

type

week

week

+

+

previewday

folder

folder

day

day

+

preview3 hours

preview

n items

n items

3 hours

3 hours

_

_

play

+

block½h

item

item

½ hour

½ hour

++


blindSight... windows of low information content?

…is a phenomenon in which people who are perceptually blind in a certain area of their visual field demonstrate some visual awareness, without any qualitative experience

[wikipedia]


blind windows of low information content?

sight

don’t mode me in

tactile features

10 design rulesto allow eyes-free use and flow


phones… windows of low information content?

…are in in a mobile situation If they requires visual attention,users will fail at their current activity

interference with social activitiesdrive off the road…


interfaces windows of low information content?

vs.

baseline


ad