BlindSight:
This presentation is the property of its rightful owner.
Sponsored Links
1 / 77

blindSight: Eyes-free mobile phone interaction PowerPoint PPT Presentation


  • 93 Views
  • Uploaded on
  • Presentation posted in: General

blindSight: Eyes-free mobile phone interaction. Kevin Li , University of California, San Diego Patrick Baudisch , Microsoft Research Ken Hinckley , Microsoft Research. blindSight. “How ab out Monday morning?”. calendar. “Monday 9am”. preview. “tic, tic, sssssh”.

Download Presentation

blindSight: Eyes-free mobile phone interaction

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Blindsight eyes free mobile phone interaction

blindSight:

Eyes-free

mobile phone interaction

Kevin Li, University of California, San Diego

Patrick Baudisch, Microsoft Research

Ken Hinckley, Microsoft Research


Blindsight eyes free mobile phone interaction

blindSight

“How about Monday morning?”

calendar

“Monday 9am”

preview

“tic, tic, sssssh”

“Yeah, looks likeI’m free after 10”


Blindsight eyes free mobile phone interaction

blindSight

is an application running on Microsoft Windows Smartphone

is launched when user places or receive a call. It then replaces the in-call menu

unlike the in-call menu, blindSight uses auditory feedback


Blindsight eyes free mobile phone interaction

why?


Blindsight eyes free mobile phone interaction

PCs…

PC screens have the users’ undivided attention

 design for the visual channel


Blindsight eyes free mobile phone interaction

eyePhone


Blindsight eyes free mobile phone interaction

environment


Blindsight eyes free mobile phone interaction

visual impairment


Blindsight eyes free mobile phone interaction

screen-less device


Blindsight eyes free mobile phone interaction

can’t seescreen


Blindsight eyes free mobile phone interaction

Lots of information is stored on

mobile phones…


Blindsight eyes free mobile phone interaction

… the interfaces are visual


Blindsight eyes free mobile phone interaction

survey

“I need to access as part of a phone conversation:”

# of participants


Blindsight eyes free mobile phone interaction

  • Ok, so let’s just translate all text from visual to auditory

    • “Menu: Press 1 to search contacts; press 2 to add a contact; press 3 to access your calendar…”

  • Wait, that sounds familiar


Blindsight eyes free mobile phone interaction

Please listen carefully as ouroptions have changed…


Blindsight eyes free mobile phone interaction

related work


Blindsight eyes free mobile phone interaction

interactive voice

response

  • User’s should be able to “dial ahead” [Perugini et al.,CHI 2007]

  • Zap and Zoom allows users to jump to locations using shortcuts [Hornstein, UBILAB Rep 1994]

  • Use visual channel to inform users about options [Yin and Zhai, CHI 2006]


Blindsight eyes free mobile phone interaction

phone interactionmid-conversation

  • Time compress audio[Dietz and Yerazunis, UIST 2001]

  • Integrate speech commands into the conversation [Lyons et al., CHI 2004]


Blindsight eyes free mobile phone interaction

blindSight’s

auditory feedback


Blindsight eyes free mobile phone interaction

audio is heard only by the user,not by the person at the other end


Blindsight eyes free mobile phone interaction

rationale

human-human conversation contains redundancy

 people can recover from audio interruptions

as long as interruption is short

can we use this redundancy to injectauditory feedback from the device?


Blindsight eyes free mobile phone interaction

how do we make sure device feedback fits into these time windows of low information content?


Blindsight eyes free mobile phone interaction

rules

1. feedback only on-demand

home

recordvoice

addcontact

calendar

hear

voice note

findcontact

hearemails

heartask list

hear textmessage

speaker phone

mute


Blindsight eyes free mobile phone interaction

rules

type 6

“200 hits”

type 2

“12 hits”

type 7

“Marion”

2. brevity

find contact

2

1

3

abc

def

5

4

6

jkl

ghi

mno

8

7

9

pqrs

tuv

wxyz

play

delete

next


Blindsight eyes free mobile phone interaction

rules

(what if the content is a long list,such as appointments for a day?)

3. non-speech previews of composites

calendar

+

whereAmIgo today

week

week

+

previewday

day

day

preview3 hours

3 hours

3 hours

_

block½h

½ hour

½ hour

++


Blindsight eyes free mobile phone interaction

rules

(what if the content is a long list,such as appointments for a day?)

4. decomposition

+

whereAmIgo today

week

week

+

previewday

day

day

preview3 hours

3 hours

3 hours

_

next

½ hour

½ hour


Blindsight eyes free mobile phone interaction

rules

5. interruptability

user interface runs as a separate thread


Blindsight eyes free mobile phone interaction

rules

modes

2

2

1

1

3

3

5

5

4

4

6

6

8

8

7

7

9

9

action

action

delete

delete

save

save

6. minimizemodes

pick day

start time

end time

tue

mon

wed

fri

thu

sat

sun

action

delete

save


Blindsight eyes free mobile phone interaction

rules

6. minimizemodes (avoid wizards)

whereAmIgo today

+

week

week

+

previewday

day

day

preview3 hours

+

3 hours

3 hours

_

+

block½h

½ hour

½ hour

++


Blindsight eyes free mobile phone interaction

home

recordvoice

addcontact

calendar

hear

voice note

findcontact

hearemails

heartask list

hear textmessage

speaker phone

mute


Blindsight eyes free mobile phone interaction

add contact

2

1

3

5

4

6

8

7

9

0

delete

save


Blindsight eyes free mobile phone interaction

patterns

iterator

menu

2

1

3

5

4

6

8

action

7

9

action

delete

save


Blindsight eyes free mobile phone interaction

calendar

whereAmIgo today

+

week

week

+

previewday

day

day

preview3 hours

+

3 hours

3 hours

_

block½h

+

½ hour

½ hour

++


Blindsight eyes free mobile phone interaction

demo video

(shows fast usage by an experienced user)


Blindsight eyes free mobile phone interaction

hardware


Blindsight eyes free mobile phone interaction

space


Blindsight eyes free mobile phone interaction

epoxy dots

enlarged spaces


Blindsight eyes free mobile phone interaction

error

Flip

Ear

Visual


Blindsight eyes free mobile phone interaction

.

2

1

3

5

4

6

8

7

9

*

0

#


Blindsight eyes free mobile phone interaction

error

Flip

Ear

Visual


Blindsight eyes free mobile phone interaction

blindSight

evaluation


Blindsight eyes free mobile phone interaction

interfaces

vs.

BlindSight (eyes-free)

Smartphone 2003 (sighted)


Blindsight eyes free mobile phone interaction

task

(1) schedule appointments and (2) add contacts

idle

while “driving”


Blindsight eyes free mobile phone interaction

results

Prefer Overall

Prefer if driving and talking

Better for setting meeting times

Felt in control of the conversation

Knew what day/time I was at

Knew position in the menu

Was not missing information

blindSight

Smartphone

0

1

2

3

4

5

6

7

8

Overall preference


Blindsight eyes free mobile phone interaction

lessons

  • 1. brevity is good, but use in moderationclarification of navigation overrides brevity

  • 2. predictable/modeless user interface is key

  • 3. auditory feedback goes a long way even during phone call(disclaimer: need to study how it interferes with activities… driving)


Blindsight eyes free mobile phone interaction

next:

can’t seescreen

environment

screen-lessdevice

visual impairment


Blindsight eyes free mobile phone interaction

eyePhone


Blindsight eyes free mobile phone interaction

?

eyePhone

eyesFreePhone


Blindsight eyes free mobile phone interaction

blindSight:

Eyes-free

mobile phone interaction

Kevin Li, University of California, San Diego

Patrick Baudisch, Microsoft Research

Ken Hinckley, Microsoft Research


Blindsight eyes free mobile phone interaction

extra slides


Blindsight eyes free mobile phone interaction

contributions

  • built a system

  • a set of eyes-free design rules

  • keypad modifications enabling eyes-free

  • user study comparing with a product (Smartphone 2003)


Blindsight eyes free mobile phone interaction

rules

1. feedback only on-demand

2. brevity

3. non-speech previews of composites

4. decomposition

5. interruptability

6. minimize modes


Blindsight eyes free mobile phone interaction

patterns

iterator

menu

2

1

3

5

4

6

8

action

7

9

action

delete

save


Blindsight eyes free mobile phone interaction

home

recordvoice

addcontact

calendar

hear

voice note

findcontact

hearemails

heartask list

hear textmessage

speaker phone

mute


Blindsight eyes free mobile phone interaction

add contact

2

1

3

5

4

6

8

7

9

0

delete

save


Blindsight eyes free mobile phone interaction

patterns

iterator

menu

2

1

3

5

4

6

8

action

7

9

action

delete

save


Blindsight eyes free mobile phone interaction

calendar

whereAmIgo today

+

week

week

+

previewday

day

day

preview3 hours

+

3 hours

3 hours

_

block½h

+

½ hour

½ hour

++


Blindsight eyes free mobile phone interaction

menu

home

find contact

add contact

recordvoice

addcontact

calendar

hear

voice note

findcontact

hearemails

2

2

1

1

3

3

abc

def

heartask list

hear textmessage

5

5

4

4

6

6

jkl

ghi

mno

speaker phone

8

mute

7

8

9

7

9

pqrs

tuv

wxyz

play

0

delete

save

delete

next

hold bottom left for

home

email, tasks, voice, SMS

calendar

hold bottom right for

help

+

+

whereAmIgo today

type

type

week

week

+

+

previewday

folder

folder

day

day

+

preview3 hours

preview

n items

n items

3 hours

3 hours

_

_

play

+

block½h

item

item

½ hour

½ hour

++


Blindsight eyes free mobile phone interaction

blindSight...

…is a phenomenon in which people who are perceptually blind in a certain area of their visual field demonstrate some visual awareness, without any qualitative experience

[wikipedia]


Blindsight eyes free mobile phone interaction

blind

sight

don’t mode me in

tactile features

10 design rulesto allow eyes-free use and flow


Blindsight eyes free mobile phone interaction

phones…

…are in in a mobile situation If they requires visual attention,users will fail at their current activity

interference with social activitiesdrive off the road…


Blindsight eyes free mobile phone interaction

interfaces

vs.

baseline


  • Login