blindSight: Eyes-free mobile phone interaction - PowerPoint PPT Presentation

slide1 n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
blindSight: Eyes-free mobile phone interaction PowerPoint Presentation
Download Presentation
blindSight: Eyes-free mobile phone interaction

play fullscreen
1 / 73
blindSight: Eyes-free mobile phone interaction
118 Views
Download Presentation
minna
Download Presentation

blindSight: Eyes-free mobile phone interaction

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. blindSight: Eyes-free mobile phone interaction Kevin Li, University of California, San Diego Patrick Baudisch, Microsoft Research Ken Hinckley, Microsoft Research

  2. blindSight “How about Monday morning?” calendar “Monday 9am” preview “tic, tic, sssssh” “Yeah, looks likeI’m free after 10”

  3. blindSight is an application running on Microsoft Windows Smartphone is launched when user places or receive a call. It then replaces the in-call menu unlike the in-call menu, blindSight shows nothing on the screen, but speaks to the user

  4. why?

  5. PCs… PC screens have the users’ undivided attention  design for the visual channel

  6. eyePhone

  7. environment

  8. visual impairment

  9. screen-less device

  10. can’t seescreen

  11. Lots of information is stored on mobile phones…

  12. … the interfaces are visual

  13. survey “I need to access as part of a phone conversation:” # of participants

  14. Ok, so let’s just translate all text from visual to auditory • “Menu: Press 1 to search contacts; press 2 to add a contact; press 3 to access your calendar…” • Wait, that sounds familiar

  15. Please listen carefully as ouroptions have changed…

  16. related work

  17. interactive voice response • User’s should be able to “dial ahead” [Perugini et al.,CHI 2007] • Zap and Zoom allows users to jump to locations using shortcuts [Hornstein, UBILAB Rep 1994] • Use visual channel to inform users about options [Yin and Zhai, CHI 2006]

  18. Interactingmid-conversation • Time compress audio[Dietz and Yerazunis, UIST 2001] • Integrate speech commands into the conversation [Lyons et al., CHI 2004]

  19. blindSight’s auditory feedback

  20. audio is heard only by the user,not by the person at the other end

  21. rationale human-human conversation contains redundancy  people can recover from audio interruptions as long as interruption is short can we use this redundancy to injectauditory feedback from the device?

  22. so how do we make sure device feedback fits into these time windows of low information content?

  23. rules 1. feedback only on-demand home recordvoice addcontact calendar hear voice note findcontact hearemails heartask list hear textmessage speaker phone mute

  24. rules type 6 “200 hits” type 2 “12 hits” type 7 “Marion” 2. brevity find contact 2 1 3 abc def 5 4 6 jkl ghi mno 8 7 9 pqrs tuv wxyz play delete next

  25. rules (what if the content is a long list,such as appointments for a day?) 3. non-speech previews of composites calendar – + whereAmIgo today week week – + previewday day day – preview3 hours 3 hours 3 hours _ block½h ½ hour ½ hour ++

  26. rules (what if the content is a long list,such as appointments for a day?) 4. decomposition – + whereAmIgo today week week – + previewday day day – preview3 hours 3 hours 3 hours _ next ½ hour ½ hour

  27. rules 5. interruptability user interface runs as a separate thread

  28. rules modes 2 2 1 1 3 3 5 5 4 4 6 6 8 8 7 7 9 9 action action delete delete save save 6. minimizemodes pick day start time end time tue mon wed … fri thu sat sun action delete save

  29. rules 6. minimizemodes (avoid wizards) – whereAmIgo today + week week – + previewday day day – preview3 hours + 3 hours 3 hours _ + block½h ½ hour ½ hour ++

  30. demo video (shows fast use by an experienced user)

  31. hardware

  32. space

  33. epoxy dots enlarged spaces

  34. error Flip Ear Visual

  35. . 2 1 3 5 4 6 8 7 9 * 0 #

  36. error Flip Ear Visual

  37. blindSight evaluation

  38. interfaces vs. BlindSight (eyes-free) Smartphone 2003 (sighted)