blindSight: Eyes-free mobile phone interaction. Kevin Li , University of California, San Diego Patrick Baudisch , Microsoft Research Ken Hinckley , Microsoft Research. blindSight. “How ab out Monday morning?”. calendar. “Monday 9am”. preview. “tic, tic, sssssh”.
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
blindSight: Eyes-free mobile phone interaction Kevin Li, University of California, San Diego Patrick Baudisch, Microsoft Research Ken Hinckley, Microsoft Research
blindSight “How about Monday morning?” calendar “Monday 9am” preview “tic, tic, sssssh” “Yeah, looks likeI’m free after 10”
blindSight is an application running on Microsoft Windows Smartphone is launched when user places or receive a call. It then replaces the in-call menu unlike the in-call menu, blindSight shows nothing on the screen, but speaks to the user
PCs… PC screens have the users’ undivided attention design for the visual channel
Lots of information is stored on mobile phones…
survey “I need to access as part of a phone conversation:” # of participants
Ok, so let’s just translate all text from visual to auditory • “Menu: Press 1 to search contacts; press 2 to add a contact; press 3 to access your calendar…” • Wait, that sounds familiar
interactive voice response • User’s should be able to “dial ahead” [Perugini et al.,CHI 2007] • Zap and Zoom allows users to jump to locations using shortcuts [Hornstein, UBILAB Rep 1994] • Use visual channel to inform users about options [Yin and Zhai, CHI 2006]
Interactingmid-conversation • Time compress audio[Dietz and Yerazunis, UIST 2001] • Integrate speech commands into the conversation [Lyons et al., CHI 2004]
blindSight’s auditory feedback
rationale human-human conversation contains redundancy people can recover from audio interruptions as long as interruption is short can we use this redundancy to injectauditory feedback from the device?
so how do we make sure device feedback fits into these time windows of low information content?
rules 1. feedback only on-demand home recordvoice addcontact calendar hear voice note findcontact hearemails heartask list hear textmessage speaker phone mute
rules type 6 “200 hits” type 2 “12 hits” type 7 “Marion” 2. brevity find contact 2 1 3 abc def 5 4 6 jkl ghi mno 8 7 9 pqrs tuv wxyz play delete next
rules (what if the content is a long list,such as appointments for a day?) 3. non-speech previews of composites calendar – + whereAmIgo today week week – + previewday day day – preview3 hours 3 hours 3 hours _ block½h ½ hour ½ hour ++
rules (what if the content is a long list,such as appointments for a day?) 4. decomposition – + whereAmIgo today week week – + previewday day day – preview3 hours 3 hours 3 hours _ next ½ hour ½ hour
rules 5. interruptability user interface runs as a separate thread
rules modes 2 2 1 1 3 3 5 5 4 4 6 6 8 8 7 7 9 9 action action delete delete save save 6. minimizemodes pick day start time end time tue mon wed … fri thu sat sun action delete save
rules 6. minimizemodes (avoid wizards) – whereAmIgo today + week week – + previewday day day – preview3 hours + 3 hours 3 hours _ + block½h ½ hour ½ hour ++
demo video (shows fast use by an experienced user)
epoxy dots enlarged spaces
error Flip Ear Visual
. 2 1 3 5 4 6 8 7 9 * 0 #
error Flip Ear Visual
interfaces vs. BlindSight (eyes-free) Smartphone 2003 (sighted)