1 / 20

The Sound of One Hand: A Wrist-mounted Bio-acoustic Fingertip Gesture Interface

The Sound of One Hand: A Wrist-mounted Bio-acoustic Fingertip Gesture Interface. Brian Amento, Will Hill AT&T Labs – Research Loren Terveen University of Minnesota. Outline. Motivation Gesture Interfaces Signal Classifiers Prototype Applications Future Work. Motivation.

xandy
Download Presentation

The Sound of One Hand: A Wrist-mounted Bio-acoustic Fingertip Gesture Interface

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Sound of One Hand:A Wrist-mounted Bio-acoustic Fingertip Gesture Interface Brian Amento, Will Hill AT&T Labs – Research Loren Terveen University of Minnesota

  2. Outline • Motivation • Gesture Interfaces • Signal Classifiers • Prototype Applications • Future Work

  3. Motivation • Small wearable digital devices increasingly popular (Cellphones, PDAs, Rios, etc..) • Nonlinear access to linear media will increase • Voicemail, Music, Video, Radio, Text • Controls: Device Select, Play, Stop, Scan forward, Scan backward, Faster, Slower, Item Select, Exit

  4. Current Interfaces to Mobile Devices • Two-handed control mechanisms • Pressing device buttons • Writing/selecting with stylus • Un-holstering a wearable is a pain (i.e., wristwatches beat pocket watches) • Speech recognition • Noise or social setting may rule out voice control • Our Goal: Invisible, weightless, un-tethered and cost-free

  5. How about a gesture interface?

  6. Body tracking Polhemus 2000 Teresa Martin 1997

  7. Datagloves

  8. Image hand tracking Cullen Jennings, 1999

  9. Our Approach “Natural” fingertip gestures

  10. What’s “natural” • Small - max displacement of 5 cm • Gentle, < 10% of pressing strength (e.g. no finger snap) • Few gestures, little memory work • Avoid ring and pinky finger • Examples: • Thumb as anvil - index, middle as hammer • Thumbpad to fingerpad • Thumbpad to fingernail edge

  11. Fingertip Gestures • Tap, double tap • Finger and thumb pads rub • Money gesture and reverse • Finger and thumb pads press • Soft Flick

  12. Fingertip Gesture Interface • Wristband-mounted piezo-electric contact microphones positioned on the styloid bones • Sense bone conducted sounds produced by gentle fingertip gestures

  13. Simple Classifier • Allows real-time analysis and control • 800 samples every 10th of a second • Take max absolute, quantize to 10 levels • Finite state machine outputs Taps and Rubs • Intermediate states filter background noise • Buffer states allow continuous gestures • Surprisingly accurate: ~90%

  14. Example Signals

  15. More Sophisticated Classifier • Noticeable differences in audio signals • Hidden Markov Models • Gesture and noise models trained with sampled data • Confidence levels for each trained gesture

  16. HMM Classifier Accuracy • Using 3 subjects, collected 100 instances of gestures rub, tap and flick • 80 used for training, 20 for testing

  17. Wrist Display Prototype • Timex Internet Messenger watch • Tap to cycle through messages • Double-tap to rewind

  18. Other Prototypes • Cellphone dialing application • Rub scrolls list in one direction • Tap dials phone number • Powerpoint slide control • Tap moves forward one slide • Double tap moves back

  19. Future Work • Miniaturization of device • Hitachi SH5 controller • Improved gesture classifiers • Finger Identification • Analyze signals from multiple microphone locations • User Studies • Usefulness: Compare performance to current cellphone, PDA and desktop control interfaces. • Social impact: Study how users exploit private control techniques to mobile devices

  20. Conclusion • Fingertip gestures • sensed acoustically at the wrist • can be communicated wirelessly to nearby devices • show promise as a control method.

More Related