Demonstrating the Feasibility of Using Forearm Electromyography for Muscle-Computer Interfaces

1. Demonstrating the Feasibility of Using Forearm Electromyography for Muscle-Computer Interfaces T. Scott Saponas University of Washington Desney S. Tan Microsoft Research Dan Morris Microsoft Research RavinBalakrishnanUniversity of Toronto

2. Physical Transducers Leverage Human Expertise

3. Need for Hands Free Input

4. Advances in Muscle Sensing Enable Muscle-Computer Interfaces …

5. Muscles Activate via Electrical Signal

6. Muscles Activate via Electrical Signal Electrical Signal can be sensed by Electromyography (EMG)

7. EMG for Diagnostics, Prosthetics & HCI Jacobsen, et al. “Utah Arm”

8. EMG for Diagnostics, Prosthetics & HCI Jacobsen, et al. “Utah Arm” Costanza, et al. “Intimate interfaces in action”

9. EMG for Diagnostics, Prosthetics & HCI Naik, et al. “Hand gestures” Jacobsen, et al. “Utah Arm” Costanza, et al. “Intimate interfaces in action”

10. EMG for Diagnostics, Prosthetics & HCI Naik, et al. “Hand gestures” Jacobsen, et al. “Utah Arm” Wheeler & Jorgensen “Neuroelectric joysticks” Costanza, et al. “Intimate interfaces in action”

11. Detecting Finger Gestures Challenging

12. Sensors Placed on Upper Forearm

13. Sensors Placed on Upper Forearm

14. Gesture Sets

15. Gesture Sets

16. Lift

17. Tap

18. Position

19. Pressure

20. Lift Tap Position Pressure

21. Gesture Classification Technique • X 8 Sensors machine learning 250 millisecond sample training data • Support Vector • Machine Features • user model test data evaluation

22. Gesture Classification Technique • X 8 Sensors machine learning 250 millisecond sample training data • Support Vector • Machine Features • Root Mean Square (RMS) • 28 ratios between channels Frequency Energy 10 Hz bins • user model • Phase Coherence • 28 ratios between channels test data evaluation

23. Randomized Block Design 1 2 3 4 random delay

24. Randomized Block Design X 50 1 2 3 4 4 2 3 1 random delay random order random order

25. 12 participants • aged 20 – 63 years (mean 46) • 8 female; 4 male • daily computer users • right-handed • 90 minutes

26. Ten-Fold Cross-Validation Lift

27. Ten-Fold Cross-Validation Tap

28. Ten-Fold Cross-Validation Position

29. Ten-Fold Cross-Validation Pressure

30. How much training data?

31. What are we really measuring? • Skin moving over muscle creates noise • Distant muscle contractions • Gestures are complex movements

32. Limitations of Current Evaluation • Works best for SINGLE user SINGLE session • Offline Analysis • Approximation of sensor armband

33. Forearm Electromyography for Muscle-Computer Interfaces Demonstrated possibility of gesture sets using pressure, position, & all five fingers Future: • Wireless & dry sensors • Dense auto-configurable band • Cross-user models • Quick compound gestures

34. Interaction Possibilities • Virtual keyboards • Hands busy controls • 3D gestural interaction • Eye-free mobile interaction

35. thanks! acknowledgements: SumitBasu, James Fogarty, Jon Froehlich, Kayur Patel, Meredith Skeels and our study participants … http://research.microsoft.com/users/dan/muci/ • T. Scott SaponasUniversity of Washington • Desney S. Tan Microsoft Research • Dan MorrisMicrosoft Research • RavinBalakrishnanUniversity of Toronto

37. Lift Tap Position Pressure

38. Labeling Training Data With Best Data stimulus rest label stimulus label

39. Single Sample Classification stimulus stimulus Whole Trial Classification thumb 1 index 5 middle 1 winner ? ? ? ? ?

40. Collect Pilot Data, Develop Classification Techniques, Evaluation define gesture sets collect pilot data develop classification techniques offline analysis collect test data experiment