Mobile HCI

1. Mobile HCI Presented by Bradley Barnes

2. Mobile vs. Stationary • Desktop – Stationary • Users can devote all of their attention to the application. • Very graphical, detailed • Use the keyboard and mouse for input

3. Mobile and Wearable Devices • Users in motion – can’t devote all of attention to the application • Limited screen real estate • Input and output capabilities are restricted for users on the move

4. Mobile Device Interaction • The interface for mobile and wearable devices continues to mimic those of desktop computers. • New interaction techniques are needed to safely accommodate users on the move. • Interaction should be subtle, discreet, and unobtrusive.

5. Mobile Interaction Methods • Keyboard • Touch Screen • Speech Recognition • Head motion, 3-D sound • Eyeglass displays with Gestural Interaction

6. CHI2005 Paper • Toward Subtle Intimate Interfaces for Mobile Devices Using an EMG Controller • Using a mobile device in a social context should not cause embarrassment and disruption to the immediate environment.

7. Intimate Interfaces • Discrete interfaces that allow control of mobile devices through subtle gestures in order to gain social acceptance. • Must take into account the social context where the interaction will occur. • Most interaction occurs around other people (bus, train, street, etc.)

8. Electromyographic Signals • EMG signals are generated by muscle contraction. • Signals are picked up by contact electrodes. • Allows a definition of “subtle” or “motionless gestures” that can be used to issue commands to mobile devices. • Can sense muscular activity not related to movement.

9. The System • Armband controller recognizes gestures • Signals transmitted via Bluetooth • Compliant device receives signals and performs appropriate action. • Can be PDA, mobile phone, etc. • The armband works on all users: no calibration or training required.

10. When combined with eyeglass displays, the system becomes “hands free”. • Can be operated when users are carrying items. • Can be used in specific fields, such as maintenance, for assistance when the user’s hands are tied.

11. Design Process • Iterative process centered on users. • Three pilot studies and one formal Study. • Pilot Study 1: Bicep is chosen muscle, and the gesture is defined as a brief unnoticeable contraction of the bicep.

12. Pilot Study 2: Refine the gesture definition, and create an algorithm for its detection. • New subjects w/ variety of muscle volumes • Gesture not fully described to subjects • Compared EMG signals of gesture to those of normal activity. • Algorithm detects peaks in the EMG signals.

13. Pilot Study 3: Fine tuning of the system, wanted to test for false positives and false negatives • Consisted of new and returning users • Worked with the algorithm until the number of false positives approached zero • Also, they decided to try a gesture alphabet with two gestures. They are defined as two short contractions of different duration.

14. Formal Study: Validation of Results • Pilot studies set up the system parameters by testing gestures on subjects who were not mobile. • Conducted to assess the usability of EMG as a subtle interaction technique for mobile devices. • Evaluated the system usability in a mobile context.

15. Formal Experiment Design • 10 adult participants-Ages 23 to 34 • Perform 5 walking tasks – one with no contraction to calculate misclassification rate, and other four with contractions of different durations. • Subjects make laps around obstacles while doing the contractions

16. Familiarization sessions preceded the walking tasks. • These involved standing and making contractions. The participants were prompted to contract by a MIDI piano tone delivered through the wireless headphones. • System recognized contractions between 0.3 and 0.8 seconds, but the subjects did not know the duration of their contraction: only that the system recognized it.

17. Tasks • 1) Walking, No Contractions – 10 laps • 2) Standing, Familiarization, Generic Contractions • 3) Walking, Stimulus-Response, Generic Contractions • 4) Standing, Familiarization, Short Contractions

18. 5) Walking, Stimulus-Response, Short Contractions • 6) Standing, Familiarization, Long Contractions • 7) Walking, Stimulus-Response, Long Contractions • 8) Walking, Stimulus-Response, Mixed Long and Short Contractions (low tone – long contractions, high tone – short)

19. Results • The online recognition rates for the four walking tasks were: • Generic: 96% • Short: 97% • Long: 94% • Mixed: 87%

20. Conclusion • An EMG based wearable input device can be used for subtle and intimate interaction. • The system presented can recognize motionless gesture without training or calibration. • EMG gestures can be utilized as a socially acceptable alternative for mobile device interaction

21. Future Work • Expand gesture alphabet • Test in more “real world” scenarios, like when lifting something.

22. References • Lumsden, J., Brewster, S. A paradign shift: alternative interaction techniques for use with mobile & wearable devices. Proc. Of the 13th Annual IBM Centers for Advanced Studies Conference CASCON’2003. • Costanza, E., Inverso, S., Allen, R. Toward Subtle Intimate Interfaces for Mobile Devices Using an EMG Controller.