1. Inertial Gesture Recognition Ari Y. Benbasat Responsive Environments Group MIT Media Laboratory

2. Compact Inertial Measurement Unit • Full sensor set for 3D motion detection in compact wireless package. • Implementation • 3(+1) Accelerometers • 3 Gyroscopes • 12-bit ADC/Microcontroller • 900 MHz wireless link • Low-power (75mW)

3. S.M. Thesis Work • Create analysis and interpretation framework for such devices: • Analysis: Activity detection • Gesture Recognition: Parameterized atomic gestures • Output Scripting: Links gestures to outputs • Applications: • Current: Re-implementation of (void*) • Future: Gesture-based control and learning Project Organization

4. Activity Detection • Simple scheme based on windowed variance • Piecewise model of model used to analytically find threshold • Finds areas of interest in data streams to be analyzed by the gesture recognition system • Err on side of false positives • Stuttering gestures OK

5. Gesture Recognition • Parameterized • Magnitude and duration are properties of the detected gestures, not fundamental to the process • Atomic • Considered on axis at a time • Considered only in units of a number of peaks • Algorithm • Expects net zero sum (accelerometers) • Non-trivial size (gyroscopes) • Pieces together stuttering gestures by combining failed gestures • Breaks gestures if polarity of adjacent peaks is identical

6. Output / Applications • Simple JPython script allows temporal and logical combinations of gestures to be linked to output routines • Value in Applications: • Allows direct, in situ sensing of quantities of interest • Compact / low-power→useable in a wide variety of situations • Low complexity of algorithms allows for stand-alone devices → combined perception and expertise in single device • Not limited to human gesture

7. Sample Analysis Perform Gestures and Collect Data Find Areas of Activity y x (no rotation) Run Recognition Recombine Atomic Output 2 Peaks = + = = 3 Peaks

8. Sample Analysis (2) • Perform gesture • Sweeping twist • Find gestures in stream • One axis at a time • Note baseline subtraction • Recombine atoms • Can be tied to output 1 Peak (gyro) = = 2 Peaks (acc) sound light etc. = causes +