300 likes | 356 Views
Feature Extraction. Spring Semester, 2010. Accelerometer Based Gestural Control of Browser Applications. M. Kauppila et al., In Proc. of Int. Workshop on Real Field Identification, UCS 2007, pp. 2-17, 2007. Outline. Motivation Previous Work H/W Architecture Recognizer Experiments
E N D
Feature Extraction Spring Semester, 2010
Accelerometer Based Gestural Control of Browser Applications M. Kauppila et al., In Proc. of Int. Workshop on Real Field Identification, UCS 2007, pp. 2-17, 2007.
Outline • Motivation • Previous Work • H/W Architecture • Recognizer • Experiments • Discussion
Motivation • Large screen interface • Train, bus stations, marketing places, and other public places • Provide different kinds of interactions between the users and the displays
Previous Work • DBN, SVM • Samsung AIT, “Two-stage Recognition of Raw Acceleration Signals for 3-D Gesture-Understanding Cell Phones,” 2006. • Classify {0~9, O, X} by using DBN • SVM is used for the confusing pair of (6, O)
Communication Architecture • Scenario • Data flow
Communication Architecture • Accelerometers • Developed at our university • Send data at 50Hz • UPnP virtual sensor • Use a simple asynchronous Java API: decouple the recognizer and its clients
Browser Control • General browser / photo album app.
Recognizer • Segmentation • Preprocessing and normalization • Classification
Recognizer Segmentation • Feature vector for segmentation • Continuous acceleration signal: • Discretized acceleration signal: • Approximated derivative: • Approximated velocity: • Two-state (non-gestural / gestural) HMM
Recognizer Segmentation Example
Recognizer Preprocessing • Sensor model • Dynamic component (gesture): ad(t) • Static component (gravity): as=(0, 0, g)T • Measured acceleration: • R: orthogonal matrix (Describing the orientation of the sensor) • Gravity estimation • Ras: Mean of the measured acceleration
Recognizer Preprocessing • Tilt Compensation • Let u = v/|v|, where v is the axis of rotation • Remember the measured acceleration • Finally,
Recognizer Normalization • Power normalization • Frobenius normalization • Tempo normalization • Rescale the gesture tempo so that all gestures have 30 samples • Downscaling: Box filtering • Upscaling: linear interpolation
Recognizer Classification • Training set • A single person, 16 samples per gesture • Recognizer • 12-state hidden Markov model per gesture • Choose the gesture class corresponding to the HMM with the highest score
Experiments • 11 Subjects • Confusion matrix
Experiments User Study • Three stages + a questionnaire (feedback) • Blind stage • Freely use the system without any prior training • Task stage • Solve a specific browsing task after training • Photo album stage
Experiments Subject Feedback • Five questions • Free-worded feedback • Stress of hands • Unintuitivity and learning overhead
Discussion • False positives still pose a problem • Rejection mechanism is needed • Recoiling problem • Avoiding the use of overly simplistic gestures
Human Activity Recognition with User-Free Accelerometers in the Sensor Networks S. Wang, et al., Int. Conf. Neural Networks and Brain, 2005. pp. 1212-1217, 2005.
Outline • Motivation • Feature Extraction • Classification • Experiments • Summary
Motivation • Human’s activities can be represented from three aspects • Movements of human bodies • Movements of the objects associated with the activities • Person-object interaction • Wearing the sensors is uncomfortable for users
Feature Extraction • Using a sliding window with 50% overlap • 19 features • Six features from each of the three axes • Acceleration, mean, standard deviation (stability), energy (data periodicity), frequency-domain entropy, correlation normalized into [-1.1] • One feature represents vibration of the sensor (|ax2+ay2+az2-g2|)
Classification • Recognition algorithms • C4.5, MLP, SVM • Three types of tests • Self-consistency test: Training set = test set • Cross-validated test • Leave-one-subject-out validation
Experiments • System setup • Accelerometer: KXP74 (32Hz, -2g~+2g) • Fixed to the rear of the telephone receiver, base of the cup, and on the top of the pen • SVM-based feature selection
Experiments Data collection • Three activities was performed by four subjects • Drinking, phoning, and writing • Positive actions + negative actions • Positive: Write on the table or on the blackboard • Negative: Rotate the pen with fingers, … • Each lasted 5 minutes
Experiments Results • Self-consistency test: Accuracies > 95% • Cross-validated test • Leave-one-subject-out validation test
Experiments Discussion on Feature Selection • Attribute sequence • Drinking • Phoning • Writing • A: Acceleration, E: Mean, S: Stdev., G: Energy, P: Entropy, C: Correlation, Delta: Vibration
Summary • Accelerometer based gestural control of browser applications • Segmentation feature • Acceleration, derivative, and velocity • Tilt compensation • Human activity recognition with user-free accelerometers in the sensor networks • Feature extraction and selection