Closing the Sensorimotor Loop: Haptic Feedback Facilitates Decoding of Arm Movement Imagery

1. 1MPI for Biological Cybernetics 2Stanford University3Brain-Computer Interface Laboratory, Wadsworth Center 3Werner Reichardt Centre for Integrative NeuroscienceEberhard Karls University Tuebingen Closing the Sensorimotor Loop: Haptic Feedback Facilitates Decoding of Arm Movement Imagery M. Gomez-Rodriguez1,2 J. Peters1J. Hill3 B. Schölkopf1A. Gharabaghi4 M. Grosse-Wentrup1 SMC Workshop in Shared-Control for BMI, October 2010

2. BCI + robot-assisted therapy Brain Computer Interface (BCI) + robot-assisted physical therapy for neurorehabilitation of: Hemiparetic syndromes due to brain damage may outperform traditional therapy Our approach to rehabilitation Traditional rehabilitation Sensorimotor loop is broken They do not help for severe motor impairment We close the sensorimotor loop Synchronize subject’s attempt and robot arm

3. Stand-alone BCI and robot-assisted therapy It has been shown that Motor imagery Robot-assisted physical therapy are beneficial for rehabilitation as stand-alone therapies[2,3]but loop is still broken! Next logical step is to combine both in an integrated rehabilitation to close the loop

4. Hebbian plasticity: Why closing the loop? Closing artificially the sensorimotor loop is likely to result in increased cortical plasticity because we induce Hebbian plasticity. Hebbian plasticity [1] “A positive feedback-mediated plasticity in which synapses between presynaptic and postsynaptic neurons that are coincidently active are strengthened.” Requirements Instantaneous feedback:Delays in the order of ms. High accuracy: On-line decoding of arm movement intention. High specificity: Focus on motor and sensorimotor cortex.

5. BCI decoding: Effect of closing the loop Our work builds on analyzing theeffect of artificially closing the sensorimotor loop on BCI-decoding. Combining BCI and robot-assisted physical therapy opens many research questions. Previous studies: Passive and active movements induce patterns in the brain similar to those induced by motor imagery [2, 3]. Random haptic feedback has been shown to be beneficial for BCI-decoding [4]. In our work: We show how haptic feedback (closing the sensorimotor loop) influencesBCI-decoding.

6. Outline • Experimental Design: Human subjects, recording and task & feedback conditions. • Methods: Signal processing, on-line decoding and conditions comparison. • Results: Analysis of the haptic feedback effect on decoding performance and spatial/frequency features. • Conclusions

7. Subjects and recordings Human subjects: 6 right-handed healthy subjects between 22 and 32 years old. Recording: 35 EEG channels250 Hz sampling rateQuickamp with built-in CARBCI2000 + BCPy2000 Pre-motor, primary motor and somatosensory cortex are covered

8. Task and haptic feedback conditions Subject’s task:Think about moving the right arm forward (extension) or backward (flexion) in the same way the robot does. Condition Training Test + + Condition I + Condition II + Condition III Condition IV

9. Haptic feedback conditions + Rest: 3sMI: 5s Rest: 3sMI: min(5s, robot hits border) Trial duration X Robot moves while motor imagery Training(25s per condition) Arrow in a screen + robot moves-stops according to classifier while motor imagery Arrow in a screen moves-stops according to classifier while motor imagery Test(consecutively after training)

10. Outline • Experimental Design: Human subjects, recording and task & feedback conditions. • Methods: Signal processing, on-line decoding and conditions comparison. • Results: Analysis of the haptic feedback effect on decoding performance and spatial/frequency features. • Conclusions

11. Signal Processing Preprocessing Surface Laplacian Filter Band-pass filtering (2-115Hz) Notch filtering (50 Hz) Features Computation • Power spectral densities over 2Hz frequency bins for each electrode are used as features. • Welch’s method over overlapping incrementally bigger time segments each 5-s movement or 3-s resting periods. Larger segments →Less noise and more reliable estimates. Shorter segments → Necessary for on-line feedback.

12. On-line Decoding • During the test periods, on-line classification between movement and resting using spectral features: • Every 300 ms, • One classifier output. • Visual on-line feedback and depending on the condition also haptic feedback is updated. + A linear support vector machine (SVM) is generated each run on-line after the training period and its outputs are mapped to probabilistic outputs by fitting a sigmoid.

13. Conditions comparison To discover how haptic feedback influences the BCI we compare the BCI performance for each condition of haptic feedback by computing: • Two-way analysis of variance (ANOVA) over probabilistic outputs in each condition. • Average accuracy per condition. • Area under the receiving operating characteristic (AUC) per condition We expect all three to support the same conclusions to strengthen the empirical evidence.

14. ANOVA and AUC ANOVA AUC • For each condition, we group M probabilistic outputs from all subjects. • Compute ANOVA at significant levelα = 0.05 with Bonferroni multiple-comparison correction. • ANOVA tell us if we can reject the hypothesis that the probabilistic outputs means are equal between conditions. • For each subject and condition, we have N probabilistic outputs. • We sweep over different thresholds in (0, 1) to classify mov/rest and compute the accuracy for each. • The area under the curve threshold versus accuracy is our AUC.

15. Outline • Experimental Design: Human subjects, recording and task & feedback conditions. • Methods: Signal processing, on-line decoding and conditions comparison. • Results: Analysis of the haptic feedback effect on decoding performance and spatial/frequency features. • Conclusions

16. ANOVA ANOVA confidence intervals Training Test + + + Conditions + Average probabilistic on-line (every 300ms) output Condition Ioutperforms the rest, very clearly condition IV!

17. Average accuracy • In group average, Condition Ioutperforms the rest. • Condition I outperforms Conditions III and IV for all subjects, and it outperforms II for all subjects except two. Average accuracy + Test + The results are coherent with ANOVA! Training + + Conditions

18. AUC • In group average, Condition Ioutperforms the rest. • Condition I outperforms the rest for all subjects. AUC + Test + The results are coherent with ANOVA and average accuracy! Training + + Conditions

19. Spatial and spectral features Average classifiers weights for each electrode over the frequency band (2 – 40 Hz) Condition I and II(Robot moves during training) Condition III and IV(Robot does not move during training) When the robot moves, we have higher weights in the motor/somatosensory area

20. Outline • Experimental Design: Human subjects, recording and task & feedback conditions. • Methods: Signal processing, on-line decoding and conditions comparison. • Results: Analysis of the haptic feedback effect on decoding performance and spatial/frequency features. • Conclusions

21. Conclusions Artificially closing the sensorimotor feedback loop facilitates decoding of movement intention in healthy subjects. Our results indicate the feasibility of future integrated rehabilitation therapy that combines robot-assisted physical therapy with decoding of movement intention by a BCI. We assume that the results presented here with healthy subjects can be transferred to stroke patients. We speculate that haptic feedback support subjects in initiating a voluntary modulation of their SMR. In a shared-control scenario in BMIs, we may improve performance by means of haptic feedback.