Convolutional Neural Network for Multi-Category Rapid Serial Visual Presentation BCI

1. Convolutional Neural Network for Multi-Category Rapid Serial Visual Presentation BCI [Ran Manor and Amir B.Geva] • Outlines • Review • EEG/ERP (Event-Related Potential) • P300 • Spatio-temporal data • CNN Network + regularization • Results and discussion Yehu Sapir

2. ReviewCategories of electrical cortical activities in Brain Computer Interface (BCI) μ β • μ (8–12 Hz) and β (12–30 Hz) rhythms. • Response of motoroutput of the brain. • P300 Evoked Potential (EP). dementia / Alzheimer • dementia / Alzheimer • Auditory, visual, or somatosensoryEvent-Related Potentials (ERP) (1–30 μV). • 300–500 mspost-stimulus • Visual N100 and P200, ERPs with short latency • Response of the brain to a rapid visual stimulus. • Steady-State Visual Evoked Potentials (SSVEP). • Visual stimulations (3.5 - 75 Hz) generates brain responses at the same freq. • Slow Cortical Potentials (SCP). • slow potential variations 0.5–10 sec after presenting stimulus • users control these potentials and use them to control the movement of a cursor on screen • movement • reduced cortical activation.

3. ReviewP300 signal • P300 ERP at electrode Pz, • P300 = target detection for • RSVP (Rapid Serial Visual Presentation) tasks. • The amplitude and latency have a large variance Average of multiple single trials Amplitude Pz Single trial response Time [ms]

4. Subjects & Stimuli Aim classify ERPs into target and non-target images. Subjects 15 subjects participated in a RSVP experiment, 8 females, 7 males with age 25±5 years. Stimuli 360 × 360 pixels, gray-scale images. The images have the same mean luminance and contrast. 5 different categories including 145 exemplars each of faces, cars, painted eggs, watches, and planes. one category is the target

5. Experimental Procedure Brain computer interface (BCI) • Images presented in 4 blocks, • 6525 images in each block,presented every 90–110ms (~10Hz). • 20% of the images were targets, randomly distributed within each block. • different target category in each block • watches were not used as targets. • Use same blocks order across subjects. • Subjects should count targets. • Pause every 80–120 image • Reportethe Number of target. • Restart the count.

6. EEG Acquisition and Preprocessing • 64 electrodes + 7 additional electrodes (to reject trials when blinking). • 256Hz sampling rate • 51Hz online LPF to prevent aliasing of high frequencies and remove powerline noise. • 0.1Hz offline high-pass filter to remove slow drifts (DC). • Data segmentation: 1sec event-related segments (-100ms to +900ms of image presentation) Single trial =1 Image 256 Samples 64channels (electrodes) Down-sampling, Normalizing & Removing DC 64 target 64 µ=0,σ=1 +900ms -100ms µ=0,σ=1

7. Neural Network Architecture Implementation on Caffeand GPU Time Samples ReLU Dropout ReLU Dropout ReLU softmax ReLU ReLU channels # output filters

8. Spatio-Temporal Regularizer penalty term : Where: - conv. output l at time t - regularization coefficient , Where: - a matrix of zeros with 1 in position i, j Regularized No-regularization

9. Parameters updates Train the Net. By Minimize the loss function where Nsamples= training samples, x(i) = itraining sample, y(i) = true label of sample i hk= NN output unit k Minimized with SGD, Momentum update: V= µ*V + η*dW Weights updates: W=W-V Learning rate (η) = 0.001 Momentum coefficient (µ) = 0.9 network parameters were chosen empirically by cross-validation.

10. Performance correct positives + correct negatives random cross-validation procedure 80% training 20% testing (4 blocks per subject) average of 10 runs (true positive + true negative) /2 false positive Area Under the Curve true positive Correct is distorted, due to imbalanced classes Mean across subjects

11. Compare to Spatially Weighted FLD-PCA (SWFP)

12. Features Analysis • Representative spatial filters and their temporal features from a sample subject (504). • Spatial maps are similar to the spatial distribution of the P300 with a high amplitude at central-parietal electrodes. spatial filters distributed on the scalp mean temporal activations of the spatial filter

13. Sample of spatial filters and temporal activations for subjects 501 502 503 507

14. Sample of spatial filters and temporal activations for subjects 508 511 512 513

15. BCI Competition 2004 Benchmark • Compare our CNN model to BCI Competition, 2004 • P300 Speller task. • Two subjects (A, B)

16. Cross-session performance (true positive + true negative) /2 correctly classified true positive false positive

17. Thank You!