Brain Computer Interfaces

1. Brain Computer Interfaces orKrang’s Body

2. What are BCI good for? • Brain Computer Interfaces as AST • Communication for • Locked-in persons • People with little muscle control (i.e. not enough control for EMG or gaze tracking) • People with ALS, spinal injuries • Low bandwidth (bit rate) • Medical Applications • Early diagnostics (dyslexia, sleep disorders, peripheral neuropathy, …) • Rehabilitation (restor.of movement, comb.with VR,..)

3. What are BCI good for? • AST • Medical applications • Non-medical appl. • Neurofeedback (treating ADHD, rehabilitation, psych.disorders, …, guiding meditation) • Hands-free appl. (combined with HCI) • Security and authentication • Passive BCI help monitoring user’s mental state (e.g. drivers with motion sickness) • Games (e.g. learning to control stress levels) • Learning environment (control of understanding)

4. Signals used in BCI • Invasive techniques • Intracortical • Cortical surface (ECoG) – can distinguish kinematic parameters for 5-class finger flexion and for motor imaginery pairs (left X right hand, finger X tongue movement) • Noninvasive Techniques • Magneteoencephalogrraphy (MEG) • Functional magnetic resonance imaging (fMRI) • Functional infrared spectroscopy (NIRS) • Electroencephalogram (EEG)

5. What is an EEG? • An electroencephalogram is a measure of the brain's voltage fluctuations as detected from scalp electrodes. • It is an approximation of the cumulative electrical activity of neurons.

6. EEG Background • 1875 - Richard Caton discovered electrical properties of exposed cerebral hemispheres of rabbits and monkeys. • 1924 - German Psychiatrist Hans Berger discovered alpha waves in humans and invented the term “electroencephalogram” • 1950s - Walter Grey Walter developed “EEG topography” - mapping electrical activity of the brain.

7. Physical Mechanisms • EEGs require electrodes attached to the scalp with sticky gel (dry electrodes are being tested) • Require physical connection to the machine

8. Electrode Placement • Standard “10-20 System” • Spaced apart 10-20% • Letter for region • F - Frontal Lobe • T - Temporal Lobe • C - Center • O - Occipital Lobe • Number for exact position • Odd numbers - left • Even numbers - right

9. Electrode Placement • A more detailed view:

10. Brain “Features” • User must be able to control the output: • use a feature of the continuous EEG output that the user can reliably modify (waves), or • evoke an EEG response with an external stimulus (evoked potential)

11. Continuous Brain Waves • Generally grouped by frequency: (amplitudes are about 100µV max)

12. Brain Waves Transformations • wave-form averaging over several trials • auto-adjustment with a known signal • Fourier transforms to detect relative amplitude at different frequencies

13. Alpha and Beta Waves • Studied since 1920s • Found in Parietal and Frontal Cortex • Relaxed - Alpha has high amplitude • Excited - Beta has high amplitude • So, Relaxed -> Excited means Alpha -> Beta

14. Mu Waves • Studied since 1930s • Found in Motor Cortex • Amplitude suppressed by Physical Movements, or intent to move physically • (Wolpaw, et al 1991) trained subjects to control the mu rhythm by visualizing motor tasks to move a cursor up and down (1D)

15. Mu Waves

16. Mu and Beta Waves • (Wolpaw and McFarland 2004) used a linear combination of Mu and Beta waves to control a 2D cursor. • Weights were learned from the users in real time. • Cursor moved every 50ms (20 Hz) • 92% “hit rate” in average 1.9 sec

17. Mu and Beta Waves • How do you handle more complex tasks? • Finite Automata, such as this from (Millán et al, 2004)

18. BCI signals for communication • Evoked potentials (EP) • = el.response to presentation of a stimulus (event related potentials –ERP) • ERP components, e.g. P300, N100 are characterized by amplitude (P-positive, N-neg) and latency (ms after stimuli) • Event related synchronization (ERS or ERD) • Some events (performing mental tasks as arithmetic, object rotation, motor imaginery) modulate power of the current EEG signal • This can be utilized (ERD-decrease of signal)

19. P300 (Evoked Potentials) • occurs in response to a significant but low-probability event • 300 milliseconds after the onset of the target stimulus • found in 1965 by (Sutton et al., 1965; Walter, 1965) • focus specific

20. P300 Experiments • Farwell and Donchin, 1988 • The user can select a command (BCI looks for P300 signals when the desired command flashed • 95% accuracy at 1 character per 26s

21. P300 (Evoked Potentials) • (Polikoff, et al 1995) allowed users to control a cursor by flashing control points in 4 different directions • Each sample took 4 seconds • Threw out samples masked by muscle movements (such as blinks)

22. (Polikoff, et al 1995) Results • 50% accuracy at ~1/4 Hz • 80% accuracy at ~1/30 Hz

23. VEP - Visual Evoked Potential • Detects changes in the visual cortex • Similar in use to P300 • Close to the scalp

24. Model Generalization (time) • EEG models so far haven’t adjusted to fit the changing nature of the user. • (Curran et al 2004) have proposed using Adaptive Filtering algorithms to deal with this.

25. Model Generalization (users) • Many manual adjustments still must be made for each person (such as EEG placement) • Currently, users have to adapt to the system rather than the system adapting to the users. • Current techniques learn a separate model for each user.

26. Model Generalization (users) • (Müller 2004) applied typical machine learning techniques to reduce the need for training data. • Support Vector Machines (SVM) and Regularized Linear Discriminant Analysis (RLDA) • This is only the beginning of applying machine learning to BCIs!

27. BCI Examples - Prostheses • (Wolpaw and McFarland 2004) allowed a user to move a cursor around a 2 dimensional screen • (Millán, et al. 2004) allowed a user to move a robot around the room.

28. BCI Examples - Music • 1987 - Lusted and Knapp demonstrated an EEG controlling a music synthesizer in real time. • Atau Tanaka (Stanford Center for Computer Research in Music and Acoustics) uses it in performances to switch synthesizer functions while generating sound using EMG.

29. In Review… • Brain Computer Interfaces • Allow those with poor muscle control to communicate and control physical devices • High Precision (can be used reliably) • Requires somewhat invasive sensors • Requires extensive training (poor generalization) • Low bandwidth (today 24 bits/minute, or at most 5 characters/minute)

30. Future Advances • Improving physical methods for gathering EEGs • Improving generalization • Improving knowledge of how to interpret waves (not just the “new phrenology”) • Interesting to see: • https://www.youtube.com/watch?v=I7lmJe_EXEU

31. References • http://www.cs.man.ac.uk/aig/staff/toby/research/bci/richard.seabrook.brain.computer.interface.txt • http://www.icad.org/websiteV2.0/Conferences/ICAD2004/concert_call.htm • http://faculty.washington.edu/chudler/1020.html • http://www.biocontrol.com/eeg.html • http://www.asel.udel.edu/speech/Spch_proc/eeg.html • Toward a P300-based Computer Interface James B. Polikoff, H. Timothy Bunnell, & Winslow J. Borkowski Jr.Applied Science and Engineering LaboratoriesAlfred I. Dupont Institute • Various papers from PASCAL 2004 • Original Paper on Evoked Potential: https://access.web.cmu.edu/http://www.jstor.org/cgi-bin/jstor/viewitem/00368075/ap003886/00a00500/0?searchUrl=http%3a//www.jstor.org/search/Results%3fQuery%3dEvoked-Potential%2bPotential%2bCorrelates%2bof%2bStimulus%2bUncertainty%26hp%3d25%26so%3dnull%26si%3d1%26mo%3dbs&frame=noframe&dpi=3&currentResult=00368075%2bap003886%2b00a00500%2b0%2c07&userID=80020b32@cmu.edu/01cce4403532f102af429e95&backcontext=page

32. Invasive BCIs • Have traditionally provided much finer control than non-invasive EEGs (no longer true?) • May have ethical/practical issues • (Chapin et al. 1999) trained rats to control a “robot arm” to fetch water • (Wessberg et al. 2000) allowed primates to accurately control a robot arm in 3 dimensions in real time.