ECE5320 Mechatronics Assignment#01: Literature Survey on Sensors and Actuators Topic: Neural sensors

1. ECE5320 MechatronicsAssignment#01: Literature Survey on Sensors and Actuators Topic: Neural sensors Prepared by: Philip Jones Dept. of Electrical and Computer Engineering Utah State University T: (435)797-4658; F: (435)797-3054 (ECE Dept.) 3/11/2005

2. Outline • Introduction • Reference list • To probe further • Major applications • Measurement Classifications • Interpretation Types • Sensors: Retailers and Cost ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

3. Introduction Neural sensors are sensors designed to pick up the bio-electric signals that the brain and nervous system of all living creatures produce. There are a number of different ways to detect these signals and a number of techniques for interpreting what is detected. Most of the applications are still experimental but the potential for these sensors is limitless. ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

4. Reference list Prof Kevin Warwick - University of Reading Jessica Bayliss, Ph.D. - Rochester Institute of Technology Jonathan Wolpaw - Wadsworth Center Toby Howard – University of Manchester Hui-Fang Do – Utah State University ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

5. To explore further General http://www.cs.man.ac.uk/aig/staff/toby/research/bci/ http://www.dataq.com/index.html http://www.pemed.com/neuro/neuro.htm http://www.cyber.rdg.ac.uk/ Cognitive Engineering http://www.wheelweb.com/brainimplant.htm http://maven.smith.edu/~orourke/Robotics/jaba/ http://www.cis.upenn.edu/~bracy/brain/ Silent Speech Recognition http://www.nasa.gov/centers/ames/news/releases/2004/04_18AR.html http://www.nasa.gov/centers/ames/news/releases/2004/subvocal/subvocal.html http://www.newscientist.com/article.ns?id=dn4795 http://www.trnmag.com/Stories/2004/032404/System_susses_out_silent_speech_032404.html Galvanic Skin Response http://www.trans4mind.com/psychotechnics/gsr.html http://www.qubitsystems.com/qubit/home/products/sensors/electrophysiology_sensors/galvanic_skin_response.php# http://www.plazaearth.com/usr/gasperi/gsr.htm http://www.bio-medical.com/news_display.cfm?mode=GSR&newsid=6 Electromyelogram http://www.teleemg.com/ http://www.noraxon.com/index.php3 http://www.mndcentre.org.uk/Care%20Services/tests.html Electroencephalogram http://www.chw.org/display/PPF/DocID/2479/router.asp http://www.ibva.com/html/core.htm http://nwnfa.tripod.com/hometraining/id1.html Electrocardiogram http://www.ecglibrary.com/ecghome.html http://www.hgcardio.com/HRhythm/NonInvasive/noninvasive.html http://www.cincinnatichildrens.org/health/heart-encyclopedia/diagnostic/ekg.htm Interesting Miscellaneous http://www.wired.com/wired/archive/8.02/warwick_pr.html EEG Interpretation Techniques http://www.cs.rit.edu/~jdb/research/ http://www.wadsworth.org/resnres/wolpaw.htm http://www.vh.org/adult/patient/neurology/visualevokedpotentialtest/ http://www.virginiamason.org/dbNeurophysiology/sec1824.htm http://www.msu.edu/unit/neuro/subspecialties/vep.html http://www.ophth.kpu-m.ac.jp/e/current_research/crt10.html ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

6. Major applications • Current Applications Medical Diagnosis for neural, heart or psychological damage Biofeedback therapy Lie Detection Recreation • Experimental Applications Silent communication Direct brain to computer control Prosthesis control for paralysis victims ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

7. Obstacle 1: Signal Detection There are two major obstacles to overcome in making useful measurements of the neural system. The first obstacle is detecting the signals. These signals are usually only a few millivolts in amplitude and must be measured through the non-homogenous medium of the human body. A number of different techniques have been invented to make these measurements. ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

8. Typical Setup Connector Signal Input/ Signal Conditioning Software Processing/Interface Human Interface ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

9. Measurement Classifications • galvanic skin response (GSR)-This test measures the resistance of the skin to a small current passed though it. The resistance of the skin will change depending on the mood and emotional reaction of the subject • electroencephalogram (EEG) – The measurement of electrical activity in the brain usually through electrodes or a type of skullcap • electromyelogram (EMG) – The measurement of electrical activity in muscles. Usually measured through needles inserted into the muscles. • electrocardiogram (ECG/EKG) – The measurement of electrical activity in the heart • silent speech detection – A specialized type of EMG. Small sensors are placed on the throat to detect the neural impulses made by speaking or thinking about speaking. • cognitive engineering – An experimental advance on EEG, this techniques uses electrodes implanted directly in the brain to read brain electrical activity. ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

10. GSR: Principles and Applications A sensor pad is placed on the subject, usually the finger and the conductivity of the skin is measured. The resistance of the skin will change according to the mental state of the subject. Shock, stress, surprise, and general mood will affect the resistance of the skin. GSR has been used in tests on anxiety and stress, in biofeedback therapy, and sometimes as a lie detector. Some research is being done in using GSR sensors to help with athlete mental attitude training. Interpretation of GSR data is not a firm science and there is a lot of debate over the interpretation and usefulness of the data. At this point in time this is probably the least useful of the neural sensors. ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

11. GSR: Sensors A simple GSR sensor can be made with Velcro, a DC battery, and a few electrical components. More advanced systems with greater accuracy and graphing capabilities can be purchased but are quite a bit more expensive Retailer: Qubit Systems Price: $570 ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

12. ECG/EKG: Principles and Applications The electrocardiogram is one of the most common and best understood of the different classification of neural sensors. Electrocardiograms are used to investigate heart disease. Very specific patterns have been determined for specific diseases. For specific details visit http://www.ecglibrary.com/ecghome.html ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

13. ECG/EKG: Retailer: DRE Price: Contact Retailer Retailer: MSEC Medical Supplies & Equipment Company Price: $2950-$3200(used) ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

14. EEG: Principles and Applications The most challenging and fascinating type of neural sensor is the electroencephalogram. This sensor is used to detect electrical activity in the brain or “brain waves.” Electrodes, bands, or a skullcap are placed on the head and used to pick up the signals. Currently most practical uses of EEG are medical. The EEG can be used to detect the presence of brain damage or neurological disorders. EEG can also be used in biofeedback therapy. Currently a lot of research is being done on using EEG to interface with computers and control objects or play video games. The possibilities for such applications is endless. ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

15. Basic Electrode Placement Sites ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

16. EMG: Principles and Applications An electromyelogram uses pads or needles to sense neural signals in the muscles. EMG is used extensively to test for nerve damage either from injury or degenerative disease. ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

17. EEG/EMG: Sensors Retailer: Dataq Instruments Price: $25 - $300 Items: Data Acquisition Hardware Software Retailer: Noraxon Price: Contact Retailer Items: Data Acquisition Hardware Software Retailer: TeleEMG.com Price: $3-$15 Items: Electrodes Retailer: PMED Price: $1,000 - $10,000 Items: Data Acquisition Hardware Retailer: IBVA Technologies, Inc. Price: (1 channel) $1,295 (2 channel) $2,295 Items: Interface(headband) Data Acquisition Hardware Software ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

18. Obstacle 2: Signal Interpretation • A larger problem than signal detection is extracting useful information from the signal detected. The human nervous system is a mass of signals of varying strength. Pinning down a specific signal for a specific thought or action can be extremely difficult. A number of techniques have been developed for interpreting the information. ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

19. EEG Interpretation It is one thing to be able to measure and record brain electrical activity but the real difficulty is making any sense out of the data. Several techniques are used to interpret and use the collected data Some of the most popular are: • P300 evoked potential • Mu Wave Manipulation/Biofeedback • VEP – Visual Evoked Potential • EEG Pattern Mapping ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

20. Mu Rhythm Conditioning Mu rhythm is a 8-12 Hz brain rhythm centered on the sensorimotor cortex. With biofeedback training the 80% of subjects are able to control the amplitude of the mu wave. The wave amplitude is used for external control, either moving a cursor or selecting different buttons. Jonathan Wolpaw at the Wadsworth Center is doing research in using Mu conditioning to control a computer cursor. He is currently able to create one dimensional control and is working on creating two dimensional control. The Cognitive Engineering research being done by Bakay and Kennedy is basically using the same approach. The only difference is that they are tapping directly into the brain. ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

21. P300 evoked potential This is a brain signal that occurs about 300 ms after the subject has seen something he is looking for. Jessica Bayliss of the University of Rochester is currently doing research in using P300 detection to control objects in a 3D world. She has done significant research in the detection of the signal even in an electrically noisy environment. ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

22. Visual Evoked Potential (VEP) This is a system that involves monitoring the visual pathways in the brain. Generally a set of images or colors is viewed and the corresponding brain patterns are recorded. When the images are viewed again the controlling software can recognize the recorded patterns and recognize when the pattern or color is seen again. This technique is currently used to evaluate optic neuritis, optic tumors, retinal disorders and demyelineating diseases such as multiple sclerosis. ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

23. EEG Pattern Mapping The EEG patterns for specific activities such as saying a certain vowel, moving a specific limb, etc. are mapped out. This is the type of signal interpretation that most closely resembles “mind reading” it is also one of the more difficult to carry out successfully. The nerve signal mapping done for silent speech detection is very similar to this type of signal processing. ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

24. Silent Speech Detection: Principle and Application When a person reads or thinks to himself the nerve cells in the throat are stimulated even if the person doesn’t think out loud. NASA’s Ames Research Center is currently doing research on developing a system that will detect and interpret those signals. When fully developed this could result in completely silent voice activated systems. Basically you could “think” to your cell phone and the cell phone could pick up the signals just like your voice. ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

25. Silent Speech Detection: Current Research Currently the system developed at Ames can recognize specific letters. A system is being developed to allow the system to recognize simple command words. Full speech recognition is still a long way off and may or may not be possible. ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

26. Cognitive Engineering: Principles and Application This type of sensor is still in the experimental phase and is being conducted by Dr. Roy Bakay and Dr. Philip Kennedy at Emory University. The benefit of the system is that sensor is placed directly into the brain so there is no signal distortion by the skull and skin. Before surgery an MRI is used to determine the area of the brain used for muscle control and the implant is placed precisely in that spot. Unfortunately the system is that is inherently dangerous, requiring very delicate brain surgery and that it requires advanced electrical and software interfaces. Some groups worry about the social impact and potentially dangerous application of this technology. If the implant could be made to work both ways the possibility of mind control is feasible. ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

27. Cognitive Engineering: Device The implanted device is a neurotrophic electrode. This is a small glass case containing gold wires and small amount of nerve tissue from the leg. The nerve cells in the cone cause the surround nerve tissue to grow into the cell and form connections. When the brain’s neurons fire the signals are detected by the wires and transmitted to receivers and amplifiers placed outside of the scalp. ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators