1 / 65

Biologically Inspired Intelligent Systems

Biologically Inspired Intelligent Systems. Lecture 3 Dr. Roger S. Gaborski. Regions for Hearing, Touch, etc. How is each area unique?. Regions for Speech, Touch , etc. How is each area unique? INPUTS ARCHITECTURE OUTPUTS. Neural Networks as Systems. Networks have specialized

nitesh
Download Presentation

Biologically Inspired Intelligent Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Biologically Inspired Intelligent Systems Lecture 3 Dr. Roger S. Gaborski

  2. Regions for Hearing, Touch, etc. • How is each area unique?

  3. Regions for Speech, Touch, etc. • How is each area unique? • INPUTS • ARCHITECTURE • OUTPUTS

  4. Neural Networks as Systems Networks have specialized architecture in order to perform specific information processing actions Single neurons connected together to form networks of neurons Central nervous system strongly depends on interaction of many specialized subsystems

  5. System LevelWhat Function is Performed? • Evidence from: • Brain damage • Areas selectively activated during a particular activity (detected by fMRI, PET, etc.) • What inputs go to the region (record activity of neurons) • Which regions receive outputs from region being investigated?

  6. How Does a Particular Region Perform a Function?Neuron Level • How many inputs from each source received by a neuron in a given region? • Rules and mechanisms that determine synaptic connectivity and modifiability • How is information represented by neural responses

  7. Dyes • Voltage-sensitive dyes are dyes which change their spectral properties in response to voltage changes. • They are able to provide linear measurements of firing activity of single neurons, large neuronal populations http://en.wikipedia.org/wiki/Voltage-sensitive_dye

  8. Songbird’s Nerve Cell – Dyed Green http://media.charlotteobserver.com/smedia/2012/03/11/16/45/aq7OV.Em.138.jpg

  9. At the Neuron Level • What calculations are performed by a single neuron or group of neurons?? • We need to understand: • How are connections altered and why? • How is information about the problem to be solved represented and stored? • How do neurons in a region interact? • Together, how do these features enable useful computations to be performed?

  10. Our ability to • Perform complex behaviors • Process complex concepts • Learn • Remember • Depends on communications between a large number of neurons • Communication between neurons depend of action potentials (spikes) and synapses

  11. Connectionism • Compute using large number of elements • Elements do NOT model real neurons • Learning rules are not biologically feasible (backward error propagation) • This course is not about common connectionism approaches, i.e. backward error propagation • We are interested in how real neurons in networks compute • Basis for understanding the brain

  12. Neuron Models • Example of a neuron model

  13. Real Neurons www.alanturing.net/

  14. How Do Real Neurons ‘Operate’ ? http://ei.cs.vt.edu/~history/NEURLNET.HTML

  15. A Careful Examination of Neurons and the Flow of Information • BACKGROUND INFORMATION: Three factors influence the flow of ions into and out of neurons: • Charge • Diffusion • Concentration

  16. Charge • Recall, like charges repel (both positive or both negative), opposite charges attract (positive and negative) • Ions are charged - Number of electrons do not equal number of protons • Sodium, Na+ • Potassium, K+ • Chloride, Cl- • Bicarbonate HCO3- • Protein-

  17. Diffusion • Diffusion – random movement of ions or molecules from a high concentration to a low concentration • Illustration: • Place a drop of red dye into a container of clean water • The red dye will distribute itself equally moving from a high concentration to areas of low concentration

  18. Concentration Gradient • Concentration Gradient – the difference in concentration of a material between two spatial regions • Voltage Gradient- when a salt solution composed of positive and negative ions is poured into a glass of water • Ions move down a voltage gradient from area of high charge to an area of low charge • Positive and negative ions distribute themselves equally (electrostatic gradient)

  19. Cell Membrane V -+ +- -+ +- -+ -+ +- -+ +- -+ -+ + -+ + -+ - - - Add salt to contain of water with a barrier (cell Membrane). Ions cannot pass through Membrane now has holes so that only Cl- ions can pass. After a period of time Cl- ions will diffuse to right hand side. Cl- is not equally distributed because some are attracted to positive charges sodium, Na+ resultingin a Voltage difference across the membrane

  20. Axon • Assume axon is modeled by a cylinder. • The walls of the cylinder is the membrane. • This membrane contains ion channels that when open allow specific ions to pass

  21. Action Potential Image from: http://scienceblogs.com/clock/2006/06/bio101_lecture_6_physiology_re.php

  22. Resting Potential • The unequal distribution of positively and negatively charged ions across the membrane results in a voltage potential • Resting potential is typically -70mv, but can vary for different neurons

  23. Action Potential • The action potential is a rapid change of the membrane potential caused by the opening and closing of ion channels • At the start of an action potential the sodium channels open • What will the voltage difference between the inside and the outside of the axon have on the sodium ions?

  24. Action Potential • At the start of an action potential the sodium channels open • What will the voltage difference between the inside and the outside of the axon have on the sodium ions? • The positively charged sodium ions will move to the inside (opposite charges attract) • As more positive ions enter the axon it becomes more positively charged

  25. Movement of Action Potential Down the Axon • You can think of the action potential being generated by the flow of ions crossing the membrane as first the sodium channel opens, then closes and the potassium channel opens, then closes • This sequence of opening and closing of channels flows down the length of the axon resulting in the action potential flowing down the length of the axon

  26. http://faculty.washington.edu/chudler/ap.html

  27. Action Potential • http://www.dnatube.com/video/1105/Action-potential

  28. Pre-Synaptic and Post-Synaptic Neurons • The action potential travels down the axon from the cell body to the synapse (pre-synaptic neuron) • At the synapse the electrical signal is converted to a chemical signal • The chemical signal is propagated to the post-synapse neuron’s dendrite (Dendrites extend from the post-synaptic cell body and receive inputs from other pre-synaptic neurons through connections called synapses) • Neurons also communicate with muscle cells through chemical synapses

  29. http://en.wikipedia.org/wiki/Image:SynapseIllustration.png

  30. Action Potential and Synapse • http://www.youtube.com/watch?v=HnKMB11ih2o&feature=fvwp • http://www.youtube.com/watch?v=U0NpTdge3aw&feature=endscreen&NR=1 • http://www.youtube.com/watch?v=ifD1YG07fB8 • http://www.youtube.com/watch?v=9nUY6o-LCWY&NR=1 • http://www.youtube.com/watch?v=LT3VKAr4roo&feature=related

  31. Neuron/muscle Communication • Action potential reaches the axon terminal • The depolarization of the terminal membrane causes Ca++ channels to open • Ca++ ions enter the terminal • Ca++ ions triggers the release of the neurotransmitter acetylcholine into the synaptic cleft • The neurotransmitter binds with a receptor on the post-synaptic membrane • A channel on the post-synaptic membrane opens and Na+ ions enter the post-synaptic cell • This accomplishes the transmission of information from the pre-synaptic cell to the post-synaptic cell • The neurotransmitter is broken down by an enzyme and the ion channel closes

  32. Neurotransmitters • There are more than 25 known neurotransmitters • The combination of neurotransmitter and receptor either excites or inhibits the post-synaptic cell

  33. SUMMARY:Action Potential Generation • Step 1: Synaptic terminal receives action potential from its neuron • Step 2: Neurotransmitter (chemical) is released which crosses the synaptic cleft • Step 3: Results in depolarization in post-synaptic neuron by opening ion channels

  34. Action Potential Generation • Step 4: Summation of a sufficient number of depolarizations within the time constant of the receiving neuron (20-30 ms) produces sufficient depolarization that the neuron fires an action potential • 5000-20000 inputs per neuron

  35. Summary • Neuron: • Computational element • Sums inputs within a time constant • When sum of excitatory-inhibitory inputs exceed a threshold an action potential is produced which propagates down the neuron’s axon to all of its outputs

  36. Abstraction • Make simplifications to a system without loosing important features we want to understand • Study problem at various levels and make connections between different levels to explain how low level factors can (biochemical processes in a single neuron) influence large scale systems, such as behavior

  37. Light and Switch Example • What does an electrician need to model? • What would a light bulb designer need to model Need enough information model system you are trying to understand

  38. Neurons as Information Processing Units • Electrophysical and chemical processes enable specific information processing mechanisms in the brain • How can biologically complex neurons be modeled sufficiently to create information processing systems?

  39. Neural Computation • Need simplified models to make computations with large numbers of neurons tractable and highlight minimal features necessary to enable emergent properties of networks • Important to study sophisticated computational abilities of neurons future advances may depend on understanding more sophisticated processes in single neurons and their behavior in large networks

  40. Computational Neuroscience • Major focus  development and evaluation of models • Use computers because complexity of models make analytical analysis intractable, but analytical studies can provide deeper insight into features of models and the reasons behind numerical findings

  41. Neuron Models

  42. Artificial Neurons vk = bk*1 + x1*w1 + x2*w2 + …xn*wn Redefine vk as net and bk as simply b net = b + ∑ xi * wi The activation function, f, can take several forms, including The identity function, binary step function, bipolar step function, sigmoid function or a bipolar sigmoid function

  43. Artificial Neural Networks/Neural Network Basicshttp://en.wikibooks.org/wiki/Artificial_Neural_Networks/Neural_Network_Basics

  44. 2 Input Neuron Decision Boundary f(net) = 1 if net ≥ 0 = -1 if net < 0 Where net = b + ∑ xi * wi where i = 1 or 2 Decision Boundary: The line separating the positive and negative output values of the net as a function of the weights w1 and w2 b + x1w1+ x2w2 = 0 Assuming w2 ≠ 0, x2 = - (w1/w2)x1- (b/w2)

  45. 2 Input Neuron Decision Boundary Requirement for positive output: b + x1w1+ x2w2 > 0 During training the values for w1 , w2 and b are determined so that the neuron will have a positive output (correct response) during training

  46. Response for AND Function INPUT (x1 , x2 ) Output (t) (1,1) +1 (1, -1) -1 (-1,1) -1 (-1,-1) -1 Assume weights have been determined to be: b = -1, w1 = 1 and w2 = 1 b + x1w1+ x2w2 = 0, x2 = - (w1/w2)x1- (b/w2) x2 = -x1 –(-1/1) = -x1 +1

  47. Response for given weights (AND) x2 x2 = -x1 +1 Decision Boundary x1 x2 0 +1 -1 +2 +1 0 Responses + 2 1 x1 1 2 -1 -1 -

  48. Response for OR Function INPUT (x1 , x2 ) Output (t) (1,1) +1 (1, -1) +1 (-1,1) +1 (-1,-1) -1 Assume weights have been determined to be: b = 1, w1 = 1 and w2 = 1 b + x1w1+ x2w2 = 0, x2 = - (w1/w2)x1- (b/w2) x2 = -x1 –(1/1) = -x1 -1

  49. Response for given weights (OR) x2 x2 = -x1 -1 Decision Boundary x1 x2 0 -1 -1 0 +1 -2 Responses + 2 1 x1 1 2 -1 -1 -

  50. How do we find the weight values? Training Neurons – HEBB Rule • Initialize weights to 0 • Input training vector and output pair, s:t • For each training pair • For inputs: xi = si • For output: y = t • Adjust the weights: • wi(new) = wi(old) + xiy • Adjust bias: b(new) = b(old)+y

More Related