Artificial vs Biological Neural Networks: models and debates

1. Artificial vs Biological Neural Networks: models and debates A presentation based on Lehky & Sejnowski’s network model of shape-from-shading Presented by Clara Boyd and Angelos Stavrou

2. Different Types: (if the neurons of one of the net's layers may be connected among each other) • Feed Forward • Feed Back A Brief Overview of Artificial Neural Networks • Different Learning Algorithm: (A mathematical algorithm that a neural net uses to learn specific problems) • Backpropagation • Delta Learning Rule • Forward Propagation • Hebb Learning Rule • Simulated Annealing

3. Perceptron • The Perceptron was first introduced by F. Rosenblatt in 1958 A Brief Overview of Artificial Neural Networks Type: Feedforward Neuron layers: 1 input layer 1 output layer Input value types: Binary Learning Method: Supervised

4. Multi-Layer-Perceptron • The Multi-Layer-Perceptron was first introduced by M. Minsky and S. Papert in 1969 Type: Feedforward Neuron layers: 1 input layer 1 or more hidden layers 1 output layer Input value types: Binary Learning Method: Supervised A Brief Overview of Artificial Neural Networks

5. Backpropagation Network • The Backpropagation Net was first introduced by G.E. Hinton, E. Rumelhart and R.J. Williams in 1986 Type: Feedforward Neuron layers: 1 input layer 1 or more hidden layers 1 output layer Input value types: Binary Learning Method: BackPropagation A Brief Overview of Artificial Neural Networks

6. Hubel & Wiesel • Area V1 in the Monkey: • Receptive Fields (orientation selectivity to bar of light) • Vision based on a set of EMERGENT properties • Each cortical cell extracts a different feature of the visual image Simple Cell Complex Cell

7. Macrocircuitry Between Visual Areas MT 1.Redundancy of Connections PO V3 VP PIP V2 2. Bidirectional Transport V1 3. Hierarchical Organization 4. Parallel Pathways

8. Hierarchical Arrangement Of Visual Processing Stages

9. The Visual Pathway Decisions & Actions (& Conscious Awareness?) Prefrontal Areas & Premotor Areas “Higher” Visual Areas (V2, V3, V4, Medial Temporal) Striate Cortex (V1/area 17) Lateral Geniculate Nucleus Retina

10. Microcircuitry: V1 Organization Layer Specific 1. Main Input: from different parts (I,P,M) of LGN terminate in different lamina (mostly lamina #4) 2. Other Inputs: (V2,V3,etc) avoid lamina #4 3. Resident Cells: characteristic for a given layer a) lamina to lamina – recurrent/colateral branches form circuit b) projection axons – exhibit lamina specificity Highly Localized Processing - most V1 projections don’t go very far - more vertical than horizontal Many Synapses - convergence and divergence - stellate cells/local interneurons & pyramidal neurons

15. Discussion and Open Questions • Learning using a Back propagation technique vs pure Feed Forward models of Hubel & Wiesel • How extensive is the inherited genetic knowledge? Equivalency of models of Artificial neural networks to Biological systems (Strong / Weak)

16. Discussion and Open Questions • But is our knowledge of learning adequate? • How the Feed-Forward network is created? Although 80% of the artificial neural networks work using Back propagation there is no strong biological support this rule. • Different “modes” of learning Feed-Forward vs Back-Propagation but same result? • Intrinsic properties are necessary in any case of a biological network, evidence of “prenatal neural networks”

17. Discussion and Open Questions • Master / Slave approach and Rule based Learning. But is Back-propagation learning achieved by an “outer” and bigger environment/network? • Maybe the truth is a hybrid of genetically inherited knowledge and learning rules on hierarchical unstructured neural networks.