1 / 91

Neural networks

Neural networks. Eric Postma IKAT Universiteit Maastricht. Overview. Introduction: The biology of neural networks the biological computer brain-inspired models basic notions Interactive neural-network demonstrations Perceptron Multilayer perceptron Kohonen’s self-organising feature map

tareq
Download Presentation

Neural networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Neural networks Eric Postma IKAT Universiteit Maastricht

  2. Overview Introduction: The biology of neural networks • the biological computer • brain-inspired models • basic notions Interactive neural-network demonstrations • Perceptron • Multilayer perceptron • Kohonen’s self-organising feature map • Examples of applications

  3. A typical AI agent

  4. Two types of learning • Supervised learning • curve fitting, surface fitting, ... • Unsupervised learning • clustering, visualisation...

  5. An input-output function

  6. Fitting a surface to four points

  7. (Artificial) neural networks The digital computer versus the neural computer

  8. The Von Neumann architecture

  9. The biological architecture

  10. Digital versus biological computers 5 distinguishing properties • speed • robustness • flexibility • adaptivity • context-sensitivity

  11. Speed: The “hundred time steps” argument The critical resource that is most obvious is time. Neurons whose basic computational speed is a few milliseconds must be made to account for complex behaviors which are carried out in a few hudred milliseconds (Posner, 1978). This means that entire complex behaviors are carried out in less than a hundred time steps. Feldman and Ballard (1982)

  12. Graceful Degradation performance damage

  13. Flexibility: the Necker cube

  14. vision = constraint satisfaction

  15. Adaptivitiy processing implies learning in biological computers versus processing does not imply learning in digital computers

  16. Context-sensitivity: patterns emergent properties

  17. Robustness and context-sensitivitycoping with noise

  18. The neural computer • Is it possible to develop a model after the natural example? • Brain-inspired models: • models based on a restricted set of structural en functional properties of the (human) brain

  19. The Neural Computer (structure)

  20. Neurons, the building blocks of the brain

  21. Neural activity out in

  22. Synapses,the basis of learning and memory

  23. Learning: Hebb’s rule neuron 1 synapse neuron 2

  24. Connectivity An example: The visual system is a feedforward hierarchy of neural modules Every module is (to a certain extent) responsible for a certain function

  25. (Artificial) Neural Networks • Neurons • activity • nonlinear input-output function • Connections • weight • Learning • supervised • unsupervised

  26. Artificial Neurons • input (vectors) • summation (excitation) • output (activation) i1 a = f(e) i2 e i3

  27. 1 f(x) = 1 + e -x/a Input-output function • nonlinear function: a  0 f(e) a   e

  28. wAB A B Artificial Connections (Synapses) • wAB • The weight of the connection from neuron A to neuron B

  29. The Perceptron

  30. Learning in the Perceptron • Delta learning rule • the difference between the desired output tand the actual output o, given inputx • Global error E • is a function of the differences between the desired and actual outputs

  31. Gradient Descent

  32. Linear decision boundaries

  33. The history of the Perceptron • Rosenblatt (1959) • Minsky & Papert (1961) • Rumelhart & McClelland (1986)

  34. The multilayer perceptron input hidden output

  35. Training the MLP • supervised learning • each training pattern: input + desired output • in each epoch: present all patterns • at each presentation: adapt weights • after many epochs convergence to a local minimum

  36. phoneme recognition with a MLP Output: pronunciation input: frequencies

  37. Non-linear decision boundaries

  38. Compression with an MLPthe autoencoder

  39. hidden representation

  40. Learning in the MLP

  41. Preventing Overfitting GENERALISATION = performance on test set • Early stopping • Training, Test, and Validation set • k-fold cross validation • leaving-one-out procedure

  42. Image Recognition with the MLP

  43. Hidden Representations

  44. Other Applications • Practical • OCR • financial time series • fraud detection • process control • marketing • speech recognition • Theoretical • cognitive modeling • biological modeling

  45. Some mathematics…

  46. Perceptron

  47. Derivation of the delta learning rule Target output Actual output h = i

  48. MLP

  49. Sigmoid function • May also be thetanhfunction • (<-1,+1> instead of <0,1>) • Derivative f’(x) = f(x) [1 – f(x)]

More Related