1 / 28

ARTIFICIAL NEURAL NETWORKS – Multilayer Perceptrons

ARTIFICIAL NEURAL NETWORKS – Multilayer Perceptrons. Activation Function. Note: For a linear unit, the threshold activation function is replaced by a linear function. (Desired Value). (For a linear activation function). NOTE:

dom
Download Presentation

ARTIFICIAL NEURAL NETWORKS – Multilayer Perceptrons

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. ARTIFICIAL NEURAL NETWORKS – Multilayer Perceptrons CI Lectures: Feb 2011

  2. Activation Function Note: For a linear unit, the threshold activation function is replaced by a linear function. CI Lectures: Feb 2011

  3. (Desired Value) CI Lectures: Feb 2011

  4. CI Lectures: Feb 2011

  5. CI Lectures: Feb 2011

  6. (For a linear activation function) CI Lectures: Feb 2011

  7. NOTE: If the training examples are not linearly separable, we need a Multilayer Perceptron) CI Lectures: Feb 2011

  8. F1, F2: Features extracted from Speech Signal Head, Hid, etc.: Words or Classes CI Lectures: Feb 2011

  9. A B B A B A A B B A B A A B B A B A Different non linearly separable problems Types of Decision Regions Exclusive-OR Problem Classes with Meshed regions Most General Region Shapes Structure Single-Layer Half Plane Bounded By Hyperplane Two-Layer Convex Open Or Closed Regions Abitrary (Complexity Limited by No. of Nodes) Three-Layer Neural Networks – An Introduction Dr. Andrew Hunter CI Lectures: Feb 2011

  10. Sigmoid Activation Function descent CI Lectures: Feb 2011

  11. CI Lectures: Feb 2011

  12. CI Lectures: Feb 2011

  13. CI Lectures: Feb 2011

  14. (Sigmoid is linear in the middle region) CI Lectures: Feb 2011

  15. CI Lectures: Feb 2011

  16. CI Lectures: Feb 2011

  17. CI Lectures: Feb 2011

  18. CI Lectures: Feb 2011

  19. Radial Basis Functions (RBFs) • Features • One hidden layer • The activation of a hidden unit is determined by the distance between the input vector and a prototype vector Outputs Radial units Inputs CI Lectures: Feb 2011

  20. RBF hidden layer units have a receptive field which has a centre Generally, the hidden unit function is Gaussian The output Layer is linear Realized function CI Lectures: Feb 2011

  21. Learning • The training is performed by deciding on • How many hidden nodes there should be • The centers and the sharpness of the Gaussians • 2 steps • In the 1st stage, the input data set is used to determine the parameters of the basis functions • In the 2nd stage, functions are kept fixed while the second layer weights are estimated ( Simple BP algorithm like for MLPs) CI Lectures: Feb 2011

  22. Classification MLPs separate classes via hyperplanes RBFs separate classes via hyperspheres Learning MLPs use distributed learning RBFs use localized learning RBFs train faster Structure MLPs have one or more hidden layers RBFs have only one layer RBFs require more hidden neurons => curse of dimensionality MLPs versus RBFs MLP X2 X1 X2 RBF X1 CI Lectures: Feb 2011

  23. Statistics vs. Neural Networks CI Lectures: Feb 2011

  24. See It in ActionDEMO CI Lectures: Feb 2011

  25. Summary of ANN Algorithms CI Lectures: Feb 2011

  26. CI Lectures: Feb 2011

  27. Conclusion on Neural Networks • Neural networks are utilized as statistical tools • Adjust non linear functions to fulfill a task • Need of multiple and representative examples but fewer than in other methods • Neural networks enable to model complex static phenomena (FF) as well as dynamic ones (RNN) • NN are good classifiers BUT • Good representations of data have to be formulated • Training vectors must be statistically representative of the entire input space • Unsupervised techniques can help • The use of NN needs a good comprehension of the problem CI Lectures: Feb 2011

  28. References • A. K. Jain, J.Mao, K.Mohiuddin, “ANN a Tutorial”, IEEE Computer, 1996 March, pp 31-44 (Some Figures and Tables taken from this reference) • B. Yegnanarayana, Artificial Neural Networks, Prentice Hall of India, 2001. • Y. M. Zurada, Introduction to Artificial Neural Systems, Jaico, 1999. • MATLAB neural networks toolbox and manual CI Lectures: Feb 2011

More Related