1 / 70

Introduction to Artificial Neural Networks

Introduction to Artificial Neural Networks. 主講人 : 虞台文. Content. Fundamental Concepts of ANNs. Basic Models and Learning Rules Neuron Models ANN structures Learning Distributed Representations Conclusions. Introduction to Artificial Neural Networks. Fundamental Concepts of ANNs.

maeko
Download Presentation

Introduction to Artificial Neural Networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Introduction to ArtificialNeural Networks 主講人: 虞台文

  2. Content • Fundamental Concepts of ANNs. • Basic Models and Learning Rules • Neuron Models • ANN structures • Learning • Distributed Representations • Conclusions

  3. Introduction to ArtificialNeural Networks Fundamental Concepts of ANNs

  4. What is ANN? Why ANN? • ANN  Artificial Neural Networks • To simulate human brain behavior • A new generation of information processing system.

  5. Applications • Pattern Matching • Pattern Recognition • Associate Memory (Content Addressable Memory) • Function Approximation • Learning • Optimization • Vector Quantization • Data Clustering • . . .

  6. Traditional Computers are inefficient at these tasks although their computation speed is faster. Applications • Pattern Matching • Pattern Recognition • Associate Memory (Content Addressable Memory) • Function Approximation • Learning • Optimization • Vector Quantization • Data Clustering • . . .

  7. The Configuration of ANNs • An ANN consists of a large number of interconnected processing elements called neurons. • A human brain consists of ~1011 neurons of many different types. • How ANN works? • Collective behavior.

  8. The Biologic Neuron

  9. The Biologic Neuron 二神經原之神經絲接合部分 軸突 樹狀突

  10. The Biologic Neuron Excitatory or Inhibitory

  11. x1 wi1 x2  yi i wi2 f (.) a (.) wim xm The Artificial Neuron

  12. x1 wi1 x2  yi i wi2 f (.) a (.) wim xm The Artificial Neuron

  13. x1 wi1 x2  yi i wi2 f (.) a (.) wim xm wij positive  excitatory negative  inhibitory zero  no connection The Artificial Neuron

  14. x1 wi1 x2  yi i wi2 f (.) a (.) wim xm The Artificial Neuron Proposed by McCulloch and Pitts [1943] M-P neurons

  15. y x2 w1 x1 + w2 x2 =   x1 w1 w2 x1 x2 What can be done by M-Pneurons? • A hard limiter. • A binary threshold unit. • Hyperspace separation. 0 1

  16. What ANNs will be? • ANN  A neurally inspired mathematical model. • Consists a large number of highly interconnected PEs. • Its connections (weights) holds knowledge. • The response of PE depends only on local information. • Its collective behavior demonstrates the computation power. • With learning, recalling and, generalization capability.

  17. Three Basic Entities of ANN Models • Models of Neurons or PEs. • Models of synaptic interconnections and structures. • Training or learning rules.

  18. Introduction to ArtificialNeural Networks Basic Models and Learning Rules Neuron Models ANN structures Learning

  19. i f (.) a (.) Processing Elements Extensions of M-P neurons What integration functions we may have? What activation functions we may have?

  20. M-P neuron  Quadratic Function i Spherical Function f (.) a (.) Polynomial Function Integration Functions

  21. i a f (.) a (.) 1 f Activation Functions M-P neuron: (Step function)

  22. i a f (.) a (.) 1 f 1 Activation Functions Hard Limiter (Threshold function)

  23. i f (.) a (.) a 1 f 1 Activation Functions Ramp function:

  24. i f (.) a (.) Activation Functions Unipolar sigmoid function:

  25. i f (.) a (.) Activation Functions Bipolar sigmoid function:

  26. y L1 L3 L1 L2 L3 L2 y x x Example: Activation Surfaces

  27. y L1 1=1 3= 4 2=1 L3 L1 L2 L3 1 0 1 1 L2 1 0 y x x Example: Activation Surfaces x1=0 xy+4=0 y1=0

  28. y L1 L3 L1 L2 L3 L2 y x x Example: Activation Surfaces 010 Region Code 011 110 111 001 101 100

  29. z y L1 L4 L3 L1 L2 L3 L2 y x x Example: Activation Surfaces z=0 z=1

  30. z y L1 4=2.5 L4 L3 1 1 1 L1 L2 L3 L2 y x x Example: Activation Surfaces z=0 z=1

  31. z L4 L1 L2 L3 y x Example: Activation Surfaces M-P neuron: (Step function)

  32. =2 z =3 L4 L1 L2 L3 =5 =10 y x Unipolar sigmoid function: Example: Activation Surfaces

  33. Introduction to ArtificialNeural Networks Basic Models and Learning Rules Neuron Models ANN structures Learning

  34. ANN Structure (Connections)

  35. y1 y2 yn . . . w1m w2m wn1 w21 w22 wn2 w11 wnm w12 x1 x2 xm Single-Layer Feedforward Networks

  36. y1 y2 yn . . . Output Layer . . . . . . . . . Input Layer x1 x2 xm Multilayer Feedforward Networks Hidden Layer

  37. Pattern Recognition Multilayer Feedforward Networks Where the knowledge from? Classification Output Analysis Learning Input

  38. Single Node with Feedback to Itself Feedback Loop

  39. y1 y2 yn . . . x1 x2 xm Single-Layer Recurrent Networks

  40. y1 y2 y3 . . . . . . x1 x2 x3 Multilayer Recurrent Networks

  41. Introduction to ArtificialNeural Networks Basic Models and Learning Rules Neuron Models ANN structures Learning

  42. Learning • Consider an ANN with n neurons and each with m adaptive weights. • Weight matrix:

  43. How? Learning To “Learn” the weight matrix. • Consider an ANN with n neurons and each with m adaptive weights. • Weight matrix:

  44. Learning Rules • Supervised learning • Reinforcement learning • Unsupervised learning

  45. Supervised Learning • Learning with a teacher • Learning by examples  Training set

  46. y x ANN W Supervised Learning d Error signal Generator

  47. Reinforcement Learning • Learning with a critic • Learning by comments

  48. y x ANN W Reinforcement Learning Reinforcement Signal Critic signal Generator

  49. Unsupervised Learning • Self-organizing • Clustering • Form proper clusters by discovering the similarities and dissimilarities among objects.

  50. y x ANN W Unsupervised Learning

More Related