introduction to neural networks
Download
Skip this Video
Download Presentation
Introduction to Neural Networks

Loading in 2 Seconds...

play fullscreen
1 / 18

Introduction to Neural Networks - PowerPoint PPT Presentation


  • 57 Views
  • Uploaded on

Introduction to Neural Networks. John Paxton Montana State University Summer 2003. Chapter 5: Adaptive Resonance Theory. 1987, Carpenter and Grossberg ART1: clusters binary vectors ART2: clusters continuous vectors. General.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Introduction to Neural Networks' - kyla-shelton


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
introduction to neural networks

Introduction to Neural Networks

John Paxton

Montana State University

Summer 2003

chapter 5 adaptive resonance theory
Chapter 5: Adaptive Resonance Theory
  • 1987, Carpenter and Grossberg
  • ART1: clusters binary vectors
  • ART2: clusters continuous vectors
general
General
  • Weights on a cluster unit can be considered to be a prototype pattern
  • Relative similarity is used instead of an absolute difference. Thus, a difference of 1 in a vector with only a few non-zero components becomes more significant.
general1
General
  • Training examples may be presented several times.
  • Training examples may be presented in any order.
  • An example might change clusters.
  • Nets are stable (patterns don’t oscillate).
  • Nets are plastic (examples can be added).
architecture
Architecture
  • Input layer (xi)
  • Output layer or cluster layer – competitive (yi)
  • Units in the output layer can be active, inactive, or inhibited.
sample network
Sample Network
  • t (top down weight), b (bottom up weight)

t11

x1

y1

xn

ym

bnm

nomenclature
Nomenclature
  • bij: bottom up weight
  • tij: top down weight
  • s: input vector
  • x: activation vector
  • n: number of components in input vector
  • m: maximum number of clusters
  • || x ||: S xi
  • p: vigilance parameter
training algorithm
Training Algorithm

1. L > 1, 0 < p <= 1 tji(0) = 1 0 < bij(0) < L / (L – 1 + n)

2. while stopping criterion is false do

steps 3 – 12

3. for each training example do

steps 4 - 12

training algorithm1
Training Algorithm

4. yi = 0

5. compute || s ||

6. xi = si

7. if yj (do for each j) is not inhibited then

yj = S bij xi

8. find largest yj that is not inhibited

9. xi = si * tji

training algorithm2
Training Algorithm

10. compute || x ||

11. if || x || / || s || < p then yj = -1, go to step 8

12. bij = L xi / ( L – 1 + || x || )

tji = xi

possible stopping criterion
Possible Stopping Criterion
  • No weight changes.
  • Maximum number of epochs reached.
what happens if all units are inhibited
What Happens If All Units Are Inhibited?
  • Lower p.
  • Add a cluster unit.
  • Throw out the current input as an outlier.
example
Example

x1

  • n = 4
  • m = 3
  • p = 0.4 (low vigilance)
  • L = 2
  • bij(0) = 1/(1 + n) = 0.2
  • tji(0) = 1

y1

x2

y2

x3

y3

x4

example1
Example

3. input vector (1 1 0 0)

4. yi = 0

5. || s || = 2

6. x = (1 1 0 0)

7. y1 = .2(1) + .2(1) + .2(0) + .2(0) = 0.4

y2 = y3 = y4 = 0.4

example2
Example

8. j = 1 (use lowest index to break ties)

9. x1 = s1 * t11 = 1 * 1 = 1

x2 = s2 * t12 = 1 * 1 = 1

x3 = s3 * t13 = 0 * 1 = 0

x4 = s4 * t14 = 0 * 1 = 0

10. || x || = 2

11. || x || / || s || = 1 >= 0.4

example3
Example

12. b11 = 2 * xi / (2 - 1 + || x ||)

= 2 * 1 / (1 + 2) = .667

b21 = .667

b31 = b41 = 0

t11 = x1 = 1

t12 = 1

t13 = t14 = 0

exercise
Exercise
  • Show the network after the training example (0 0 0 1) is processed.
observations
Observations
  • Typically, stable weight matrices are obtained quickly.
  • The cluster units are all topologically independent of one another.
  • We have just looked at the fast learning version of ART1. There is also a slow learning version that updates just one weight per training example.
ad