1 / 18

# Introduction to Neural Networks - PowerPoint PPT Presentation

Introduction to Neural Networks. John Paxton Montana State University Summer 2003. Chapter 5: Adaptive Resonance Theory. 1987, Carpenter and Grossberg ART1: clusters binary vectors ART2: clusters continuous vectors. General.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about ' Introduction to Neural Networks' - kyla-shelton

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Introduction to Neural Networks

John Paxton

Montana State University

Summer 2003

• 1987, Carpenter and Grossberg

• ART1: clusters binary vectors

• ART2: clusters continuous vectors

• Weights on a cluster unit can be considered to be a prototype pattern

• Relative similarity is used instead of an absolute difference. Thus, a difference of 1 in a vector with only a few non-zero components becomes more significant.

• Training examples may be presented several times.

• Training examples may be presented in any order.

• An example might change clusters.

• Nets are stable (patterns don’t oscillate).

• Nets are plastic (examples can be added).

• Input layer (xi)

• Output layer or cluster layer – competitive (yi)

• Units in the output layer can be active, inactive, or inhibited.

• t (top down weight), b (bottom up weight)

t11

x1

y1

xn

ym

bnm

• bij: bottom up weight

• tij: top down weight

• s: input vector

• x: activation vector

• n: number of components in input vector

• m: maximum number of clusters

• || x ||: S xi

• p: vigilance parameter

1. L > 1, 0 < p <= 1 tji(0) = 1 0 < bij(0) < L / (L – 1 + n)

2. while stopping criterion is false do

steps 3 – 12

3. for each training example do

steps 4 - 12

4. yi = 0

5. compute || s ||

6. xi = si

7. if yj (do for each j) is not inhibited then

yj = S bij xi

8. find largest yj that is not inhibited

9. xi = si * tji

10. compute || x ||

11. if || x || / || s || < p then yj = -1, go to step 8

12. bij = L xi / ( L – 1 + || x || )

tji = xi

• No weight changes.

• Maximum number of epochs reached.

• Lower p.

• Add a cluster unit.

• Throw out the current input as an outlier.

x1

• n = 4

• m = 3

• p = 0.4 (low vigilance)

• L = 2

• bij(0) = 1/(1 + n) = 0.2

• tji(0) = 1

y1

x2

y2

x3

y3

x4

3. input vector (1 1 0 0)

4. yi = 0

5. || s || = 2

6. x = (1 1 0 0)

7. y1 = .2(1) + .2(1) + .2(0) + .2(0) = 0.4

y2 = y3 = y4 = 0.4

8. j = 1 (use lowest index to break ties)

9. x1 = s1 * t11 = 1 * 1 = 1

x2 = s2 * t12 = 1 * 1 = 1

x3 = s3 * t13 = 0 * 1 = 0

x4 = s4 * t14 = 0 * 1 = 0

10. || x || = 2

11. || x || / || s || = 1 >= 0.4

12. b11 = 2 * xi / (2 - 1 + || x ||)

= 2 * 1 / (1 + 2) = .667

b21 = .667

b31 = b41 = 0

t11 = x1 = 1

t12 = 1

t13 = t14 = 0

• Show the network after the training example (0 0 0 1) is processed.

• Typically, stable weight matrices are obtained quickly.

• The cluster units are all topologically independent of one another.

• We have just looked at the fast learning version of ART1. There is also a slow learning version that updates just one weight per training example.