1 / 28

CS 621 Artificial Intelligence Lecture 29 – 22/10/05 Prof. Pushpak Bhattacharyya

CS 621 Artificial Intelligence Lecture 29 – 22/10/05 Prof. Pushpak Bhattacharyya SOM, Theoretical Aspects Of Machine Learning, Probably Approximately Correct Learning. A. …. P neurons o/p layer. W p. … …. n neurons. Clusters:. A :. A. : :. B :. C :.

virgo
Download Presentation

CS 621 Artificial Intelligence Lecture 29 – 22/10/05 Prof. Pushpak Bhattacharyya

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. CS 621 Artificial Intelligence Lecture 29 – 22/10/05 Prof. Pushpak Bhattacharyya SOM, Theoretical Aspects Of Machine Learning, Probably Approximately Correct Learning.

  2. A … P neurons o/p layer Wp … … . n neurons Clusters: A : A : : B : C :

  3. Wp(n+1) = Wp(n) + η (I(n) – Wp(n)) Competitive learning: Weights to the winner neuron only are changed.

  4. Generating function application to recurrence relation solving. Example: Max. No of regions on 2-Dim plane produced by n lines.

  5. L1 L3 y L2 R1 = 2 R2 = R1 + 2 = 2+ 2 = 4 L4 0 R3 = R2 + 3 = 4 + 3 = 7 R4 = 7 + 4 = 11 Rn = Rn-1 + n R1 = 2 x

  6. Rn = Rn-1 + n Rn-1 = Rn-2 + (n-1) Rn-2 = Rn-3 + (n-2) . . . R2 = R1 + 2

  7. Rn = n + (n-1) + (n-2)+ …..+R1 + 2 = n+(n-1)+(n-2)+…..+2+2 =n(n+1)/2 + 1

  8. Generating Function Method Key idea: Rns are generated by infinite series function ,i.e, F(x) = R1 + R2x + R3x2 + ………………+ Rnxn-1+ ….. ∞ Rn = Coeff of xn-1

  9. (Continued) Rn = Rn-1 + n Rn - Rn-1 = n R2 - R1 = 2 R3 – R2 = 3 . .

  10. (Continued) F(x) = R1 + R2x + R3x2 + ………………+ Rnxn-1+ ….. ∞ (1) xF(x) = R1x + R2x2 + R3x3 + ……………+ Rnxn+ ….. ∞ (2) - 2 gives (1-x) F(x) = R1 + (R2 – R1)x + (R3 – R2)x2 + ……………+ (Rn – Rn-1)xn-1+ ….. ∞

  11. (Continued) (1-x) F(x) = 2 + 2x + 3x2 + ……………+ nxn-1+ ….. ∞ F(x) = (1-x)-1(2 + 2x + 3x2 + ……………+ nxn-1+ ….. ∞) = ( 1 + x + x2 + x3 +…+ xn-1+…..) (2 + 2x + 3x2 + 4x3 +…+ nxn-1+…..)

  12. (Continued) Co eff of Xn-1 from the expression for f(x) = ( n + n-1 + n-2 + … + 2+ 2 ) n(n+1)/ 2 + 1

  13. Clustering Algos 1. Competitive learning 2. K – means clustering 3. Self organization / Kohonen net 4. Counter Propagation

  14. Two packages 1. SNNS – Stuttgart Neural Net Simulator 2. WEKA Package 3. MATLAB

  15. Data to train IRIS data: 3 - class data X- OR, parity, Majority

  16. IRIS Data Classification of flowers based on four attributes in 3 classes c1, c2 and C2 1 Petal length A1 2 Petal width A2 3 Sepal length A3 4 Sepal width A4

  17. Google on “Machine learning Data” IRIS 120 – data points < A1, A2, A3, A4 >1 --- C1 < A1, A2, A3, A4 >2 --- C3 ………. ……….

  18. C1, C2, C3 each has 50 data points in it 80 – 20 rule for ML Training – 40 and Testing -- 10

  19. What you have to do for IRIS choose 40 points randomly from C1, C2 and C3 1. Run BP on FF n/w 2. Run SOM assuming unlabelled data. See if you can discover 3 cluster

  20. Two part assignment Supervised Hidden layer to be found 4 neurons for 4 attributes

  21. error 1% BP for IRIS Training iterations EPOCHS Very good performance on test data should be achieved.

  22. Cluster Discovery By SOM/Kohenen Net 4 I/p neurons A4 A1 A2 A3

  23. …… 26 neurons K – means clustering K o/p neurons are required from the knowledge of k clusters being present. Full connection …… n neurons

  24. Steps 1. Initialize the weights randomly. 2. Ik is the vector presented at kth iteration. 3. Find W* such that |w* - Ik| < |wj - Ik| for all j

  25. 4. make W*(new) = W* (old) + η(Ik - w* ). 5. k  K +1 ; go to 3. 6. Go to 2 until the error is below a threshold.

  26. K means 1. Initialize weights randomly. 2. Ik in the I/p vector presented at kth. step of one iteration over all the training data (EPOCH). 3. Find the W* such that |w* - Ik| is minimum. 4. Find W* for Ik+1..

  27. 5. See the partitions at the end of the epoch. 6. Find centroid of each partition. 7. Make the weight equal to the centroid of the partition. 8. Take next epoch.

  28. 9. Keep doing until the error is below a threshold. Key idea form competitive learning: Weight change incorporate after every epoch.

More Related