Hopefully a clearer version of Neural Network

1. Hopefully a clearer version of Neural Network

2. O1 I1 H1 I2 H2 O2

3. Layers of Weights • We Name Sets of Weights between layers As W1 for weights between input Layer and First Hidden Layer W2 for weights between next 2 layers and WN-1 for Weights between N-1th and Nth Layer(i.e. Output Layer) In our example Net we just have 3 layers Input Hidden and Output So we have just W1 and W2

4. W1 W2 O1 I1 H1 I2 H2 O2

5. Weights along Individual Links • Convention • Each Weight is named as follows • WNij • N refers to the Layer of Weights • So Between Input and First Hiden Layer i.e. W2ij is the Reference • Between Hidden and Output W2ij

6. Individual Weights within a layer • Reference WNij • WN refers to the Weight Layer • ij refers to the indices of the source and destination nodes. • So for example the weight between hidden node h1 and output node o2 • It belongs to weight layer 2 so W2 • i = 1 and j = 2 so Weight is W212

7. W1 W2 O1 I1 H1 W212 I2 H2 O2

8. Full Naming of Weight Set

9. W2 W1 W111 W211 O1 I1 H1 W121 W221 W212 W112 I2 H2 O2 W122 W222

10. With Actual Weights

11. W2 W1 1 -1 O1 I1 H1 0 0 0 -1 I2 H2 O2 1 -1

12. Inputs • 1 and 0 • Target outputs {1,1}

13. W2 W1 1 1 -1 O1 I1 H1 0 0 0 -1 0 I2 H2 O2 1 -1

14. Hidden Layer Computation • Xi =iW1 = • 1 * 1 + 0 * -1 = 1, • 1 * -1 + 0 * 1 = -1 = • { 1 - 1} = {Xi1,Xi2} = Xi

15. h = F(X) • h1 = F(Xi1) = F(1) • h2 = F(Xi2) = F(-1)

16. W2 W1 0.73 1 1 -1 O1 I1 H1 0 0 0 -1 0 I2 H2 O2 1 -1 0.27

17. Next Outputs

18. Output Layer Computation • X = hW2 = • 0.73 * -1 + 0.27 * 0 = -0.73, • 0.73 * 0 + 0.27 * -1 = -0.27 = • { -0.73 - 0.27} = {X1,X2} = X

19. O = F(X) • O1 = F(X1) • O2 = F(X2)

20. W2 W1 0.73 1 1 -1 O1 I1 H1 0.325 0 0 0 -1 0 I2 H2 O2 1 -1 0.433 0.27

21. Error • D= Output(1 – Output)(Target – Output) • Target T1 = 1 , O1 = 0.325 = 0.33 • d1 = 0.33( 1 -0.33)(1 -0.33 ) = 0.33 (0.67)(0.67) = 0.148 • Target T2 = 1 , O2 = 0.433 = 0.43 • d2 = 0.43(1 - 0.43)(1-0.43) = 0.43(0.57)(0.57) = 0.14

22. Weight Adjustment • △W2t = αhd + Θ △W2t-1 • where α = 1 • Time t = 1 so no previous time

23. Weight Adjustments

24. Weight Change

25. Equals

26. Equals

27. Putting these new weights in the diagram • To get

28. W2 W1 1 -0.891 O1 I1 H1 0 0.04 0.102 -1 I2 H2 O2 1 -0.962

29. Next • Calculate Change on W1 layer weights

30. Error Calculatione = h(1 - h)W2d

31. Another Way to write the error

32. What is this • Outputs are O1 and O2 • So k = {1,2} • So if i = 1

34. W2 W1 1 -0.891 O1 I1 H1 0 0.04 0.102 -1 I2 H2 O2 1 -0.962

35. This equals • e1 = (h1(1-h1)W11 D1 +W12D2 • e2 = (h2(1-h2)) W21 D1 +W22D2 • d1 = 0.15 d2 = = 0.14 e1 = (0.73(1-0.73))( -1* 0.15 +0*0.14) • e2 =( 0.27(1-0.27)) (0 *0.15 +-1*0.14) • e1 = (0.73(0.27)( -0.15)) • e2 =( 0.27(0.73)) (-0.14) • e1 = -0.03 • e2 = -0.028

36. Weight Adjustment • △W1t = αIe + Θ △W2t-1 • where α = 1

37. Weight Adjustment

38. Existing W1

39. Weight Change W1

40. New W1

41. Changing Net

42. W2 W1 0.97 -1.109 O1 I1 H1 0 -0.04 -0.102 -1.028 I2 H2 O2 1 -1.038