1 / 42

Neural Network Weights Implementation and Adjustment

Understand how to manage weights between layers in a neural network and make adjustments for accurate outputs and error reduction.

daquan-burch
Download Presentation

Neural Network Weights Implementation and Adjustment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Hopefully a clearer version of Neural Network

  2. O1 I1 H1 I2 H2 O2

  3. Layers of Weights • We Name Sets of Weights between layers As W1 for weights between input Layer and First Hidden Layer W2 for weights between next 2 layers and WN-1 for Weights between N-1th and Nth Layer(i.e. Output Layer) In our example Net we just have 3 layers Input Hidden and Output So we have just W1 and W2

  4. W1 W2 O1 I1 H1 I2 H2 O2

  5. Weights along Individual Links • Convention • Each Weight is named as follows • WNij • N refers to the Layer of Weights • So Between Input and First Hiden Layer i.e. W2ij is the Reference • Between Hidden and Output W2ij

  6. Individual Weights within a layer • Reference WNij • WN refers to the Weight Layer • ij refers to the indices of the source and destination nodes. • So for example the weight between hidden node h1 and output node o2 • It belongs to weight layer 2 so W2 • i = 1 and j = 2 so Weight is W212

  7. W1 W2 O1 I1 H1 W212 I2 H2 O2

  8. Full Naming of Weight Set

  9. W2 W1 W111 W211 O1 I1 H1 W121 W221 W212 W112 I2 H2 O2 W122 W222

  10. With Actual Weights

  11. W2 W1 1 -1 O1 I1 H1 0 0 0 -1 I2 H2 O2 1 -1

  12. Inputs • 1 and 0 • Target outputs {1,1}

  13. W2 W1 1 1 -1 O1 I1 H1 0 0 0 -1 0 I2 H2 O2 1 -1

  14. Hidden Layer Computation • Xi =iW1 = • 1 * 1 + 0 * -1 = 1, • 1 * -1 + 0 * 1 = -1 = • { 1 - 1} = {Xi1,Xi2} = Xi

  15. h = F(X) • h1 = F(Xi1) = F(1) • h2 = F(Xi2) = F(-1)

  16. W2 W1 0.73 1 1 -1 O1 I1 H1 0 0 0 -1 0 I2 H2 O2 1 -1 0.27

  17. Next Outputs

  18. Output Layer Computation • X = hW2 = • 0.73 * -1 + 0.27 * 0 = -0.73, • 0.73 * 0 + 0.27 * -1 = -0.27 = • { -0.73 - 0.27} = {X1,X2} = X

  19. O = F(X) • O1 = F(X1) • O2 = F(X2)

  20. W2 W1 0.73 1 1 -1 O1 I1 H1 0.325 0 0 0 -1 0 I2 H2 O2 1 -1 0.433 0.27

  21. Error • D= Output(1 – Output)(Target – Output) • Target T1 = 1 , O1 = 0.325 = 0.33 • d1 = 0.33( 1 -0.33)(1 -0.33 ) = 0.33 (0.67)(0.67) = 0.148 • Target T2 = 1 , O2 = 0.433 = 0.43 • d2 = 0.43(1 - 0.43)(1-0.43) = 0.43(0.57)(0.57) = 0.14

  22. Weight Adjustment • △W2t = αhd + Θ △W2t-1 • where α = 1 • Time t = 1 so no previous time

  23. Weight Adjustments

  24. Weight Change

  25. Equals

  26. Equals

  27. Putting these new weights in the diagram • To get

  28. W2 W1 1 -0.891 O1 I1 H1 0 0.04 0.102 -1 I2 H2 O2 1 -0.962

  29. Next • Calculate Change on W1 layer weights

  30. Error Calculatione = h(1 - h)W2d

  31. Another Way to write the error

  32. What is this • Outputs are O1 and O2 • So k = {1,2} • So if i = 1

  33. W2 W1 1 -0.891 O1 I1 H1 0 0.04 0.102 -1 I2 H2 O2 1 -0.962

  34. This equals • e1 = (h1(1-h1)W11 D1 +W12D2 • e2 = (h2(1-h2)) W21 D1 +W22D2 • d1 = 0.15 d2 = = 0.14 e1 = (0.73(1-0.73))( -1* 0.15 +0*0.14) • e2 =( 0.27(1-0.27)) (0 *0.15 +-1*0.14) • e1 = (0.73(0.27)( -0.15)) • e2 =( 0.27(0.73)) (-0.14) • e1 = -0.03 • e2 = -0.028

  35. Weight Adjustment • △W1t = αIe + Θ △W2t-1 • where α = 1

  36. Weight Adjustment

  37. Existing W1

  38. Weight Change W1

  39. New W1

  40. Changing Net

  41. W2 W1 0.97 -1.109 O1 I1 H1 0 -0.04 -0.102 -1.028 I2 H2 O2 1 -1.038

More Related