backpropagation n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Backpropagation PowerPoint Presentation
Download Presentation
Backpropagation

Loading in 2 Seconds...

play fullscreen
1 / 18

Backpropagation - PowerPoint PPT Presentation


  • 70 Views
  • Uploaded on

Backpropagation. Linear separability constraint. 3. 1. 0. w 1. w 2. 1. 2. 0. 1. What if we add an extra layer between input and output?. 5. w 5. w 6. 3. 4. w 2. w 3. w 1. w 4. 1. 2. Same as a linear network without any hidden layer!. What if we use thresholded units?. 5.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Backpropagation' - shay-farmer


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide3

3

1

0

w1

w2

1

2

0

1

slide5

5

w5

w6

3

4

w2

w3

w1

w4

1

2

Same as a linear network without

any hidden layer!

slide7

5

w5

w6

If netj > thresh, aj = 1

Else aj = 0

3

4

w2

w3

w1

w4

1

2

slide8

0

1

1

1

0

0

1

0

1

0

5

If netj > 9.9, aj = 1

Else aj = 0

10

-10

0

1

3

4

10

10

5

5

Unit 3

1

2

Unit 4

slide9

So with thresholded units and a hidden layer, solutions exist…

  • …and solutions can be viewed as “re-representing” the inputs, so as to make the mapping to the output unit learnable.
  • BUT, how can we learn the correct weights instead of just setting them by hand?
slide10

But what if:

Simple delta rule:

…What function should we use for aj?

slide11

1.00

0.90

0.80

0.70

0.60

0.50

Change in activation

0.40

Activation

0.30

0.20

0.10

0.00

-10

-5

0

5

10

Net input

slide13

5

w5

w6

3

4

w2

w3

w1

w4

1

2

slide14

5

6

Targets

For outputs delta computed directly based on error.

Delta is stored at each unit and also used directly to adjust each incoming weight.

5

6

Output

3

4

For hidden units, there are no targets; “error” signal

is instead the sum of the output unit deltas. These are used to compute deltas for the hidden units, which are again stored with unit and used to directly change incoming weights.

Hidden

1

2

Deltas, and hence error signal at output, can propagate backward through network through many layers until it reaches the input.

Input

slide16

Sum-squared error:

5

w5

w6

3

4

w2

w3

w1

w4

1

2

Cross-entropy error:

slide17

5

w5

w6

3

4

w2

w3

w1

w4

1

2

slide18

1

0

0

1

3

w1

w2

1

2

2