1 / 11

120 likes | 319 Views

Recurrent Neural Networks or Associative Memories. Ranga Rodrigo February 24, 2014. Introduction. In a network, the signal received at the output was sent again to the network input. Such circulation of the signal is called feedback .

Download Presentation
## Recurrent Neural Networks or Associative Memories

**An Image/Link below is provided (as is) to download presentation**
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.
Content is provided to you AS IS for your information and personal use only.
Download presentation by click this link.
While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

**Recurrent Neural Networks or Associative Memories**Ranga Rodrigo February 24, 2014**Introduction**• In a network, the signal received at the output was sent again to the network input. • Such circulation of the signal is called feedback. • Such neural networks are are called recurrent neural networks. • Recurrent neural networks: • Hopfield neural network • Hamming neural network • Real Time Recurrent Network (RTRN) • Elman neural network • Bidirectional Associative Memory (BAM)**1**w10 w11 w21 wn1 z-1 wn2 w12 w22 y1 wnn w2n w1n 1 z-1 w20 . . . y2 . . . 1 z-1 wn0 yn**Hopfied Structure**• It is a one-layer network with a regular structure, made of many neurons connected one to the other. • Output is fedback after one-step time delay. • There are no feedbacks in the same neuron. • During learning weights wkjget modified depending on the value of learning vector x. • In retrieval mode, the input signal stimulates the network which, through the feedback, repeatedly receives the output signal at its input, until the answer is stabilized.**1**w10 w11 w21 wn1 z-1 wn2 w12 w22 y1 wnn w2n w1n 1 z-1 w20 . . . y2 . . . 1 z-1 wn0 yn**Learning Methods for Hopfield: Hebb**• The generalized Hebbian rule • for M leaning vectors of the form • The maximum number of patterns which the network is able to memorize using this rule is only 13.8% of the number of neurons.**Learning Methods for Hopfield: Pseudoinverse**• for M leaning vectors of the form • Form a matrix of learning vectors • The nn weights matrix W is found as • In Matlab • W = X*pinv(X)**Operation of the Hamming NN**• In the first layer, there are p neurons, which determine the Hamming distance between the input vector and each of the p desired vectors coded in the weights of this layer. • The second layer is called MAXNET. It is a layer corresponding to the Hopfield network. However, in this layer feedback covering the same neuron are added. The weights in these feedbacks are equal to 1. The values of weights of other neurons of this layer are selected so that they inhibit the process. • Thus, in the MAXNET layer there is the extinction of all outputs except the one which was the strongest in the first layer. • The neuron of this layer, which is identified with the winner, through the weights of output neurons with a linear activation function, will retrieve the output vector associated with the vector coded in the first layer.

More Related