Bit-True Modeling of Neural Network SILab presentation Ali Ahmadi June 2007

1 / 22

# Bit-True Modeling of Neural Network SILab presentation Ali Ahmadi June 2007 - PowerPoint PPT Presentation

Bit-True Modeling of Neural Network SILab presentation Ali Ahmadi June 2007. Outline. Introduction structures of Neural Network Hopfield LAM BAM Bit-True Arithmetic Training modes for NN hardware Bit-True model of networks Simulation results. Hopfield Network. Single layer

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about ' Bit-True Modeling of Neural Network SILab presentation Ali Ahmadi June 2007' - jaden-franco

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Bit-True Modeling of Neural Network

SILab presentation

June 2007

Outline
• Introduction structures of Neural Network
• Hopfield
• LAM
• BAM
• Bit-True Arithmetic
• Training modes for NN hardware
• Bit-True model of networks
• Simulation results
Hopfield Network
• Single layer
• Fully connected

[1]

Weight calculation

Tji = Tij = if i ≠ j; Tii = 0;

where api is ith element of pth pattern

• Updating Neuron for input neuron U

Sj = then uj =

[1]

LAM (Linear Associative Memory) Network

single-layer feed-forward network

recover the output pattern from full or partial information in the input pattern

[1]

Weight calculation

Wij = (( 2* aim -1)( 2* bjm -1))

where aim is ith element of mth pattern

Threshold values are Ti=

• Output calculation

For input pattern b output pattern is a

Ui = then ai =

BAM (Bidirectional Associative Memory) Network

bidirectional

Two layer with different dimension

For each pattern we have pair (a, b) related to each layer

In X to Y pass W

In Y to X pass WT

• Weight calculation

Wij =

• Output calculation

In forward pass input of jth neuron in layer Y is:

Net y(j) = then yj =

In backward pass input of jth neuron in layer X is:

Net x(j) = then xj =

[1]

Bit-True Arithmetic
• SUM

Inputs are 2’s complementwith length (WL-1)

output is 2’s complement with length WL

If inputs haven\'t same sign, based on carry we make Sign-Extension

Bit-True Arithmetic
• Multiply

Inputs are 2’s complementwith length WL

output is 2’s complement with length WL

Training modes for Neural Network hardware
• Off-chip learning: training process is performed out of chip with high precision, forward propagation pass in the recall phase is performed on-chip.
• Chip-in-the-loop learning: chipisused during training but only in forward propagation.
• On-chip learning: training is done entirely on-chip, sensitive to the use of limited precision weights.
Bit-true Model of Hopfield
• Part of code in updating neuron

sum+=t[j][i] * neuron[i]; // high precision arithmetic

// arithmetic with finite word-length

b 1 = t[j][i];

a1 = neuron[i];

a = Decimal2TwosComplement(a1, Word-length-1);

b = Decimal2TwosComplement(b1, Word-length-1);

c = MulBitTruePrecise(a, b, Word-length-1);

s = Decimal2TwosComplement(sum, Word-length-1);

s1 = SumBitTruePrecise(s, c, Word-length-1);

sum = TwosComplement2Decimal(s1 , Word-length);

Bit-true Model of LAM
• Part of code that calculate value of output neuron for an input pattern (propagation)

RawOutVect[i]+=W[i][j] * inVect[j]; // high precision arithmetic

// arithmetic with finite word-length

b1= W[i][j];

a1 = inVect[j];

b = Decimal2TwosComplement(b1, Word-length-1);

a = Decimal2TwosComplement(a1, Word-length-1);

c = mulBitTruePrecise(a, b, Word-length-1)

s = Decimal2TwosComplement(RawOutVect[i], Word-length-1);

s1 = SumBitTruePrecise(s, c, Word-length-1);

RawOutVect[i] = TwosComplement2Decimal(s1 , Word-length

Bit-true Model of BAM

//High precision Arithmetic

Sum += To->Weight[i][j] * From->Output[j];

//Finit precision Arithmetic

a1 = To->Weight[i][j] ;

b1 = From->Output[j];

a = Decimal2TwosComplement(a1,Wordlength-1);

b = Decimal2TwosComplement(b1,Wordlength-1);

c = mulBitTruePrecise(a, b, Wordlength-1);

s = Decimal2TwosComplement(Sum,Wordlength-1);

s1 = SumBitTruePrecise(s, c, Wordlength-1);

Sum = TwosComplement2Decimal(s1 , Wordlength );

Input pattern for train network

Input test pattern

Output pattern for different Word-Length

4 bit

6 bit

7 bit

8 bit

32 bit

5 bit

Simulation result of Hopfield network

Input pattern for train network

Input test patterns

Output patterns for WL = 5 bit

Output patterns for WL = 6 bit

Output patterns for WL = 7 bit

Output patterns for WL = 32 bit

Simulation result of LAM network

"TINA “ "6843726"

"ANTJE“ "8034673"

" LISA " "7260915"

input test pattern

"TANE "

"ANTJE"

"RISE "

Output for WL=32 bit

TINA -> | TINA -> 6843726

ANTJE -> | ANTJE -> 8034673

LISA -> | LISA -> 7260915

6843726 -> | 6843726 -> TINA

8034673 -> | 8034673 -> ANTJE

7260915 -> | 7260915 -> LISA

TANE -> | TINA -> 6843726

ANTJE -> | ANTJE -> 8034673

RISE -> | DIVA -> 6060737

Simulation result of BAM network

Output for WL=2 bit

TINA @ -> | TINA @ -> FENHGKO?

[email protected] -> | &165:? -> _+87&9)@

LISA @ -> | LISA @ -> FENHGKO?

[email protected] -> | [email protected] -> ^L^;GI

[email protected] -> | [email protected] -> &165:?

[email protected] -> | [email protected] -> &165:?

TANE @ -> | TANE @ -> FENHGKO?

[email protected] -> | [email protected] -> H0(@=^/5

RISE @ -> | RISE @ -> "#.\$6Z7,

Simulation result of BAM network

Output for WL=3 bit

TINA -> | TINA -> 8034673

ANTJE -> | TINA -> 8034673

LISA -> | TINA -> 8034673

6843726 -> | 6060737 -> DIVA

8034673 -> | 8034673 -> TINA

7260915 -> | 8034673 -> TINA

TANE -> | TINA -> 8034673

ANTJE -> | +61>_? -> GOLKIHL?

RISE -> | TINA -> 8034673

Simulation result of BAM network

Output for WL=8 bit

TINA -> | TINA -> 6843726

ANTJE -> | ANTJE -> 8034673

LISA -> | LISA -> 7260915

6843726 -> | 6843726 -> TINA

8034673 -> | 8034673 -> ANTJE

7260915 -> | 7260915 -> LISA

TANE -> | TINA -> 6843726

ANTJE -> | ANTJE -> 8034673

RISE -> | DIVA -> 6060737

Simulation result of BAM network

References

[1] A.S. Pandya, “Pattern Recognition with Neural network using C++ ,” , 2nd ed. vol. 3, J. New York: IEEE PRESS.

[2] p.Moerland, E. Fiesler “Neural Network Adaptation for Hardware Implementation”, Handbook of Neural Computation. JAN 97