ee459 neural networks n.
Skip this Video
Loading SlideShow in 5 Seconds..
EE459 Neural Networks PowerPoint Presentation
Download Presentation
EE459 Neural Networks

Loading in 2 Seconds...

play fullscreen
1 / 23

EE459 Neural Networks - PowerPoint PPT Presentation

  • Uploaded on

EE459 Neural Networks. Examples of using Neural Networks. Kasin Prakobwaitayakit Department of Electrical Engineering Chiangmai University. Two examples of real life applications of neural networks for pattern classification: RBF networks for face recognition

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'EE459 Neural Networks' - jaimie

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
ee459 neural networks
EE459 Neural Networks

Examples of using Neural Networks

Kasin Prakobwaitayakit

Department of Electrical Engineering

Chiangmai University

Two examples of real life applications of neural networks for pattern classification:

RBF networks for face recognition

FF networks for handwritten recognition

face recognition
The problem:

Face recognition of persons of a known group in an indoor environment.

The approach:

Learn face classes over a wide range of poses using an RBF network.

Sussex database (university of Sussex)

100 images of 10 people (8-bit grayscale, resolution 384 x 287)

for each individual, 10 images of head in different pose from face-on to profile

Designed to asses performance of face recognition techniques when pose variations occur

datasets sussex
Datasets (Sussex)

All ten images for classes 0-3 from the Sussex database, nose-centred and subsampled to 25x25 before preprocessing

approach face unit rbf
A face recognition unit RBF neural networks is trained to recognize a single person.

Training uses examples of images of the person to be recognized as positive evidence, together with selected confusable images of other people as negative evidence.

Approach: Face unit RBF
network architecture
Input layer contains 25*25 inputs which represent the pixel intensities (normalized) of an image.

Hidden layer contains p+a neurons:

p hidden pro neurons (receptors for positive evidence)

a hidden anti neurons (receptors for negative evidence)

Output layer contains two neurons:

One for the particular person.

One for all the others.

The output is discarded if the absolute difference of the two output neurons is smaller than a parameter R.

Network Architecture
rbf architecture for one face recognition
RBF Architecture for one face recognition

Output units



RBF units



Input units

hidden layer
Hidden nodes can be:

Pro neurons: Evidence for that person.

Anti neurons: Negative evidence.

The number of pro neurons is equal to the positive examples of the training set. For each pro neuron there is either one or two anti neurons.

Hidden neuron model: Gaussian RBF function.

Hidden Layer
the parameters

of a pro neuron: the corresponding positive example

of an anti neuron: the negative example which is most similar to the corresponding pro neuron, with respect to the Euclidean distance.

Spread: average distance of the center vector from all other centers. If , h hidden nodes, H total number of hidden nodes then:

Weights: determined using the pseudo-inverse method.

A RBF network with 6 pro neurons, 12 anti neurons, and R equal to 0.3, discarded 23 pro cent of the images of the test set and classified correctly 96 pro cent of the non discarded images.

The Parameters
dataset and nn architecture
7,291 for training

9,298 examples used

2,007for testing

Both training and test set contain ambiguous and unclassifiable examples.

256input neurons. (16x16)

10 output neurons. (each represents a numeral)

A feedforward NN with 3 hidden layers is used.

Dataset and NN architecture
the idea
Shared weights: all neurons in a feature share the same weights (but not the biases). In this way all neurons detect the same feature at different positions in the input image. The detected features are combined to reach shift-invariant feature detection.

This is combined with layer implementing subsampling to decrease the resolution and the sensitivity to distorsions.

The Idea
convolutional nn
If a neuron in the feature map fires, this corresponds to a match with the template.

Neurons of the feature map react to the same pattern at different positions in the input image.

For neurons in the feature map that are one neuron apart (in the matrix representation of the feature map) their templates in the input image are two pixels apart. Thus the input image is undersampled, and some position information is eliminated.

A similar 2-to-1 undersampling occurs as one goes from H1 to H2. The rationale is that although high resolution may be needed to detect a feature, its exact position need not be determined at equally high precision.

Convolutional NN
Input layer: 256 neurons with input values in range [-1, 1].

Hidden layer H1: consists of 12 feature maps H1.1, … , H1.12.

Feature map:

It consists of 8x8 neurons.

Each neuron in the feature map has the same incoming weights , but is connected to a square at a unique position in the input image. This square is called a template.

Hidden layer H2: consists of 12 sub-sampling maps H2.1, … , H2.12.

Sub-sampling map:

Consists of4x4neurons.

Each neuron of the sub-sampling map is connected to a 5x5 square of H1.j, for each j in 8 of the 12 feature maps.

All neurons of the sub-sampling map share the same 25 weights.

Hidden layer H3:

Consists of30 neurons.

H3 is completely connected to the sub-sampling layer (H2).

Output layer: consists of 10 neurons, numbered 0, … , 9 and the neuron with the highest activation value is chosen. The digit recognized is equal to the cell number.

A Dutch master thesis on Le Cun shared weights NN:

master thesis of D. de Ridder:

“shared weights NN’s in image anlyses”, 1996

Backpropagation was used to learn the weights.

The hyperbolic tangent was chosen as neuron model.

After training the number of misclassified patterns was 10 on the training set and 102 on the test set.