Human visual system neural network
Sponsored Links
This presentation is the property of its rightful owner.
1 / 37

Human Visual System Neural Network PowerPoint PPT Presentation


  • 114 Views
  • Uploaded on
  • Presentation posted in: General

Human Visual System Neural Network. Stamatios Cheirdaris, Dmitry Nikelshpur, Charles Tappert, Alexander Cipully, Roberto Rodriguez, Rohit Yalamanchi, Abou Damon, Stephanie Pierce-Jones, and Robert Zucker. The Visual System. Hubel and Wiesel 1981 Nobel Prize for work in early 1960s

Download Presentation

Human Visual System Neural Network

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Human Visual System Neural Network

Stamatios Cheirdaris, Dmitry Nikelshpur, Charles Tappert, Alexander Cipully, Roberto Rodriguez, Rohit Yalamanchi, Abou Damon, Stephanie Pierce-Jones, and Robert Zucker


The Visual System

  • Hubel and Wiesel

    • 1981 Nobel Prize for work in early 1960s

    • Cat’s visual cortex

      • cats anesthetized, eyes open with controlling muscles paralyzed to fix the stare in a specific direction

      • thin microelectrodes measure activity in individual cells

      • cells specifically sensitive to line of light at specific orientation

    • Key discovery – line and edge detectors in the visual cortex of mammals


The Study

  • Compare Two Neural Networks

    • One without vertical and horizontal line detectors

    • One with vertical and horizontal line detectors

  • Objective

    • Show that the neural network with line detectors is superior to the one without on the six vertical-horizontal line-segment letters E, F, H, I, L, T

  • Also, experiment with the full alphabet

    • Without line detectors


Uppercase 5x7 Bit-map AlphabetHorizontal-vertical line-segment letters are E, F, H, I, L, T


Neural Network Without Line Detectors


Neural Network SpecificationWithout Line Detectors

Layers

  • Input layer: 20x20 retina of binary units

  • Hidden layer: 50 units (other numbers explored)

  • Output layer: 6 units for letters E, F, H, I, L, T

    Weights

  • 20,000 (400x50) between input and hidden layer

  • 300 (50x6) between hidden and output layer

  • Total of 20,300 variable weights, no fixed weights


Neural Network With Line Detectors


Neural Network SpecificationWith Vertical and Horizontal Line Detectors

Layers

  • Input layer: 20x20 retina of binary units

  • 576 simple vertical and horizontal line detectors

  • 48 complex vertical and horizontal line detectors

  • Hidden layer: 50 units (other numbers explored)

  • Output layer: 6 units for letters E, F, H, I, L, T

    Weights

  • 6336 (576x11) fixed weights from input to simple detectors

  • 576 fixed weights from simple and complex detectors

  • 2400 (48x50) variable weights from complex detectors to hidden layer

  • 300 (50x6) variable weights from hidden to output layer

  • Total of 6912 fixed weights and 2700 variable weights


Vertical Line Detectors

DETECTORS OVERLAP COVERING EACH POSSIBLE RETINAL POSITION FOR A TOTAL OF288 (18x16) VERTICAL LINE DETECTORS

EACH DETECTOR HAS 5 EXCITATORY AND 6 INHIBITORY INPUTS (11 FIXED WEIGHTS),

WITH A THRESHOLD OF 3

Horizontal Line Detectors are Similar


Retina Image – Letter “E” in Upper Left Area

Region of possible upper-left corners is shown in green.


Retina Image – Letter “E” in Upper Right Area

Region of possible upper-left corners is shown in green.


Retina Image – Letter “E” in Lower Right Area

Region of possible upper-left corners is shown in green.


Example of Vertical Line Detectoron Line Segment of “E” – Detector Activated


Example of Shifted Vertical Line Detectoron Letter “E” – Detector Not Activated


Example of Shifted Vertical Line Detectoron Letter “E” – Detector Not Activated


24 Vertical Complex Line Detector RegionsAny Simple Line Detector in a RegionActivates the Complex Line Detector


24 Horizontal Complex Line Detector RegionsAny Simple Line Detector in a RegionActivates the Complex Line Detector


The Corresponding 48 Complex Horizontal and Vertical Line Detectors

Complex Horizontal and Vertical Line Detector Matrix


Experiments

  • Experiment 1

    • 6 Line-Segment Letters without Line Detectors

    • 26 Letters without Line Detectors

  • Experiment 2

    • 6 Line-Segment Letters with Line Detectors


Experimental Parameter Combinations

  • Epochs:

    • 50

    • 100

    • 200

    • 400

    • 800

    • 1600

    • 32000 (occasionally)

  • Hidden Layer Units:

    • 10

    • 18*

    • 50

    • 100

    • 200 *

    • 300*

    • 500*

      *Selected cases

  • Noise:

    • 0%

    • 2%

    • 5%

    • 10%

    • 15%

    • 20%


Simulation View – Peltarion’s Synapse Product

Experiment 1


Simulation Settings Experiment 2 – Line Detectors6 Line-Segment Letters: E, F, H, I, L, T

  • Function Layer:

    • Function: Tanh Sigmoid

    • Forward Rule: No rule

    • Back Rule: Levenberg-Marquardt

    • Propagator: Function Layer

  • Weight Layer:

    • Forward Rule: No rule

    • Back Rule: Levenberg-Marquardt

    • Propagator: Weight Layer


Exp 1 – 6 Letters, No Line Detectors – 35.42% Accuracy

6 Letters: no line detectors

50 Hidden layer units

50 Epochs

0% noise


Exp 1 – 6 Letters, No Line Detectors – 36.25% Accuracy

6 Letters: no line detectors

50 Hidden layer units

1600 Epochs

0% noise


Exp 2 – 6 Letters, With Line Detectors – 67.5% Accuracy

6 Letters: with line detectors

50 Hidden layer units

50 Epochs

0% noise


Exp 2 – 6 Letters, With Line Detectors – 67.5% Accuracy

6 Letters: with line detectors

50 Hidden layer units

50 Epochs

0% noise


Exp 1 – 6 Letters, No Line Detectors – 27.69% Accuracy

6 Letters: no line detectors

10 Hidden layer units

1600 Epochs

0% noise


Exp 1 – 6 Letters, With Line Detectors – 82.5% Accuracy

6 Letters: with line detectors

10 Hidden layer units

1600 Epochs

0% noise


Exp 2 – 6 Letters, With Line Detectors – 82.5% Accuracy

6 Letters: with line detectors

10 Hidden layer units

1600 Epochs

0% noise


Exp 1 – 26 Letters, No Line Detectors – 27.69% Accuracy

26 Letters: no line detectors

50 Hidden layer units

1600 Epochs

0% noise


Exp 1 – 26 Letters, No Line Detectors – 27.69% Accuracy

26 Letters: no line detectors

50 Hidden layer units

1600 Epochs

0% noise


Exp 1 – 6 Letters, No Line DetectorsEpochs versus Percent Added Noise


Exp 1 – 26 Letters, No Line DetectorsEpochs versus Percent Added Noise


Exp 2 – 6 Letters, With Line DetectorsEpochs versus Percent Added Noise


Comparison of Line / No-Line Detector Networks6 letters, 50 hidden layer units, 1600 epochs, no noise


Main Conclusion

  • Character recognition performance and efficiency of the neural network using Hubel-Wiesel-like line detectors in the early layers is superior to that of a network using adjustable weights directly from the retina

  • Recognition performance more than doubled

  • Line detector network was much more efficient

    • order of magnitude fewer variable weights and half as many total weights

    • training time decrease of several orders of magnitude


Additional Conclusions

  • Increasing the number of hidden layer units does not translate to better accuracy, it actually reduces it.

  • Increasing the number of epochs increased the accuracy but not always

  • For Experiment 2 (6 letters with line detectors) we can achieve perfect training accuracy and very good validation accuracy

  • Training time varied from a few minutes to many hours with Experiment 1 – 26 Letters taking the longest out of all, i.e. for 500 hidden layer units it required up to 9 hours.

  • When noise is added to the retina image the accuracy of the system drops significantly, even for Experiment 2 with the line detectors


  • Login