How to make a computer think for you
Download
1 / 42

How to Make a Computer Think for You - PowerPoint PPT Presentation


  • 355 Views
  • Updated On :

How to Make a Computer Think for You Jeff Knisley, The Institute for Quantitative Biology, East Tennessee State University ALABAMA MAA STATE DINNER AND LECTURE, Feb, 2006 Soma Dendrites Synapses Axon This is a Neuron Signals Propagate from Dendrites to Soma Signals Decay at

Related searches for How to Make a Computer Think for You

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'How to Make a Computer Think for You' - paul2


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
How to make a computer think for you l.jpg

How to Make a Computer Think for You

Jeff Knisley, The Institute for Quantitative Biology, East Tennessee State University

ALABAMA MAA STATE DINNER AND LECTURE, Feb, 2006


This is a neuron l.jpg

Soma

Dendrites

Synapses

Axon

This is a Neuron

Signals Propagate from

Dendrites to Soma

Signals Decay at

Soma if below a

Certain threshold


Signals may arrive close together l.jpg
Signals May Arrive Close Together

If threshold exceeded,

then neuron “fires,”

sending a signal

along its axon.



Artificial neural network ann l.jpg
Artificial Neural Network (ANN)

  • Made of artificial neurons, each of which

    • Sums inputs from other neurons

    • Compares sum to threshold

    • Sends signal to other neurons if above threshold

  • Synapses have weights

    • Model relative ion collections

    • Model efficacy (strength) of synapse


Artificial neuron l.jpg
Artificial Neuron

Nonlinear

firing function

.

.

.



Hopfield network l.jpg

Choose ith neuron at random and calculate its new state

Hopfield Network

Imagine

Complete

Connectivity

with weights

wij between ith

and jth neurons

Blue = 1

White = 0


Energy l.jpg
Energy

Define the energy to be

Theorem: If the weights are symmetric, then the Energy

decreases each time a neuron fires.


Applications l.jpg
Applications:

  • Handwriting Recognition

    • http://faculty.etsu.edu/knisleyj/neural/neuralnet3.htm

  • Universal Classifier

    • http://faculty.etsu.edu/knisleyj/neural/neuralnet4.htm

  • Expert Systems

    • Rule-based rather than sequential programs

    • Air traffic control, Industrial Controls

  • Robotics

    • Usually using a 3-layer network


3 layer neural network l.jpg
3 Layer Neural Network

The output layer may

consist of a single

neuron

Output

Input

Hidden

(is usually much larger)


Neural nets can think l.jpg
Neural Nets can “Think”

  • A Neural Network can “think” for itself

    • Can be trained (programmed) to make decisions

    • Can be trained to classify information

This tiny 3-Dimensional

Artificial Neural Network,

modeled after neural networks

in the human brain, is helping

machines better visualize

their surroundings.

http://www.nasa.gov/vision/universe/roboticexplorers/robots_like_people.html


The mars rovers l.jpg
The Mars Rovers

  • Must choose where to explore

    • Programmed to avoid “rough” terrain

    • Programmed to choose “smooth” terrain

  • ANN decides between “rough” and “smooth”

    • “rough” and “smooth” are ambiguous

    • Programmingis by means of many “examples”(lessons)


Illustration colors terrains l.jpg

1

2

3

4

Robot

5

6

7

8

Illustration: Colors = Terrains

  • As a robot moves, it defines 8 squares of size A that define directions it can move in

    • It should avoid red (rough) terrain

    • It should prefer green (smooth) terrain

    • It should be indifferent to blue (normal) terrain

  • It is impossible to program every possible shade and variation

    • Instead, a neural network is constructed

    • Terrain Block – Color class input/output patterns are used to train the network


Train ann to classify colors l.jpg

“Houston,

we have a

problem!”

0.1

0

0

0.9

0.1

0

Train ANN to Classify Colors

Training Set

Input OutputTerrain <R,G,B>

Red

<1,0,0>

Green

<0,1,0>

Blue

ColorClass

<0,0,1>

TerrainExamples

<1,0,0>

Hidden

<0,0,0>


Ann s can also think for us l.jpg
ANN’s Can Also Think for Us

  • Mars Rovers do what Humans can do better

    • They do not “learn” on their own

    • They are taught by Humans who could make the same or better decisions in their place

    • They can use their “learning” independently

  • What about problems Humans can’t solve at all

    • Neural Networks can be used as savants dedicated to a single problem too complex for humans to decipher

    • Examples

      • Agent-Based Modeling

      • RNA, Protein Structures, DNA analysis

      • Data Mining


Division of labor in wasps nests l.jpg
Division of Labor in Wasps Nests

  • A Wasp can alternate between laborer, water forager, and pulp forager

    • Hypothesis: Individual wasps choose roles so that total water in the nest reaches equilibrium

    • Hypothesis: Pulp production is maximized when total water in the nest is stable

  • Limited confirmation of Hypotheses

    • (Karsai and Wenzel, 2002) System of 3 ode’s

    • (Karsai, *Phillips, and Knisley, 2005) Computer Simulation


Ann wasp societies l.jpg
ANN Wasp Societies

  • Problems with current models

    • Role assigned randomly w.r.t. a Weibull distribution

    • Parameters selected so model “works”

  • ANN: Role is a decision of each wasp

    • A wasp is a special type of Artificial Neuron

      • Wasp decides its own role

      • Wasp learns to make good choices

    • System is deterministic yet unpredictable


Slide19 l.jpg

“I’m empty, I need water!”

“No luck, so I’m a water forager.”


Slide20 l.jpg

“I’m so good at getting water, I think I should go forage for pulp.”

“I have water!”


Division of labor by choice l.jpg
Division of Labor forage for pulp.”by choice?

  • Numbers of foragers goes up and down

  • But water level in nest becomes stable


Data mining l.jpg
Data Mining forage for pulp.”

  • We often consider very large data sets

    • Microarrays contain about 20,000 data points

    • Typical studies use 70 – 100 microarrays

    • Most of the data is not relevant

  • ANN’s can be trained to find hidden patterns

    • Input layer = Genes

    • Network is trained repeatedly with microarrays collected in various physiological states

    • ANN’s predict which genes are responsible for a given state


Ann s in data mining l.jpg
ANN’s in Data Mining forage for pulp.”

  • Each neuron acts as a “linear classifier”

  • Competition among neurons via nonlinear firing function = “local linear classifying”

  • Method for Genes:

    • Train Network until it can classify between control and experimental groups

    • Eliminating weights sufficiently close to 0 does not change local classification scheme

  • First results obtained with a Perceptron ANN


Simple perceptron model l.jpg
Simple Perceptron Model forage for pulp.”

x1 = Gene 1

w1

x2 = Gene 2

w2

wn

xn = Gene n

The output is the “physiological state” due to the relative gene expression levels used as inputs.


Simple perceptron model25 l.jpg
Simple Perceptron Model forage for pulp.”

  • Features

    • The wi measure gene “significance”

    • Detects genes across n samples & references

    • Ref: Artificial Neural Networks for Reducing the Dimensionality of Gene Expression Data, A. Narayanan, et al. 2004.

  • Drawbacks

    • The Perceptron is a globaly linear classifier (i.e., only classifies linearly separable data)

    • We are now using a more sophisticated model


Linearly separable data l.jpg
Linearly Separable Data forage for pulp.”

Separation using Hyperplanes



How do we select w s l.jpg
How do we select w’s forage for pulp.”

  • Define an energy function

    • x1,…,xn are inputs, y = output

    • t1,…,tm are the outputs associated with the patterns to be “learned”

    • y = s( S wixi - q) for a perceptron

    • Key: Neural networks minimize energy


Back propagation l.jpg
Back Propagation forage for pulp.”

  • Minimize the Energy Function

    • Choose wi so that

    • In practice, this is hard

  • Back Propagation: s = s(1-s)

    • For each pattern ti

      • Feed Forward and Calculate E

      • Increment weights using a delta rule:

    • Repeat until E is sufficiently close to 0


Perceptron for microarray datamining l.jpg
Perceptron for Microarray DataMining forage for pulp.”

  • Remove % of genes with synaptic weights that are close to 0

  • Create ANN classifier on reduced arrays

  • Repeat 1 and 2 until only the genes that most influence the classifer problem remain

    Remaining genes are most important in classifying experimentals versus controls


Functional viewpoint l.jpg
Functional Viewpoint forage for pulp.”

  • ANN is a mapping f: Rn→ R

    • Can we train perceptron so that f(x1,…,xn) =1 if x vector is from a control and f(x1,…,xn) =0 if x is from an experimental?

    • Answer: Yes if data can be linearly separated, but no otherwise unless we use better ANN’s!

  • General ANN’s also have problems

    • Spurious states: (sometimes ANN’s get the wrong answer)

    • Hard Margins: Training set must be perfect


Multilayer network l.jpg
Multilayer Network forage for pulp.”

.

.

.

.

.

.


How do we select w s33 l.jpg
How do we select w’s forage for pulp.”

  • Define an energy function

    • t vectors are the information to be “learned”

    • Neural networks minimize energy

    • The “information” in the network is equivalent to the minima of the total squared energy function


Back propagation34 l.jpg
Back Propagation forage for pulp.”

  • Minimize the Energy Function

    • Choose wj and aj so that

    • In practice, this is hard

  • Back Propagation with cont. sigmoidal

    • Feed Forward and Calculate E

    • Modify weights using a d rule

    • Repeat until E is sufficiently close to 0


Ann as classifer l.jpg
ANN as Classifer forage for pulp.”

  • (Cybenko) For any e>0, the function f(x1,…,xn) =1 if x vector is from a control and f(x1,…,xn) =0 if x is from an experimental can be approximated to within e by a multilayer neural network.

  • The weights no longer have the one-to-one correspondence to genes, so we test significance using Monte Carlo techniques.


Ann and monte carlo methods l.jpg
ANN and Monte Carlo Methods forage for pulp.”

  • Monte Carlo methods have been a big success story with ANN’s

    • Error estimates with network predictions

    • ANN’s are very fast in the forward direction

  • Example: ANN+MC implement and outperform Kalman Filters (recursive linear filters used in Navigation and elsewhere)(De Freitas J. F. G., et. al., 2000)


Recall multilayer network l.jpg
Recall: Multilayer Network forage for pulp.”

.

.

.

.

.

.

ajcorrespond to genes,

but do not directly depend

on a single gene.

N Genes

N node

Hidden

Layer


Na ve monte carlo ann method l.jpg
Naïve Monte Carlo ANN Method forage for pulp.”

  • Randomly choose subset S of genes

  • Train using Back Propagation

  • Prune based on values of wj (or aj , or both)

  • Repeat 2-3 until a small subset of S remains

  • Increase “count” of genes in small subset

  • Repeat 1-5 until each gene has 95% probability of appearing at least some minimum number of times in a subset

  • Most frequent genes are the predicted


Additional considerations l.jpg
Additional Considerations forage for pulp.”

  • If a gene is up-regulated or down-regulated for a certain condition, then put it into a subset in step 1 with probability 1.

  • This is a simple-minded Bayesian method. Bayesian analysis can make it much better.

  • Algorithm distributes naturally across a multi-processor cluster or machine

    • Choose the subsets first

    • Distribute subsets to different machines

    • Tabulate the results from all the machines


Summary l.jpg
Summary forage for pulp.”

  • ANN’s are designed to make decisions in similar fashion to how we make decisions

    • In this way, they can “think” for themselves

    • Can be considered supplements to existing hardware and software tools

  • The ability of ANN’s to make decisions allows them to think for us as well!

    • They can find patterns in large data sets that we humans would likely never uncover

    • They can “think” for days/months on end


Any questions l.jpg

Any Questions? forage for pulp.”


References l.jpg
References forage for pulp.”

Cybenko, G. Approximation by Superpositions of a sigmoidal function, Mathematics of Control, Signals, and Systems, 2(4),1989, p. 303-314.

De Freitas J. F. G., et. al. Sequential Monte Carlo Methods To Train Neural Network Models. Neural Computation, Volume 12, Number 4, 1 April 2000, pp. 955-993(39)

L. Glenn and J. Knisley, Solutions for Transients in Arbitrarily Branching and Tapering Cables, Modeling in the Neurosciences: From Biological Systems to Neuromimetic Robotics, ed. Lindsay, R., R. Poznanski, G.N.Reeke, J.R. Rosenberg, and O.Sporns, CRC Press, London, 2004.

A. Narayan, et. al Artificial Neural Networks for Reducing the Dimensionality of Gene Expression Data. Neurocomputing, 2004.