- 394 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about 'Landscapes of the brain and mind' - mike_john

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Wan Ahmad Tajuddin Wan Abdullah*

Complex Systems Group

Department of Physics

Universiti Malaya

50603 Kuala Lumpur

*http://fizik.um.edu.my/cgi-bin/hitkat?wat

3rd MPSGC

KUALA LUMPUR 2007

Landscapes of the brain and mindKUALA LUMPUR 2007

The Little-Hopfield Neural Network Model

McCulloch-Pitts neurons:

- Binary: Vi = 0,1
- Summed inputs: hi = Σj TijVj
- Theshold: Vi→H(hi-Ui)

KUALA LUMPUR 2007

The Little-Hopfield Neural Network Model

Network: N coupled nonlinear equations –

Vi(t+τ) = H(ΣjTijVj(t)-Ui)

solve simultaneously!

KUALA LUMPUR 2007

The Little-Hopfield Neural Network Model

If

- exchange symmetry in synaptic strengths Tij = Tji
- no self-interactions Tii = 0

dynamics understood in terms of a Lyapunov function

E = - ½ Σi Σj TijViVj+Σi UiVi

Look:

ΔE = - hiΔVi

monotone decreasing wrt neuron dynamics

cf conservative forces potential function

cf spin systems (bipolar neurons, Si = 1)

Configuration

3rd MPSGC

KUALA LUMPUR 2007

The Little-Hopfield Neural Network Model

\'energy\' landscape

gradient descent

energy minimum ≡ stable configurations

Energy

Configuration

KUALA LUMPUR 2007

The Little-Hopfield Neural Network Model

Combinatorial optimization –

- map combinatorial choices to neuron configuration
- map cost function to energy function
- obtain synaptic weights
- let network relax to minimum energy configuration

KUALA LUMPUR 2007

The Little-Hopfield Neural Network Model

Associative memory –

optimize which ‘image’ stored nearest to input ‘image’

(initial configuration)

Tij := Σ(r) (2Vi (r) - 1) (2Vj(r) - 1) Ui = 0

Cooper-Hopfield prescription

Check: E = - ½ Σi Σj Σ(r) (2Vi(r) - 1) (2Vj (r) - 1)ViVj

minimum when Vi~ Vi(r)

- spurious memories – local minima
- forgetting
- temperature – simulated annealing
- basins of attraction

KUALA LUMPUR 2007

Energy landscapes

Minima = stable states / solutions

Global minima = good solutions

Local minima = spurious states / solutions

Ruggedness (measured by e.g. correlations) = difficulty in finding solution

KUALA LUMPUR 2007

Logic Programming

Bird(x)Have_feathers(x),Fly(x).

x Bird if x Have_feathers and x Fly.

Fly(Tweety).

Tweety Fly.

Have_feathers(Tweety).

Tweety Have_feather.

Have_fur(Sylvester).

Sylvester Have_fur.

Bird(Tweety)

Horn clauses – at most 1 logical atom in consequent

KUALA LUMPUR 2007

Logic Programming on Little-Hopfield networks

Logic programming ~ minimization of “logical inconsistency”

A ← B, C. A v ¬B v ¬C

D ← B. D v ¬B

C ←. C

EP = ⅛(1 - SA) (1 + SB) (1 + SC)

+ ¼ (1 - SD) (1 + SB)

+ ½ (1 - SC)

3rd order bipolar neural network

E = - ⅓ ΣiΣjΣkJijk(3)SiSjSk - ½ Σi Σj Jij(2)SiSj- Σi Ji(1)Si

Si := sign(Σj ΣkJijk(3)SjSk + Σj Jij(2)Sj+ Ji(1) )

KUALA LUMPUR 2007

Logic Programming on Little-Hopfield networks

translate clauses in the logic program

- Boolean algebraic form.

- Derive a cost function that is associated with the negation of all the clauses

- Obtain the values of connection strengths by comparing the cost function with the energy function

- Let the neural networks evolve until minimum energy is reached.

- The neural states provide a solution interpretation for the logic program, and the truth of a ground atom in this interpretation may be checked

.

KUALA LUMPUR 2007

Logic Programming on Little-Hopfield networks

translate clauses in the logic program

- how rugged is landscape from logic programming?

Computersimulations:… [SS & WATWA]

– flat energy landscape no satisfiability problem

(all clauses can be satisfiedi.e. solution always guaranteed)

KUALA LUMPUR 2007

Satisfiability

In general, general CNF clauses(conjuctions of disjunctions)not necessarily satisfiable

- depends on number of atoms in disjunctions

number of disjunctions

number of distinct atoms

Exist phase transition from easily soluble problems to difficult problems

[Zecchina & Monasson]

KUALA LUMPUR 2007

Can \'knowledge\' may be associated with the energy landscape?

What is knowledge?

? How ingrained are logical rules in neural network –

“ingrainedness”:

GX←Y,Z = < E({Si} satisfying X←Y,Z) - E({Si} not satisfying X←Y,Z) >

Can this be related to landscape ruggedness e.g. correlations?

Download Presentation

Connecting to Server..