Landscapes of the brain and mind
Download
1 / 15

Landscapes of the brain and mind - PowerPoint PPT Presentation


  • 394 Views
  • Updated On :

Wan Ahmad Tajuddin Wan Abdullah* Complex Systems Group Department of Physics Universiti Malaya 50603 Kuala Lumpur *http://fizik.um.edu.my/cgi-bin/hitkat?wat. 3 rd MPSGC KUALA LUMPUR 2007. Landscapes of the brain and mind. 3 rd MPSGC KUALA LUMPUR 2007.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Landscapes of the brain and mind' - mike_john


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Landscapes of the brain and mind l.jpg

Wan Ahmad Tajuddin Wan Abdullah*

Complex Systems Group

Department of Physics

Universiti Malaya

50603 Kuala Lumpur

*http://fizik.um.edu.my/cgi-bin/hitkat?wat

3rd MPSGC

KUALA LUMPUR 2007

Landscapes of the brain and mind


Slide2 l.jpg

3rd MPSGC

KUALA LUMPUR 2007

The Little-Hopfield Neural Network Model

McCulloch-Pitts neurons:

  • Binary: Vi = 0,1

  • Summed inputs: hi = Σj TijVj

  • Theshold: Vi→H(hi-Ui)


Slide3 l.jpg

3rd MPSGC

KUALA LUMPUR 2007

The Little-Hopfield Neural Network Model

Network: N coupled nonlinear equations –

Vi(t+τ) = H(ΣjTijVj(t)-Ui)

solve simultaneously!


Slide4 l.jpg

3rd MPSGC

KUALA LUMPUR 2007

The Little-Hopfield Neural Network Model

If

  • exchange symmetry in synaptic strengths Tij = Tji

  • no self-interactions Tii = 0

    dynamics understood in terms of a Lyapunov function

    E = - ½ Σi Σj TijViVj+Σi UiVi

    Look:

    ΔE = - hiΔVi

    monotone decreasing wrt neuron dynamics

    cf conservative forces  potential function

    cf spin systems (bipolar neurons, Si = 1)


Slide5 l.jpg

Energy

Configuration

3rd MPSGC

KUALA LUMPUR 2007

The Little-Hopfield Neural Network Model

'energy' landscape

gradient descent

energy minimum ≡ stable configurations

Energy

Configuration


Slide6 l.jpg

3rd MPSGC

KUALA LUMPUR 2007

The Little-Hopfield Neural Network Model

Combinatorial optimization –

  • map combinatorial choices to neuron configuration

  • map cost function to energy function

  • obtain synaptic weights

  • let network relax to minimum energy configuration


Slide7 l.jpg

3rd MPSGC

KUALA LUMPUR 2007

The Little-Hopfield Neural Network Model

Associative memory –

optimize which ‘image’ stored nearest to input ‘image’

(initial configuration)

Tij := Σ(r) (2Vi (r) - 1) (2Vj(r) - 1) Ui = 0

Cooper-Hopfield prescription

Check: E = - ½ Σi Σj Σ(r) (2Vi(r) - 1) (2Vj (r) - 1)ViVj

minimum when Vi~ Vi(r)

  • spurious memories – local minima

  • forgetting

  • temperature – simulated annealing

  • basins of attraction


Slide8 l.jpg

3rd MPSGC

KUALA LUMPUR 2007

Energy landscapes

Minima = stable states / solutions

Global minima = good solutions

Local minima = spurious states / solutions

Ruggedness (measured by e.g. correlations) = difficulty in finding solution


Slide9 l.jpg

3rd MPSGC

KUALA LUMPUR 2007

Logic Programming

Bird(x)Have_feathers(x),Fly(x).

x Bird if x Have_feathers and x Fly.

Fly(Tweety).

Tweety Fly.

Have_feathers(Tweety).

Tweety Have_feather.

Have_fur(Sylvester).

Sylvester Have_fur.

Bird(Tweety)

Horn clauses – at most 1 logical atom in consequent


Slide10 l.jpg

3rd MPSGC

KUALA LUMPUR 2007

Logic Programming on Little-Hopfield networks

Logic programming ~ minimization of “logical inconsistency”

A ← B, C. A v ¬B v ¬C

D ← B. D v ¬B

C ←. C

EP = ⅛(1 - SA) (1 + SB) (1 + SC)

+ ¼ (1 - SD) (1 + SB)

+ ½ (1 - SC)

3rd order bipolar neural network

E = - ⅓ ΣiΣjΣkJijk(3)SiSjSk - ½ Σi Σj Jij(2)SiSj- Σi Ji(1)Si

Si := sign(Σj ΣkJijk(3)SjSk + Σj Jij(2)Sj+ Ji(1) )


Slide11 l.jpg

3rd MPSGC

KUALA LUMPUR 2007

Logic Programming on Little-Hopfield networks

translate clauses in the logic program

  • Boolean algebraic form.

  • Derive a cost function that is associated with the negation of all the clauses

  • Obtain the values of connection strengths by comparing the cost function with the energy function

  • Let the neural networks evolve until minimum energy is reached.

  • The neural states provide a solution interpretation for the logic program, and the truth of a ground atom in this interpretation may be checked

    .


Slide12 l.jpg

3rd MPSGC

KUALA LUMPUR 2007

Logic Programming on Little-Hopfield networks

translate clauses in the logic program

  • how rugged is landscape from logic programming?

Computersimulations:… [SS & WATWA]

– flat energy landscape  no satisfiability problem

(all clauses can be satisfiedi.e. solution always guaranteed)


Slide13 l.jpg

3rd MPSGC

KUALA LUMPUR 2007

Satisfiability

In general, general CNF clauses(conjuctions of disjunctions)not necessarily satisfiable

- depends on number of atoms in disjunctions

number of disjunctions

number of distinct atoms

Exist phase transition from easily soluble problems to difficult problems

[Zecchina & Monasson]


Slide14 l.jpg

3rd MPSGC

KUALA LUMPUR 2007

Can 'knowledge' may be associated with the energy landscape?

What is knowledge?

? How ingrained are logical rules in neural network –

“ingrainedness”:

GX←Y,Z = < E({Si} satisfying X←Y,Z) - E({Si} not satisfying X←Y,Z) >

Can this be related to landscape ruggedness e.g. correlations?


Slide15 l.jpg

3rd MPSGC

KUALA LUMPUR 2007

Terima kasih