landscapes of the brain and mind
Download
Skip this Video
Download Presentation
Landscapes of the brain and mind

Loading in 2 Seconds...

play fullscreen
1 / 15

Landscapes of the brain and mind - PowerPoint PPT Presentation


  • 394 Views
  • Uploaded on

Wan Ahmad Tajuddin Wan Abdullah* Complex Systems Group Department of Physics Universiti Malaya 50603 Kuala Lumpur *http://fizik.um.edu.my/cgi-bin/hitkat?wat. 3 rd MPSGC KUALA LUMPUR 2007. Landscapes of the brain and mind. 3 rd MPSGC KUALA LUMPUR 2007.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Landscapes of the brain and mind' - mike_john


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
landscapes of the brain and mind

Wan Ahmad Tajuddin Wan Abdullah*

Complex Systems Group

Department of Physics

Universiti Malaya

50603 Kuala Lumpur

*http://fizik.um.edu.my/cgi-bin/hitkat?wat

3rd MPSGC

KUALA LUMPUR 2007

Landscapes of the brain and mind
slide2

3rd MPSGC

KUALA LUMPUR 2007

The Little-Hopfield Neural Network Model

McCulloch-Pitts neurons:

  • Binary: Vi = 0,1
  • Summed inputs: hi = Σj TijVj
  • Theshold: Vi→H(hi-Ui)
slide3

3rd MPSGC

KUALA LUMPUR 2007

The Little-Hopfield Neural Network Model

Network: N coupled nonlinear equations –

Vi(t+τ) = H(ΣjTijVj(t)-Ui)

solve simultaneously!

slide4

3rd MPSGC

KUALA LUMPUR 2007

The Little-Hopfield Neural Network Model

If

  • exchange symmetry in synaptic strengths Tij = Tji
  • no self-interactions Tii = 0

dynamics understood in terms of a Lyapunov function

E = - ½ Σi Σj TijViVj+Σi UiVi

Look:

ΔE = - hiΔVi

monotone decreasing wrt neuron dynamics

cf conservative forces  potential function

cf spin systems (bipolar neurons, Si = 1)

slide5

Energy

Configuration

3rd MPSGC

KUALA LUMPUR 2007

The Little-Hopfield Neural Network Model

\'energy\' landscape

gradient descent

energy minimum ≡ stable configurations

Energy

Configuration

slide6

3rd MPSGC

KUALA LUMPUR 2007

The Little-Hopfield Neural Network Model

Combinatorial optimization –

  • map combinatorial choices to neuron configuration
  • map cost function to energy function
  • obtain synaptic weights
  • let network relax to minimum energy configuration
slide7

3rd MPSGC

KUALA LUMPUR 2007

The Little-Hopfield Neural Network Model

Associative memory –

optimize which ‘image’ stored nearest to input ‘image’

(initial configuration)

Tij := Σ(r) (2Vi (r) - 1) (2Vj(r) - 1) Ui = 0

Cooper-Hopfield prescription

Check: E = - ½ Σi Σj Σ(r) (2Vi(r) - 1) (2Vj (r) - 1)ViVj

minimum when Vi~ Vi(r)

  • spurious memories – local minima
  • forgetting
  • temperature – simulated annealing
  • basins of attraction
slide8

3rd MPSGC

KUALA LUMPUR 2007

Energy landscapes

Minima = stable states / solutions

Global minima = good solutions

Local minima = spurious states / solutions

Ruggedness (measured by e.g. correlations) = difficulty in finding solution

slide9

3rd MPSGC

KUALA LUMPUR 2007

Logic Programming

Bird(x)Have_feathers(x),Fly(x).

x Bird if x Have_feathers and x Fly.

Fly(Tweety).

Tweety Fly.

Have_feathers(Tweety).

Tweety Have_feather.

Have_fur(Sylvester).

Sylvester Have_fur.

Bird(Tweety)

Horn clauses – at most 1 logical atom in consequent

slide10

3rd MPSGC

KUALA LUMPUR 2007

Logic Programming on Little-Hopfield networks

Logic programming ~ minimization of “logical inconsistency”

A ← B, C. A v ¬B v ¬C

D ← B. D v ¬B

C ←. C

EP = ⅛(1 - SA) (1 + SB) (1 + SC)

+ ¼ (1 - SD) (1 + SB)

+ ½ (1 - SC)

3rd order bipolar neural network

E = - ⅓ ΣiΣjΣkJijk(3)SiSjSk - ½ Σi Σj Jij(2)SiSj- Σi Ji(1)Si

Si := sign(Σj ΣkJijk(3)SjSk + Σj Jij(2)Sj+ Ji(1) )

slide11

3rd MPSGC

KUALA LUMPUR 2007

Logic Programming on Little-Hopfield networks

translate clauses in the logic program

  • Boolean algebraic form.
  • Derive a cost function that is associated with the negation of all the clauses
  • Obtain the values of connection strengths by comparing the cost function with the energy function
  • Let the neural networks evolve until minimum energy is reached.
  • The neural states provide a solution interpretation for the logic program, and the truth of a ground atom in this interpretation may be checked

.

slide12

3rd MPSGC

KUALA LUMPUR 2007

Logic Programming on Little-Hopfield networks

translate clauses in the logic program

  • how rugged is landscape from logic programming?

Computersimulations:… [SS & WATWA]

– flat energy landscape  no satisfiability problem

(all clauses can be satisfiedi.e. solution always guaranteed)

slide13

3rd MPSGC

KUALA LUMPUR 2007

Satisfiability

In general, general CNF clauses(conjuctions of disjunctions)not necessarily satisfiable

- depends on number of atoms in disjunctions

number of disjunctions

number of distinct atoms

Exist phase transition from easily soluble problems to difficult problems

[Zecchina & Monasson]

slide14

3rd MPSGC

KUALA LUMPUR 2007

Can \'knowledge\' may be associated with the energy landscape?

What is knowledge?

? How ingrained are logical rules in neural network –

“ingrainedness”:

GX←Y,Z = < E({Si} satisfying X←Y,Z) - E({Si} not satisfying X←Y,Z) >

Can this be related to landscape ruggedness e.g. correlations?

slide15

3rd MPSGC

KUALA LUMPUR 2007

Terima kasih

ad