Brain code and brain reading
This presentation is the property of its rightful owner.
Sponsored Links
1 / 65

Brain Code and Brain Reading PowerPoint PPT Presentation


  • 68 Views
  • Uploaded on
  • Presentation posted in: General

Brain Code and Brain Reading. Prof.dr. Jaap Murre University of Maastricht University of Amsterdam [email protected] http://neuromod.org. What principle should it use? Storage capacity Resistance to damage Access speed Serve necessary calculations. What constraints can be identified?

Download Presentation

Brain Code and Brain Reading

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Brain code and brain reading

Brain Code and Brain Reading

Prof.dr. Jaap Murre

University of Maastricht

University of Amsterdam

[email protected]

http://neuromod.org


How does the brain represent information

What principle should it use?

Storage capacity

Resistance to damage

Access speed

Serve necessary calculations

What constraints can be identified?

Neurons are very noisy

They can do only simple calculations

The are severely limited in how they can communicate

How does the brain represent information?


Localized versus distributed coding

Localized versus distributed coding

  • Distributed coding

    • 1010111000101100110101000110111000

  • Extremely localized coding

    • 0000000000000000010000000000000000

  • Sparse (or semi-distributed) coding

    • 0000100000100000010000000010000000


Distributed coding

Distributed coding

  • Maximizes storage capacity

  • Resistant to damage

  • Able to implement complex calculations

  • May need high metabolism (energy, O2)

  • Overlap of codes can cause severe interference in processing and learning


Localized coding

Localized coding

  • Is able to store only a few representations but also forces category formation

  • Sensitive to loss (causes very specific disorders)

  • Mainly useful for theoretical purposes


How can codes come into existence example competitive learning

How can codes come into existence? Example: Competitive learning

  • Competitive learning is a form of unsupervised learning

  • Needs a large degree of localization (or very sparse codes) to work

  • Forms categories on the basis of regularities (is also called regularity learning)

  • These networks are able to signal novelty


Example of competitive learning stimulus at is presented

Example of competitive learning:Stimulus ‘at’ is presented

1

2

a

t

o


Example of competitive learning competition starts at category level

Example of competitive learning:Competition starts at category level

1

2

a

t

o


Example of competitive learning competition resolves

Example of competitive learning:Competition resolves

1

2

a

t

o


Example of competitive learning hebbian learning takes place

Example of competitive learning:Hebbian learning takes place

1

2

a

t

o

Category node 2 now represents ‘at’


Presenting to leads to activation of category node 1

Presenting ‘to’ leads to activation of category node 1

1

2

a

t

o


Presenting to leads to activation of category node 11

Presenting ‘to’ leads to activation of category node 1

1

2

a

t

o


Presenting to leads to activation of category node 12

Presenting ‘to’ leads to activation of category node 1

1

2

a

t

o


Presenting to leads to activation of category node 13

Presenting ‘to’ leads to activation of category node 1

1

2

a

t

o


Category 1 is established through hebbian learning as well

Category 1 is established through Hebbian learning as well

1

2

a

t

o

Category node 1 now represents ‘to’


Kohonen self organizing map

Kohonen self-organizing map

  • Winner-take-all competition

  • Network has 2D (or 3D etc.) layout

  • Weights of neighbors of the winner are updated with the winner

  • The map tries to mirror the input space

  • This works even with semantic spaces


Kohonen map winner take all

Kohonen map: winner-take-all


Kohonen map winner take all1

Kohonen map: winner-take-all

Initial weights are random

Initial winners will be random as well

As time goes by, winners to similar inputs will tend to be located close together


Weights are updated to the winner and to its neighbors

Weights are updated to the winner and to its neighbors

Weights are moved towards input

For example:

Inputs are: .9 .9 .7

Weights are: .5 .3 .9

New weight are: .6 .4 .8

.9 .9 .7


Kohonen self organizing map1

Kohonen self-organizing map

  • The size and shape of the neighborhood determines the final layout

  • The size of the neighborhood is usually diminished with training

  • The learning rate is typically also slowly diminished

  • Training may take a long time


Example 2d map tries to capture 3d color space by simon lucas

Example: 2D map tries to capture 3D color space (by Simon Lucas)

  • Input vectors are drawn randomly from RGB (red, green, blue) color space

  • Node color reflects the node’s incoming weights (it will respond best to that color)

  • Depending on the learning rate and neighborhood different organizations emerge

  • No single fixed organization is best in this case

  • Other example:


Sparse coding

Sparse coding

  • Forms a good middle ground between fully distributed and extremely localized coding

  • Is computationally sound in that it allows very large numbers of representations with a small number of units


Brain codes in the motor system

Brain codes in the motor system

Codes for action and movement


Cortical anatomy of the motor system lateral view

Cortical anatomy of the motor system: lateral view


Medial view

Medial view


Brain code and brain reading

Schematic

overview of the

motor system


Basic questions regarding motor control can nowadays be answered

Basic questions regarding motor control can nowadays be answered

  • How are motor movements represented in the brain?

  • How are they used in the production of movement?

  • Which brain areas are involved?


How to be precise with noisy components

How to be precise with noisy components

Area 5 neuron during repeated reaching movements: each

individual trial gives a rather imprecise signal


Population coding

Population coding

  • Population coding allows precise representations on the basis of (very) noisy or even damaged components

  • Population coding is based on the statistics of averages

  • They rely on coarse-coded neural representations


Coarse coding

Coarse coding

  • If a neuron’s representation responds to ‘many’ inputs, this is called coarse coding

  • The advantage is that more accurate representations can be formed by suitable combination of the coarse representations


Why coarse coding works

Why coarse coding works

  • If we move along a straight line, each time we cross a receptive field boundary one neurons changes its activation:

  • the representation changes.


In primates abundant evidence exists for coarse coding

In primates abundant evidence exists for coarse coding


Georgopoulos shows that movement is coded in population vectors

Georgopoulos shows that movement is coded in population vectors


Population vectors give accurate movement direction signals

Population vectors give accurate movement direction signals


Motor cortex sets up the signal but execution is dependent upon other areas

Motor cortex sets up the signal, but execution is dependent upon other areas


Response competition

Response competition


Spinal cord

Spinal cord


Coarse maps of limb movements in the frog

Coarse maps of limb movements in the frog

  • Spinal cord of frogs does significant motor processing

  • Frog can still ‘clean’ itself after severing of cord (dogs can also still scratch themselves)

  • The data suggest that even at a spinal level coarse coding is used

  • It is likely that similar types of coding are used in mammals


Cats with severed spinal cord could still walk on a treadmill

Cats with severed spinal cord could still walk on a treadmill


Method followed by emilio bizzi

Method followed by Emilio Bizzi

Based on the idea of ‘muscles as springs’ by Feldman


Limb movements in frog spinal cord are coded with respect to their end positions

Limb movements in frog spinal cord are coded with respect to their end-positions


The interactions of force fields can be described by vector calculus

The interactions of force fields can be described by vector calculus

Fields A and B combined predict field <AB> (see C). When A and B are stimulated the resulting field (see D) corresponds to the theoretical field <AB>


Brain codes in the visual system

Brain codes in the visual system

Is our knowledge represented in a localized or distributed fashion?


Freedman riesenhuber poggio miller science 2001

Many (395) single cell recordings in monkey pre-frontal cortex

Trying to find categories in the brain

82 out of 395 were category selective

Freedman, Riesenhuber, Poggio, & Miller (Science, 2001)


Brain code and brain reading

Cells predict the correct category

(single-cell, lateral pre-frontal in monkey)


Brain reading

Brain Reading?

Jim Haxby’s (2001) study


Haxby et al 2001 categories in the human brain

Haxby et al. (2001)Categories in the human brain


Distributed respresentations

Distributed respresentations?


Brain code and brain reading

Voxels that respond to faces, also respond to houses and other categories


Analysis with neural networks by hanson matsuka haxby 2004

Analysis with neural networks by Hanson, Matsuka, & Haxby, 2004

face cat house bottle scissor shoe random chair


Brain code and brain reading

Marieke van der Linden FCDC

Miranda van Turennout FCDC

Jaap Murre UvA


Brain code and brain reading

Methods

Pretest 15 subjects

Task: 1-back with feedback

Example

Subjects will be trained on categorizing birds that are indistinguishable to the untrained eye.


Brain code and brain reading

3 Training sessions on 3 consecutive days

  • Two bird types will be assigned to categorization training (1-back same/different bird type) with feedback (“correct”, “false” or “too late”)

  • Two bird types will be assigned to visual training (same 1-back same/different bird type) with random feedback

  • Two bird types will not be trained, untrained

  • Different morphs than scan session (55, 65, 70, 80, 95%) to prevent pure repetition fx

  • Each task contains five blocks of 150 trials, 10 minutes per block, total: 50 min.

2 fMRI sessions: pre- and post training

Factors:Training (categorization, visual, untrained)

Morph level (60, 75, 90)

Design:

20 sec 12 sec 20 sec 12 sec 20 sec 12 sec 20 sec

….etc

Etc.

!

Each block will be repeated 6x, the order will be randomized

Total scan time: 57.6 min.


Cortical plasticity in the representation of object categories

Cortical Plasticity in the Representation of Object Categories


Putting it all together

Putting it all together

  • Models help us understand different roles of representations in the brain

  • At different levels of analysis, different brain codes are more effecient


Riesenhuber poggio 2002

Riesenhuber & Poggio (2002)


Visual processing in the brain palmeri gauthier 2004

Visual processing in the brain(Palmeri & Gauthier, 2004)


Natural input model nim

Natural Input Model (NIM)

Artificial Intelligence approach to cognitive modeling

Functional (computational) model of visual processing in V1/V2, V4, and IT.

With Eric Postma, Joyca Lacroix, and Jaap van den Herik (Universiteit Maastricht)


Natural input model nim1

Natural Input Model (NIM)

  • Modeling recognition and similarity of individual images (faces)

    • Cognitive Science (in press)


Summary

Summary

  • A lot is now known about motor and visual codes in the brain

  • We can understand many aspects of this circuitry in terms of ‘why this representation makes sense’

  • For example, coarse grained coding has the advantage of precise control despite noisy components


Summary continued

Summary (continued)

  • The brain employs many different representations, depending on their use and constraints

  • We are now starting to unravel many of these codes


  • Login