General mechanisms of neocortical memory
This presentation is the property of its rightful owner.
Sponsored Links
1 / 57

General mechanisms of Neocortical memory PowerPoint PPT Presentation


  • 64 Views
  • Uploaded on
  • Presentation posted in: General

General mechanisms of Neocortical memory. Jeff Hawkins Director Redwood Neuroscience Institute June 12, 2003 MIT. Outline. Top down analysis : nature of problem and solution representation time and prediction

Download Presentation

General mechanisms of Neocortical memory

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


General mechanisms of neocortical memory

General mechanisms of Neocortical memory

Jeff Hawkins

Director

Redwood Neuroscience Institute

June 12, 2003 MIT


Outline

Outline

  • Top down analysis:nature of problem and solutionrepresentationtime and prediction

  • Bottom up example:auditory memory task- deduce necessary algorithms- unique map to anatomy


General mechanisms of neocortical memory

“I conclude that cytoarchitectural difference between areas of neocortex reflect differences in their patterns of extrinsic connections. The traditional or usual ‘functions’ of different areas also reflect these differences in extrinsic connections. They provide no evidence whatsoever for differences in intrinsic structure or function..”

“Put shortly, there is nothing intrinsically motor about the motor cortex, nor sensory about the sensory cortex. Thus the elucidation of the mode of operation of the local modular circuit anywhere in the neocortex will be of great generalizing significance.”

Vernon Mountcastle, 1978


General mechanisms of neocortical memory

temporallyinvariant

spatially

invariant

motor

touch

audition

vision

temporally

specific (fast)

spatially

specific


Neocortical connectivity

Neocortical connectivity


General mechanisms of neocortical memory

temporallyinvariant

spatially

invariant

motor

touch

audition

vision

temporally

specific (fast)

spatially

specific


General mechanisms of neocortical memory

temporallyinvariant

spatially

invariant

motor

touch

audition

vision

temporally

specific (fast)

spatially

specific


General mechanisms of neocortical memory

temporallyinvariant

spatially

invariant

motor

touch

audition

vision

temporally

Specific (fast)

spatially

specific

Prediction

(spatially and temporally specific)

MacKay, Mumford, Softky, Rao & Ballard


General mechanisms of neocortical memory

temporallyinvariant

spatially

invariant

motor

touch

audition

vision

temporally

fast

spatially

specific

Prediction

(spatially and temporally specific)

Q1. Why make predictions?

Q2. How do we make predictions?

Q3. How do we form invariant representations?


Q1 why make predictions

Q1. Why make predictions

Non-mammalianbrain

Complex

behavior

Sophisticated

senses


Posterior neocortex sensory prediction

Posterior Neocortex: sensory prediction

Mammalianposterior neocortex

Predictions allow brain to react prior to events, to “see” into the future.

Complex

behavior

Sophisticated

senses


Anterior neocortex motor sequences

Anterior Neocortex: motor sequences

Mammalianposterior neocortex

Humananterior neocortex

Complex

behavior

Sophisticated

senses


Q2 how do we make predictions

Q2. How do we make predictions?

  • - Store sequence of patterns: allows prediction of future events

  • - Invariant representations cannot make specific predictions

invariant

representations

specific

afferents

time


Q2 how do you make predictions

Q2. How do you make predictions?

  • - Store sequence of patterns: allows prediction of future events

  • - Invariant representations cannot make specific predictions

  • - invariant prediction + input[t-1] = specific prediction[t]

invariant

representations

+

specific

afferents

time


Q3 how do we form invariant representations

Q3. How do we form invariant representations?

  • Spatially invariant representations require

  • - convergence of features that constitute object

  • - divergence to unite objects that although different represent the same thing

(x1⋂x2⋂x3 …) ⋃ (x4⋂x5⋂x6 …) ⋃ (x7⋂x8⋂x9 …) …


Top down summary

Top down summary

  • Every cortical region:

  • - Forms representations by convergence of features

  • - Forms invariant representations by divergence

  • - Stores and recalls sequences of invariant representationssequence memory

  • - Recalls pattern sequences auto-associatively

  • - Combines recalled patterns with input to:

  • make predictions of sensory afferents

  • drive motor efferents


Top down summary1

Top down summary

  • Every cortical region:

  • - Forms representations by convergence of featuresL4, Thalamus

  • - Forms invariant representations by divergenceL2,3 horiz

  • - Stores and recalls sequences of invariant representationsL1,2,3sequence memory

  • - Recalls pattern sequences auto-associatively

  • - Combines recalled patterns with input to:L5,6

  • make predictions of sensory afferents

  • drive motor efferents


Bottom up example

Bottom up example:

  • Auditory memory (melodies)

  • - Representations are invariant to pitch

  • recognized and recalled in any pitch

  • - Stored as sequences of associated patterns

  • have repeated elements (ggge- fffd ggge- aaag)

  • each note has a stored duration

  • - Prediction: we “hear” notes prior to occurrence

  • - Hierarchical representation, e.g. AABA structure(temporal invariance/reduction)


General mechanisms of neocortical memory

A1

L freq H

Thalamus


Pitch invariance interval representation

Pitch invariance = interval representation

octave

intervals

A2

C-C’ D-D’ E-E’ F-F’ G-G’A-A’ B-B’

frequency

C D E F G A B C1 D1 E1

A1

(x1⋂x2⋂x3 …) ⋃ (x4⋂x5⋂x6 …) ⋃ (x7⋂x8⋂x9 …) …

(C⋂C’) ⋃ (D⋂D’) ⋃ (E⋂E’) …


General mechanisms of neocortical memory

A2

L freq H

A1

L freq H

Thalamus


General mechanisms of neocortical memory

H

Intersecting inputs in layer 4define all possible intervals

A2

L

L freq H

A1

L freq H

Thalamus


General mechanisms of neocortical memory

Iso-interval bands

up

down

H

A2

L

L freq H

A1

L freq H

Thalamus


General mechanisms of neocortical memory

Freq invariant interval bands

up

down

H

A2

L2,3

L4

L

L freq H

- Intersecting inputs to L4

- Spread of activation in L2,3

A1

L freq H

Thalamus


General mechanisms of neocortical memory

How do we store the sequence of interval activations?

How do we represent unique intervals in unique songs?GGGE- FFFD GGGE- AAAG

How do we store and recall the precise time duration ofeach unique interval?


General mechanisms of neocortical memory

Layer 2,3 cells

Dense and small

High local mutual excitation

High local mutual inhibition

Long distance excitatory coll.

Dendrites in L1

Axon synapses in L5

L1

L2,3

L4

L5

L6


General mechanisms of neocortical memory

Layer 2,3 is sparsely active

Mutual excitation drives all

Strong inhibition prevents most cells from firing

Layer1 plays role in deciding who is active

L1

L2,3

L4

L5

L6


General mechanisms of neocortical memory

Layer 1 is context

1. Context from higher areas

2. Local context from L2,3

3. Input from matrix thalamus (time)

L1

L2,3

L4

L5

L6


General mechanisms of neocortical memory

Layer 1

context

L1

L2,3

Layer 2,3

unique representations of freq invariant intervals

L4

L5

L6

There is a unique sparse L2,3 activation pattern for each instance of this interval ever learned. Each unique pattern represents a particular interval in a particular melody.


General mechanisms of neocortical memory

H

L

L freq H

Layer 1

State: time & location

L1 axons link representations in sequence.Unique representations link to unique representations

Layer 4

Freq specific intervals

Converging inputs form object representations

Layer 2,3

Freq invariant intervals

Horizontal connections joinobjects to form spatially invariant representations

Song is represented as a sequence of freq invariant interval bands. Each invariant interval has a unique representation and is associatively linked to its predecessor.


General mechanisms of neocortical memory

Representing “class” and “individuality”

Activation area defines object class

Unique activation pattern defines individual object


General mechanisms of neocortical memory

How do we store and recall the precise time duration ofeach unique interval?

- Actual duration vs. relative duration (actual)

- Duration must be stored in-situ with interval

Proposal …

- Matrix thalamic nuclei emits a clock pattern to L1

- Part of L1 changes on each clock tick

- L5 cell resets clock on L4 transition or L1 match


General mechanisms of neocortical memory

L1

L2,3

L4

L5

L6

Matrix

Thalamus

New input arrives at L4, causes L5 cell to burst, inhibition shuts down L4

L5 burst teaches L5 cell to fire when exact pattern in L1 is seen in future

L5 burst also sets matrix thalamic nuclei to a deterministic state (resets clock) causing interval state transition

L5 cells encode duration of a particular state (note in song): when the elapsed time of a particular state occurs, they burst fire


How do you predict next note in proper key

How do you predict next note in proper key?

  • invariant prediction + input[t-1] = specific prediction[t]

invariant

representations

+

specific

afferents

time


General mechanisms of neocortical memory

L1

L2,3

freq

L4

Pattern from A1

L6a

L6b

Th(t)

A1(t-1)


General mechanisms of neocortical memory

L1

L2,3

Simple interval

freq

L4

Pattern from A1

L6a

L6b

freq

Th(t)

Th(t)

A1(t-1)


General mechanisms of neocortical memory

L1

Invariant unique interval

L2,3

Simple interval

freq

L4

Pattern from A1

L6a

L6b

freq

Th(t)

Th(t)

A1(t-1)


General mechanisms of neocortical memory

Associative spread

L1

Invariant unique interval

L2,3

Simple interval

freq

L4

Pattern from A1

L6a

L6b

freq

Th(t)

Th(t)

A1(t-1)


General mechanisms of neocortical memory

L1

Predicted next interval

L2,3

freq

L4

Pattern from A1(t)

L6a

L6b

freq

A1(t)


General mechanisms of neocortical memory

L1

Predicted next interval

L2,3

freq

L4

A1(t) + predicted interval

L6a

L6b

freq

A1(t)


General mechanisms of neocortical memory

L1

Predicted next interval

L2,3

freq

L4

A1(t) + predicted interval

L6a

Next predicted noteback to Thalamus

L6b

freq


General mechanisms of neocortical memory

L1

Predicted next interval

L2,3

freq

L4

A1(t) + predicted interval

L6a

Horizontal projectionsfrom stored previous richpattern to apical dendritesof predicted pattern copiesrich attributes

L6b

freq


Hierarchical representation

Hierarchical representation

  • words / melodies

sentences

phrases / songs


Hierarchical representation1

Hierarchical representation

  • words / melodies

sentences

Problem

The number of state transitions must decrease as you ascend the hierarchy.

However L2,3 projects to upper areas and it changes on every event.

phrases / songs


Hierarchical representation2

Hierarchical representation

Solution

Some cells in L2,3 learn to be stable over repeated patterns.


Hierarchical representation3

Hierarchical representation

Solution

Some cells in L2,3 learn to be stable over repeated patterns.

Therefore we should see L2,3 cells that stay active over longer periods of time. Only these cells should project to next higher cortical area.


How generic is this model

How generic is this model?

  • Performs a non-trivial memory processing function- invariant, rich predicting, branching, hierarchical, sequence memory

  • Aligns well with top down constraints

  • Accounts for much of known cortical anatomy- involves all layers, excitatory and inhibitory spread- how could other areas of cortex be fundamentally different?

  • Other cortical areas are likely variations on this theme

  • Other principles are likely in use as well


General mechanisms of neocortical memory

Redrawing A2

A2 as it might appear- limited to octave intervals- appearance of tonotopy

A2 as I have drawn it


General mechanisms of neocortical memory

Possible interpretation of A1

Broader tuned, sweep

Narrowly tuned

Broader tuned, sweep

low freq high

Compares input from two ears- inter-aural delay accentuated subcortically- predicts location of sounds in body space


Summary

Summary

  • 1)Converging L4 inputs define objects

  • 2) Horizontal connections in L2,3 create spatially invariant representations

  • 2)Sparse activation in Layers 2,3 encodes unique instances of invariant representations

  • 3) L1 mediates memory of sequences

  • 4)L5 thalamo-cortical loops encode duration of events

  • 5) Sustained activity in some L2,3 cells establishes basis for temporal invariance

  • 6) L6 cells make specific predictions from L2,3 and afferents


Summary1

Summary

  • 1)Converging L4 inputs define objects

  • 2) Horizontal connections in L2,3 create spatially invariant representations

  • 2)Sparse activation in Layers 2,3 encodes unique instances of invariant representations

  • 3) L1 mediates memory of sequences

  • 4)L5 thalamo-cortical loops encode duration of events

  • 5) Sustained activity in some L2,3 cells establishes basis for temporal invariance

  • 6) L6 cells make specific predictions from L2,3 and afferents

  • Testable - buildable - a start


General mechanisms of neocortical memory

Thank

- - -


General mechanisms of neocortical memory

  • “It is not that most neurobiologists do not have some general concept of what is going on. The trouble is that the concept is not precisely formulated. Touch it and it crumbles. What is conspicuously lacking is a broad framework of ideas within which to interpret these different approaches.”

  • Francis Crick 1979


General mechanisms of neocortical memory

All cortical regions

There is “no evidence whatsoever for differences in intrinsic structure or function. This suggests that the necortex is everywhere functionally much more uniform than hitherto supposed and that its avalanching enlargement in mammals and particularly in primates has been accomplished by replication of a basic neural module without the appearance of wholly new neuron types or qualitatively different modes of intrinsic organization.”

“Put shortly, there is nothing intrinsically motor about the motor cortex, nor sensory about the sensory cortex. Thus the elucidation of the mode of operation of the local modular circuit anywhere in the neocortex will be of great generalizing significance.”

Vernon Mountcastle, 1978


  • Login