the sternberg experiments an ersatz test program l.
Skip this Video
Loading SlideShow in 5 Seconds..
The Sternberg Experiments: An Ersatz Test Program PowerPoint Presentation
Download Presentation
The Sternberg Experiments: An Ersatz Test Program

Loading in 2 Seconds...

play fullscreen
1 / 73

The Sternberg Experiments: An Ersatz Test Program - PowerPoint PPT Presentation

  • Uploaded on

The Sternberg Experiments: An Ersatz Test Program . James Anderson Department of Cognitive and Linguistic Sciences Brown University September 10, 2007. Ersatz Results, Plans and Goals. Overall goal: design a brain-like computer architecture , both hardware and software .

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'The Sternberg Experiments: An Ersatz Test Program' - Jims

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
the sternberg experiments an ersatz test program

The Sternberg Experiments: An Ersatz Test Program

James Anderson

Department of Cognitive and Linguistic Sciences

Brown University

September 10, 2007

ersatz results plans and goals
Ersatz Results, Plans and Goals

Overall goal: design a brain-like computer architecture, both hardware and software.

Such a device must be capable of being programmed.

Such a device must compute results for interesting and important classes of problems.

And (potentially) do it better than traditional computers.

potential applications
Potential Applications

We have studied several applications areas.

First, Cognitive Signal Processing:

  • Invariant Vowel Formant Ratio Computation
  • A Morse Code reader.
operations and programs
Operations and Programs

Second, we have realized several simple, useful basic program operations.


  • Identity of two patterns.
  • Symmetry of two patterns.
  • Ratio of two amplitudes.
  • Ersatz “logic.”
a more complex program
A More Complex Program

Third, we have designed a multi-step Ersatz Program to perform a significant task.

The task is to decide whether or not a new item is part of a previously learned list.

The task has similarities to many significant practical problems.

An example: deciding whether or not a character string (MANTY) is an English word.

multiple scales
Multiple Scales

Ersatz Programs work at multiple scales.

The Ersatz Brain has a natural set of functional scaling mechanisms.

First, single cortical columns (modules) modeled as a dynamical system.

arrays of modules
Arrays of Modules

Second, we proposed large sets of these modules formed the Network of Networks.

module assemblies
Module Assemblies

Third, small numbers of active modules in a Network of Networks array can became associatively linked through learning.

These module assemblies can form self-exciting groupings that can represent concepts.

sparse representation
Sparse Representation

In the brain representations seem to be sparse.

Sparse means there are not many active neurons.

By extension, we hypothesize that not too many columns are active to represent a complex figure, percept, or concept.

Tanaka’s IT data supports that claim.

Sparseness also turns out to be very useful for designing Ersatz applications.


For a useful programmable system we need to work at multiple scales.

The next step in scale is to work with multiple module arrays.

Module assemblies in multiple arrays work together in self exciting “arrays of arrays.”

Note these levels are all technically dynamical systems based on scaled components.

development strategy low level
Development Strategy: Low Level

We are working at both low and high levels of system organization.

At a low level, we want to show that emergent structures such as module assemblies arise naturally given:

  • The Network of Networks topographically organized array.
  • Module and system dynamics.
  • Hebbian associative learning.

Several small simulations suggest this is indeed the case.

development strategy high level
Development Strategy: High Level

At a high level, we assume that module assemblies have formed and that multiple interconnected arrays can exist.

We then ask what kind of computations can this rich system perform?

How can these computations be programmed?

And how well do they work?

My part of this presentation will largely be concerned with this set of issues.


Biological Evidencefor Module Assemblies(These papers and figures played a large role in our thinking about assemblies and scaling.)

biological evidence columnar organization in inferotemporal cortex
Biological Evidence:Columnar Organization in Inferotemporal Cortex

Tanaka (2003) suggests a columnar organization of different response classes in primate inferotemporal cortex.

There seems to be internal structure in these regions: for example, spatial representation of orientation of the image in the column.

it response clusters imaging
IT Response Clusters: Imaging

Tanaka (2003) used intrinsic visual imaging of cortex. Train video camera on exposed cortex, cell activity can be picked up. 

At least a factor of ten higher resolution than fMRI.

 Size of response is around the size of functional columns seen elsewhere: 300-400 microns.

columns inferotemporal cortex
Columns: Inferotemporal Cortex

Responses of a region of IT to complex images involve discrete columns.

The response to a picture of a fire extinguisher shows how regions of activity are determined.

Boundaries are where the activity falls by a half.

Note: some spots are roughly equally spaced.

active it regions for a complex stimulus
Active IT Regions for a Complex Stimulus

Note the large number of roughly equally distant spots (2 mm) for a familiar complex image.

basic hello world operations
Basic “Hello World” Operations

We will look next at interesting operations that might be performed on single NofN arrays.

We look at two operations in details:

  • Identity of two patterns.
  • Ratio of two analog quantities.

Consider simple identity.

Two sets of (sparse coded) features are on a Network of Networks array in any location.

Are the two sets the same?

Note: The “identity” operation really mean to “very high degree of similarity” not logical identity.

pattern spread
Pattern Spread

Patterns spread laterally from their origin.

When two of the same patterns meet they add.

The interference pattern module response is large.

These are spatio-temporal interactions like optics, or surface acoustic wave filters [SAW].)

ratio computation
Ratio Computation

Need to measure ratios of quantities for programs like Identity.

Can use amplitude timing relationships to generate spatial representation of a ratio.

General neural rule: weak signals produce slower responses than strong ones.

One name: Strength-latency tradeoff.

Found from peripheral sensory systems up to cortex.


Desired Spatial Behavior

Assume a linear array of modules.

Low (q1/q2): q1oooooRooooooooooooooooooooooooooooooq2.

^ ^

Input Strength q1 Input Strength q2

High (q1/q2): q1ooooooooooooooooooooooooooooooRooooq2.

^ ^

R indicates the location of active modules. The location of R should be related to the value of the ratio, (q1/q2).


Suppose we have a linear array of modules

Assume that q1 and q2 arrive simultaneously as inputs on the ends.  

Assume the time, Δt, for a module to respond to an input is a function of the input amplitude.

Example: Δt1 = c/q1 and Δt2 = c/q2

At a given time, the stronger input has traveled further than the weaker one.

bar code for ratios
Bar Code for Ratios

Approximate spatial locations for this strength-latency relationship.



^ ^

small 1:5 1:3 1:2 1:1 2:1 3:1 5:1 large


other points
Other Points

A system much like this is used for spatial localization in the auditory system.

This is a spatially arranged parallel system so we can compute several ratios simultaneously.


\ /

\ R

R /


identity as a protean function
Identity as a Protean Function
  • The Identity function is a flexible starting point.
  • Note: Ersatz-Identity is based on a similarity measure, not logical identity.

Other application examples:

  • Ersatz Logic
  • Ersatz Symmetry

Use a variant of the identity program to compute symmetry

Use lateral pattern spread with different weighting regions and times.


Key Point: Our approach to Eratz “software” is based on observing, using, and manipulating the spatial and temporal relationships among module activity patterns.

next step multiple arrays
Next Step: Multiple Arrays

Consider complex systems involving multiple NofN arrays and multiple module assemblies.

Two ways to look at the operation of memory in Ersatz Systems.

First is simple pattern recognition. Useful but inflexible.

Call it “passive memory.”

The second is task based memory. Can be programmed and is much more flexible.

Call it “active memory.”

active memory
Active Memory

For active memory, memory and sensory data from different NofN arrays mix together at another array.

We call this array the mixer.

In some programs the activity level at the mixer provides the result of the computation.

Such an architecture can be:

  • Flexible.
  • Task dependent.
  • Programmable.
the sternberg experiment
The Sternberg Experiment

We outline next a program to perform the famous (among cognitive scientists!) Sternberg List Scanning Experiments.


  • Memorize a short list of items (2 to 5 items).
  • Decide quickly whether a new item was on the list or not.
the task
The Task

Assume we learn a short list of letters.

A, B, G, K, L

K is presented.

Press the positive response key.

M is presented.

Press the negative response key.

Record the time it takes to respond.

not trivial
Not Trivial!

For non-academics: This task is not trivial and is similar to many real world tasks.

An important example:

Is MANTY a word?

(Need to check the full lexicon to answer.)

Is Corvette a car name?

Is Peerless a car name?

Is Utopian Turtletop a car name?


One mental function that we have great interest in doing with the Ersatz system is described in detail in a recent best selling book by Malcolm Gladwell: Blink.

The subtitle of Blink is “The Power of Thinking without Thinking.”


The point of the book:

“We believe that we are always better off gathering as much information as possible and spending as much time as possible in deliberations. There are moments …when our snap judgments and first impressions can offer a much better means of making sense of the world. The first task of Blink is to convince you of a simple fact: decisions made very quickly can be every bit as good as decisions made cautiously and deliberately. (p. 13-14)

fast decision systems
Fast Decision Systems

One goal of the Ersatz project is to make an artificial system with these properties:

  • Be Fast (Work in a single or very small number of steps. No tree searches.)
  • Be able to integrate information from many sources.
  • Example: Combine information from the senses and from perceptual analysis with past information stored in memory in a fast, efficient, programmable manner.
  • Be right most of the time.

We will use the Sternberg list experiments as our first attempt at showing how this might be done.

response time data
Response Time Data

Plot average response time against list length.

Key experimental finding:

Positive (present) and negative (absent) responses have the same slope.

implications of the data
Implications of the Data

Consider how reaction time will vary with number of list items.

If we scan the items sequentially, we will scan half the list items on average for a positive response.

But we must scan all the items for a negative response.

The slope of the negative responses should grow twices as fast as the positive responses.


Positive and negative slopes are equal.

This suggests that all the list items are tested at once before a response is made.

This implies that list matching is a parallel process and items are not matched individually.

Comment: The entire lexicon (400,000) items can be scanned in less than a second.

Seems unlikely to be a serial process.

sternberg program 1 arrays

Sternberg Program 1: Arrays

We assume 26 letters represent the categories. 

sternberg program 1 arrays54
Sternberg Program: 1. Arrays

Module Arrays: Three NofN Arrays, Array A, Array B, and Array C

Connections: In line, reciprocal


Array A: Sensory Input Array (a,b,c,… z)

Array B: Intermediate representations

Assemblies {a,b,c … z}

Array C: Classifications (Stable Module Assemblies)

Assemblies: {A,B,C …, Z} 

sternberg program 2 associations56
Sternberg Program 2: Associations

Standard associative learning:

Input to Output

Array A <=> Array B <=> Array C

Data Features Category

ay => a => A

bee => b => B

cee => c => C

zee => z => Z

sternberg program 2 associations57
Sternberg Program 2: Associations

When the categories are forming, we also learn the reciprocal associations:

ay <= a <= A

bee <= b <= B

cee <= c <= C

zee <= z <= Z

sternberg program 3 formation of the list filter
Sternberg Program 3: Formation of the List Filter

List Formation. Alist is a subset of the categories in Array C, for example, {A,B,C,D,E}

We present the list in sequence at Array C.

The downward projects to form a summed associative representations in Array B.

sternberg program 3 formation of the list filter59
Sternberg Program 3: Formation of the List Filter

Step 1. Form list representation by sequential presentation: 

Array A Array B <= Array C

a <= Present list item A

a+b <= Present list item B

a+b+c <= Present list item C

a+b+c+d <= Present list item D

a+b+c+d+e <= Present list item E

Assume activity in Array B sums patterns excited by downward projections from Array C.

sternberg program 4 interaction between data and memory
Sternberg Program 4: Interaction between Data and Memory

At the mixer, new sensory data and the list filter interact. For a linear system taking an inner product between inputs to middle Array B from higher level Array C and lower level Array A:

Step 2. Interaction

Array A Array B Array C

a => a+b+c+d+e

[a,a] + noise

sternberg program 5 decision criteria
Sternberg Program 5: Decision Criteria

Note: We assume (in the simplest case) that the Mixer (Array B) takes the inner product between the test item and the list filter.

The activity at Array B when input pattern a is presented to input Array A is then

Scalar Activity in B = [a,a] + [a,b] + [a,c] + [a,d] + [a,e]

= [a,a] + noise.

Note: This technique realizes a matched filter for pattern a. The matched filter is the optimal linear filter.

Example: If [a,b,c … z] are orthogonal:

A positive activity value corresponds to presence of the test input in the list. Zero activity corresponds to absence from the list.

sternberg program 5 decision criteria64
Sternberg Program 5: Decision Criteria

Step 3. Decision Criteria (very tentative):

Positive: Large, rapid, synchronized compact activation pattern in Array B is positive.

Negative: Smaller, slow, unsynchronized, widespread activation pattern in Array B is negative. 

The decision parameters are set by the accuracy required, the nature of the categories, etc. Experience is required.

an unintuitive prediction
An Unintuitive Prediction

Detecting the presence of a item on the list does not require identification of the test item.

The system does not reason, “I see a C. > C is on the list. > Therefore I will make a positive response.”

The process is instead “The item is some item on the list. > Therefore I make a positive response.

With additional effort I can conclude the item is a C.

This curious prediction is experimentally correct.

In general: It often makes sense to operate at a level the above the finest detail, even thought the detail is available.


We can extend this to multiple inputs from the sensory systems and from memory.

We can suggest how to program (on the fly) more complex multimodal list experiments.

An Example:

List: 1, 2, 2, 3, 4

Is 3 on the list?


We can bring together multiple sources of information at the mixer.

Activity levels in the mixer can tell whether or not some task demand is satisfied.

Note the combination of analog and pattern based computation.

And systems like this may be statistically optimal!

higher level cognition
Higher Level Cognition

The most interesting applications of the Ersatz system will be to high level cogntion.

For example, we can extend these ideas to work with contextual and conceptual information.

We may now have entries both into language and into cognitive signal processing with the same approach.

higher level cognition70
Higher Level Cognition

We can make simple context neighborhood from integrating multiple data sources.

We assume that previous input words hang around and are associatively linked to their associates..

Perhaps an entire semantic network could be constructed from the union of neighborhood.

Consider a set of four sentences on a common topic:

I needed caffeine.

I walked to Starbucks.

“A vente,” I said to the barrista.

“Ah, java”, I exclaimed


Assume that word patterns from multiple sentences superimpose just like the Sternberg experiments.  

What is the appropriate meaning of ‘java’ in this sequence?

The string ‘java’ can be:

  • A slang term for coffee.
  • An island in Indonesia.
  • A programming language.  

Superposition of the associations of the words gives rise to a context that easily disambiguates ‘java.’


The Ersatz Brain Project is developing a scalable, understandable, brain-like computing architecture.

We are working to turn it into a computing system.

We are developing techniques for working with real world data.

We feel the same basic ideas will work over a wide range of scales.

We can now outline programs that can be applied over a range from cognitive signal processing to task-based memory retrieval to contextual disambiguation.