slide1 n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Steve Donaldson Department of Mathematics and Computer Science Samford University PowerPoint Presentation
Download Presentation
Steve Donaldson Department of Mathematics and Computer Science Samford University

Loading in 2 Seconds...

play fullscreen
1 / 46

Steve Donaldson Department of Mathematics and Computer Science Samford University - PowerPoint PPT Presentation


  • 116 Views
  • Uploaded on

An Artificial Neural Network for Multi-Level Interleaved and Creative Serial Order Cognitive Behavior. Steve Donaldson Department of Mathematics and Computer Science Samford University Birmingham . Alabama. Research Concern. Example. Variable binding. Smolensky, 1990.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Steve Donaldson Department of Mathematics and Computer Science Samford University' - kiona-burke


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

An Artificial Neural NetworkforMulti-Level InterleavedandCreative Serial Order Cognitive Behavior

Steve Donaldson

Department of Mathematics and Computer Science

Samford University

Birmingham. Alabama

some research concerns related to the exploration of intelligent systems

Research Concern

Example

Variable binding

Smolensky, 1990

Central executive function

Baddeley, 1992

Similarity matching

Sloman & Rips, 1998

Emotional impact on decisions

Damasio, 1994

Case based reasoning

Kolodner, 1997

Chunking

Laird, Newell, & Rosenbloom, 1987

Strategy development

Anumolu, Bray, & Reilly, 1997

Goal management and planning

Albus, 1991

Analogy development

Hofstadter, 1995

Temporal processing

Rosenblatt, 1964

Common sense reasoning

Sun, 1994

Mathematical reasoning

Anderson, 1995

Language

Gupta & Dell, 1999

Credit assignment

Holland, 1995

Rule processing

Goebel, 1991

Creativity

Hofstadter, 1995

Some Research Concerns Related to the Exploration of Intelligent Systems

(Adapted from Donaldson, 1999)

basic requirements for autonomous systems
Basic Requirements for Autonomous Systems
  • Solve multiple tasks within the framework of a composite, synergistic architecture
  • Act autonomously under the internal control of neural network type processes
  • Learn in a biologically realistic manner
  • Operate at a scale significantly larger than normally found in single purpose networks
  • Acquire knowledge in a manner consistent with biological constraints
  • Transfer information across tasks, thus dealing with new situations using previously acquired knowledge
  • Exhibit multiple memory modalities typical of human information processing
  • Perform lifetime plastic learning without catastrophic loss of previously acquired knowledge
  • Learn from internal as well as external stimuli
some cognitive skills and behaviors exhibited by humans
Some Cognitive Skills and BehaviorsExhibited by Humans

Recognition • Alphabet mastery • Spelling • Counting •

Acquisition of math facts • Memorization of a script •

Basic motor skills • Associative memory • Rehearsal •

Multiple associations • Free association • Transcription •

Solving mathematical expressions • Memory theatres •

Understanding simple pronoun referents • Complex motion •

Proto-language reading comprehension • Route following •

General inductive reasoning • Multiple trains of thought •

Acquisition and deployment of external memory strategies •

Sophisticated non-stereotypical sequence processing •

suggested comprehensive explanatory mechanisms
Suggested Comprehensive Explanatory Mechanisms

Predictive learning

Interleaved processing

Sequence creation via generalized variable binding

categorizing cognitive abilities by required mental features
Predictive Learning

Alphabet mastery

Spelling

Acquisition of math facts

Memorization of a script

Basic motor skills

Associative memory

Multiple associations

Categorizing Cognitive Abilitiesby Required Mental Features

Recognition

Interleaved Processing

  • Free association
  • Transcription
  • Route following
  • Memory theatres
  • Multiple trains of thought
  • Complex motion
  • Rehearsal

Sequence Creation

  • Counting
  • Solving mathematical expressions
  • Understanding simple pronoun referents
  • Protolanguage reading comprehension
  • General inductive reasoning
  • Acquisition and deployment of external memory strategies
  • Sophisticated non-stereotypical sequence processing
some temporal processing concepts
Some Temporal Processing Concepts
  • Pattern – a vector of values representing an idea or action in the model’s experience, typically treated as a 2D figure to aid in visualization and conceptualization.
  • Sequence - temporally ordered collection of input/output patterns.
  • Recognition - the competence of a system to identify previously learned features or concepts with minimal ambiguity, possibly from partial sensory input, and in the absence of any singular temporal contextual reference; specifically, the retrieval of a previously stored version of a pattern from long-term recognition memory.
  • Predictive learning – an ability acquired by previous exposure to a sequence to reproduce patterns in that sequence based on the current state of a context module and the current input.
  • Interleaved processing – the production and use of temporally ordered information based on sequence hierarchies (e.g. sequence A is composed of sequences B and C, sequence B is composed of sequences C, D, and E, etc.).
  • Sequence creation – production of a new sequence from an existing seed sequence and associations related to its members.
sample pattern representations
Sample Pattern Representations

Internal representation for the letter “A”

-1 1 1 1 1 1 1 1 1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 1 1 1 1 1 1 1 1 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1  

Internal representation for a “boat”

-1 -1 -1 -1 -1 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 1 1 1 1 1 1 1 1 1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 1 1 1 1 1 1 1 1 1 -1 -1 -1 1 1 1 1 1 1 1 -1 -1 -1 -1 -1 1 1 1 1 1 -1 -1 -1

predictive learning
Predictive Learning

Rosenblatt (1964)

Elman (1990)

Context state (Si) and input/output (Ii) changes in a predictive learning system

acquisition of math facts
Acquisition of Math Facts

Pattern set for restricted math fact learning

Some basic math facts considered as temporal sequences

Math fact learning represented as sequence completion

script learning as a form of prediction

Script Learning as a Form of Prediction

S2WE_THE_PEOPLE_OF_THE_UNITED_STATES,_IN_ORDER_

TO_FORM_A_MORE_PERFECT_UNION,_ESTABLISH_JUSTICE,

_INSURE_DOMESTIC_TRANQUILITY,_PROVIDE_FOR_THE_

COMMON_DEFENSE,_PROMOTE_THE_GENERAL_WELFARE,_

AND_SECURE_THE_BLESSINGS_OF_LIBERTY_TO_OURSELVES

_AND_OUR_POSTERITY,_DO_ORDAIN_AND_ESTABLISH_THIS_

CONSTITUTION_FOR_THE_UNITED_STATES_OF_AMERICA.█

S1A_PENNY_SAVED_IS_A_PENNY_EARNED█

Avoiding catastrophic interference via sparse neural firing in sequence context

basic motor skills
Basic Motor Skills

MuscleArm SegmentMovementMovement Code

1 Upper Clockwise M1

2 Upper Counter-clockwise M2

3 Lower Clockwise M3

4 Lower Counter-clockwise M4

M1 M2 M3 M4 M5 M6 M7 M8

Muscle control patterns for a simple arm

two simple movement sequences
Two Simple Movement Sequences

A “reaching” sequence

A “putting” sequence

associative memory via predictive learning
Associative Memory via Predictive Learning

Some learned associations

Associative Recall

multiple associations based on probabilistic firing in the sequence context module
Multiple Associations Based on Probabilistic Firing in the Sequence Context Module

Two sets of learned multiple associations

Recall results from several multiple association tests when probing with [mts]_ _ and [water]_ _

short term priority memory
Short-Term Priority Memory

Stylized view of short-term priority module activation gradient changes over time in the process of generating the strokes in the letters of the sequence CAT.

free association
Free Association

A trace of the pattern perception module

An associative tale

A trace of the collective microfeatures module

multiple trains of thought
Multiple Trains of Thought

Learned sequences

“Thinking” several thoughts

The effect of parameter adjustment on recall order

a route following experiment

From

To

Highway

Dumas, Texas (DU TX)

Raton, New Mexico (RAT NM)

US64

Glenwood Springs, Colorado (GS CO)

Aspen, Colorado (ASP CO)

CO82

Birmingham, AL (BIR AL)

Memphis, Tennessee (ME TN)

US78

Raton, New Mexico (RAT NM)

Denver, Colorado (DEN CO)

I25

Amarillo, Texas (AM TX)

Dumas, Texas (DU TX)

US87

Memphis, Tennessee (ME TN)

Amarillo, Texas (AM TX)

I40

Denver, Colorado (DEN CO)

Glenwood Springs, Colorado (GS CO)

I70

A Route Following Experiment

Localized route sub-sequences lacking global order

route following via interleaved processing
Route Following Via Interleaved Processing

Correctly ordered route recall after learning randomly ordered components

learning for a transcription experiment
Learning for a Transcription Experiment

Patterns

Sequences

An interleaved processing hierarchy

complex motion
Complex Motion

Muscle control output for a complex motion

memory theatres
MemoryTheatres

Conceptual approaches to temporal knowledge representation for memory theatres

several approaches to rehearsal
Several Approaches to “Rehearsal”

Pattern set for “rehearsal” simulations

E934█E934█9██3██4███9██3███4█████████9██████3█████4███████E934█9██3██4██████9████3████4███████████9███████3███████4█████████████████9████████████3████████████4█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████

One approach to sequence “repetition” via interleaved processing

Π3.14159#Π█Π3.14159#Π█3██.██1██4██1██5███9███#███Π3.14159#Π█3██.██1██4██1██5███9███#███Π3.14159#Π█3██.██1██4██1██5███9███#███Π3.14159#Π█3██.██1██4██1██5███9███#███Π3.14159#Π█3██.██1██4██1██5███9███#██

The results of another approach to “rehearsal”

ΠΠ3.14159#█ΠΠ3.14159#█ΠΠ3.14159#█ΠΠ3.14159#█ΠΠ3.14159#█ΠΠ3.14159#█ΠΠ3.14159#█ΠΠ3.14159#█ΠΠ3.14159#█ΠΠ3.14159#█ΠΠ3.14159#█ΠΠ3.14159#█ΠΠ3.14159#█ΠΠ3.14159#█ΠΠ3.14159#█ΠΠ3.14159#█ΠΠ3.14159#█ΠΠ3.14159#█

An indication of how “rehearsal” results can depend on sequence format

sequence creation
Sequence Creation

Seed Sequence

P1 P2 P3 P4 …

P41 P42 P43 … P4S

P31 P32 P33 … P3R

P21 P22 P23 … P2N

P11 P12 P13 … P1M

Previously Learned Sequences

Created Sequence

P1M P2N P3R P4S …

solving mathematical expressions
Solving Mathematical Expressions

Additional sequence learning requirements

A trace of patterns produced during the solution of a mathematical expression

protolanguage reading comprehension
Protolanguage Reading Comprehension

Donaldson, Steve (2003a). An artificial neural network model for reading comprehension. In Arabnia, H., Joshua, R., & Mun, Y. (Eds.), Proceedings of the Internal Conference on Artificial Intelligence, Volume 1. Las Vegas, NV: CSREA Press.

Patterns required for a reading experiment

Previously learned sequences necessary for reading

Assimilating letters into words and concepts

general inductive reasoning
General Inductive Reasoning

Patterns used in an inductive reasoning experiment

Sequence learning foundation for inductive reasoning

Observations preceding inductive rule formation

inductive rule formation and application
Inductive Rule Formation and Application

An inductive rule formed via sequence creation

Additional sequence learning for inductive rule application

Application of a rule learned via inductive reasoning

external memory strategies
External Memory Strategies

Targets

Objects

Destination

Relations

Strategy

Relations

Control

Patterns

Object-Target Categorization

sequence learning for an external memory strategies experiment
Sequence Learning for an External Memory Strategies Experiment

Observations preceding formation of a memory strategy

learning by example as a foundation for the creation of external memory strategies
Learning by example as a foundation for the creation of external memory strategies

Trial 1

Trial 8

Trial 10

applying a learned external memory strategy
Applying a Learned External Memory Strategy

External memory strategies learned by example

Some additional facts to be learned before strategy application

Recall and application of an external memory strategy

a non stereotypical sequence processing experiment in the domain of music
A Non-Stereotypical Sequence Processing Experiment in the Domain of Music

Donaldson, Steve (2003b). A neural network for high-level cognitive control of serial order behavior. In Ventura, D. & Das, S. (Eds.), Proceedings of the 7th Joint Conference on In-formation Sciences (6th International Conference on Computational Intelligence and Natural Computing). Research Triangle Park, NC: Association for Intelligent Machinery.

Model Expansion to accommodate embedded sequences

Key designations for the three octaves mapped below

Note to keyboard position transformation maps and a phrase from a song

non stereotypical sequence processing
Non-Stereotypical Sequence Processing

Donaldson, Steve (2003b). A neural network for high-level cognitive control of serial order behavior. In Ventura, D. & Das, S. (Eds.), Proceedings of the 7th Joint Conference on In-formation Sciences (6th International Conference on Computational Intelligence and Natural Computing). Research Triangle Park, NC: Association for Intelligent Machinery.

Flowchart of NSTSP processing in the domain of music

“Playing” a song at a designated octave as a form of NSTSP

counting
Counting

Patterns for a counting experiment

Sequences learned as a foundation for counting

Representing item abstraction for a counting task

Results of counting the members of a group of people

understanding simple pronoun referents
Understanding Simple Pronoun Referents

Simple pronoun to antecedent conversion

results
Explore low level cognitive mechanisms

Maintain close ties to biological systems

Seek generic principles subserving intelligence

Evaluate a parsimonious approach to systems design

Investigate foundations for high-level cognition

Explore interaction of multiple memory modalities

Demonstrate sufficiency of the proposed foundation

Results