slide1 n.
Skip this Video
Loading SlideShow in 5 Seconds..
Models of Cognitive Processes: Historical Introduction with a Focus on Parallel Distributed Processing Models PowerPoint Presentation
Download Presentation
Models of Cognitive Processes: Historical Introduction with a Focus on Parallel Distributed Processing Models

Loading in 2 Seconds...

play fullscreen
1 / 22

Models of Cognitive Processes: Historical Introduction with a Focus on Parallel Distributed Processing Models - PowerPoint PPT Presentation

  • Uploaded on

Models of Cognitive Processes: Historical Introduction with a Focus on Parallel Distributed Processing Models. Psychology 209 Stanford University Jan 7, 2013. Early History of the Study of Human Mental Processes. Introspectionism (Wundt, Titchener)

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Models of Cognitive Processes: Historical Introduction with a Focus on Parallel Distributed Processing Models' - huy

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Models of Cognitive Processes:Historical Introduction with a Focus on Parallel Distributed Processing Models

Psychology 209Stanford University

Jan 7, 2013

early history of the study of human mental processes
Early History of the Study of Human Mental Processes
  • Introspectionism (Wundt, Titchener)
    • Thought as conscious content, but two problems:
      • Suggestibility
      • Gaps
  • Freud suggests that mental processes are not all conscious
  • Behaviorism (Watson, Skinner) eschews talk of mental processes altogether
early computational models of human cognition 1950 1980
Early Computational Models of Human Cognition (1950-1980)
  • The computer contributes to the overthrow of behaviorism.
  • Computer simulation models emphasize strictly sequential operations, using flow charts.
  • Simon announces that computers can ‘think’.
  • Symbol processing languages are introduced allowing some success at theorem proving, problem solving, etc.
  • Minsky and Pappert kill off Perceptrons.
  • Cognitive psychologists distinguish between algorithm and hardware.
  • Neisser deems physiology to be only of ‘peripheral interest’
  • Psychologists investigate mental processes as sequences of discrete stages.
ubiquity of the constraint satisfaction problem
Ubiquity of the Constraint SatisfactionProblem
  • In sentence processing
    • I saw the grand canyon flying to New York
    • I saw the sheep grazing in the field
  • In comprehension
    • Margie was sitting on the front steps when she heard the familiar jingle of the “Good Humor” truck. She remembered her birthday money and ran into the house.
  • In reaching, grasping, typing…
interactive activation and probabilistic computation
Interactive activation and probabilistic computation
  • Rumelhart’s first effort to understand context effects in perception was formulated in explicitly probabilistic models.
  • Although he abandoned this formulation in favor of a neural network formulation, we will see that neural network models very similar to the IA model can be understood in explicit probabilistic terms.
  • Likewise, neural network models can be related to other sorts of models, including the drift-diffusion model of decision making and exemplar models of categorization and memory.
  • One of the goals of the course this year will be to explore these linkages more fully.
synaptic transmission and learning
Synaptic Transmission and Learning


  • Learning may occur by changing the strengths of connections.
  • Addition and deletion of synapses, as well as larger changes in dendritic and axonal arbors, also occur in response to experience.
  • [Recent evidence suggests that neurons may be added under certain circumstances.]


connection based learning creates implicit knowledge
Connection-based learning creates implicit knowledge
  • Connection adjustment affects processing, not necessarily conscious awareness.
  • But not all learning is implicit.
  • Connection based learning can also be used to reinstate patterns of activation or to ‘auto-associate’ some elements of a pattern with other elements.
  • Perhaps we are aware of the patterns, but not of the connections that support their activation.
cognitive neuropsychology 1970 s
Cognitive Neuropsychology (1970’s)
  • Geshwind’s disconnection syndromes:
    • Conduction Aphasia
      • Patient can understand and produce spoken language but cannot repeat sentences or nonwords
    • Alexia without Agraphia
  • Deep and surface dyslexia (1970’s):
    • Deep dyslexics can’t read non-words (e.g. VINT), make semantic errors in reading words (PEACH -> ‘apricot’)
    • Surface dyslexics can read non-words, and regular words (e.g. MINT) but often regularize exceptions (PINT).
  • Work leads to ‘box-and-arrow’ models, reminiscent of flow-charts
graceful degradation in neuropsychology
Graceful Degradation in Neuropsychology
  • Patient deficits are seldom all or none
    • This is true both at the task and at the item level.
    • Performance is slower, more errorful, and requires more contextual support.
  • And error patterns are far from random:
    • Visual and semantic errors in deep dyslexia suggest degradation, rather than loss of a module or disconnection
    • Regularization errors depend on a word’s frequency, and how many other exceptions there are that are like it
  • Effects of lesions to units and connections in distributed connectionist models nicely capture these features of neuropsychological deficits.
core principles of parallel distributed processing
Core Principles of Parallel Distributed Processing

/h/ /i/ /n/ /t/

  • Processing occurs via interactions among neuron-like processing units via weighted connections.
  • A representation is a pattern of activation.
  • The knowledge is in the connections.
  • Learning occurs through gradual connection adjustment, driven by experience.
  • Learning affects both representation and processing.


implications of this approach
Implications of this approach
  • Knowledge that is otherwise represented in explicit form is inherently implicit in PDP:
    • Rules
    • Propositions
    • Lexical entries…
  • None of these things are represented as such in a connectionist/PDP models.
  • Knowledge that others have claimed must be innate and pre-specified domain-by-domain often turns out to be learnable within the PDP approach.
  • Thus the approach provides an alternative to other ways of looking at many aspects of knowledge-dependent cognition and development.
  • While the approach allows for structure (e.g. in the organization and interconnection of processing modules; structured similarity relations among patterns of activation), processing is generally far more distributed, representation is less explicit, and causal attribution becomes more complex.
in short
In short…
  • Models that link human cognition to the underlying neural mechanisms of the brain simultaneously provide alternatives to earlier ways of understanding processing, learning, and representation at a cognitive level.
the pdp approach
The PDP Approach…
  • Attempts to explain human cognition as an emergent consequence of neural processes.
    • Global outcomes, local processes
  • Forms a natural bridge between cognitive science on the one hand and neuroscience on the other.
  • Is an ongoing process of exploration.
  • Depends critically on computational modeling and mathematical analysis.
beyond pdp
Beyond PDP
  • Since the PDP work began, several new approaches and communities have arisen
    • NIPS/Machine Learning Community
    • Computational Neuroscience Community
    • Bayesian Approaches in Cognitive Science and Cognitive Neuroscience
  • Many of the models we consider belong more to these communities than to what might be called ‘Classic PDP’
  • Much of my own work now involves either
    • Constucting models at the interface between PDP and other approaches
    • Attempting to understand the relationship between PDP models and models formulated in other frameworks, including Bayesian approaches.
  • A good fraction of the course material will cover work of this type, and links between such work and PDP models.
this course
This course…
  • Invites you to join the ongoing exploration of human cognition using PDP models and related approaches to mind, brain, and computation.
  • Focuses ultimately on human cognition and the underlying neural mechanisms, rather than abstract computational theory or artificial intelligence.
  • Includes exercises that provides an introduction to the modeling process and its mathematical foundations, preparing you to join the ongoing exploration.
assignment for wednesday
Assignment for Wednesday


  • * McClelland, J. L. (2013). Bayesian inference, generative models, and probabilistic computations in interactive neural networks. Draft, Jan. 6. 2013, Department of Psychology, Stanford University. Pages 1-28.

and for a primer on real neurons:

  • † Kolb, B. and Whishaw, I. Q. (1980). Physiological organization of the nervous system. Chapter 2 of Fundamentals of Human Neuropsychology (pp. 31-42). San Francisco: Freeman.

We will discuss connectionist units and their properties in relation both to Bayesian computations and physiology of real neurons.