The icsi berkeley neural theory of language project
This presentation is the property of its rightful owner.
Sponsored Links
1 / 28

The ICSI/Berkeley Neural Theory of Language Project PowerPoint PPT Presentation


  • 90 Views
  • Uploaded on
  • Presentation posted in: General

The ICSI/Berkeley Neural Theory of Language Project. Graduate Students Leon Barrett (CS) *Johno Bryant (CS) *Nancy Chang (CS) Ellen Dodge (Ling) Michael Ellsworth (Ling) Joshua Marker (Ling) *Eva Mok (CS) Shweta Narayan (Ling) *Steve Sinha (CS) Alumni Terry Regier (UCB Ling)

Download Presentation

The ICSI/Berkeley Neural Theory of Language Project

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


The icsi berkeley neural theory of language project

The ICSI/BerkeleyNeural Theory of Language Project

  • Graduate Students

    • Leon Barrett (CS)

    • *Johno Bryant (CS)

    • *Nancy Chang (CS)

    • Ellen Dodge (Ling)

    • Michael Ellsworth (Ling)

    • Joshua Marker (Ling)

    • *Eva Mok (CS)

    • Shweta Narayan (Ling)

    • *Steve Sinha (CS)

  • Alumni

    • Terry Regier (UCB Ling)

    • David Bailey (Google)

    • Andreas Stolcke (ICSI, SRI)

    • Dan Jurafsky (Stanford Ling)

    • Olya Gurevich (Powerset)

    • Benjamin Bergen (U. Hawaii Ling)

    • Carter Wendelken (UCB)

    • Srini Narayanan (ICSI, UCB)

    • Gloria Yang (UTD)

  • Principal investigators

    • Jerome Feldman (UCB,ICSI)

    • George Lakoff (UCB Ling)

    • Srini Narayanan (UCB,ICSI)

    • Lokendra Shastri (now India)

  • Affiliated faculty

    • Chuck Fillmore (ICSI)

    • Eve Sweetser (UCB Ling)

    • Rich Ivry (UCB Psych)

    • Lisa Aziz-Zadeh (USC)


Unified cognitive science

Unified Cognitive Science

Neurobiology

Psychology

Computer Science

Linguistics

Philosophy

Social Sciences

Experience

Take all the Findings and Constraints Seriously


The icsi berkeley neural theory of language project

Constrained Best Fit in Nature

inanimateanimate

framing, compromise

society, politics


Brains computers

1000 operations/sec

100,000,000,000 units

10,000 connections/

graded, stochastic

embodied

fault tolerant

evolves

learns

1,000,000,000 ops/sec

1-100 processors

~ 4 connections

binary, deterministic

abstract, disembodied

crashes frequently

explicitly designed

is programmed

Brains ~ Computers


Fast brain slow neurons

Fast Brain ~ Slow Neurons

Mental Connections are Active Neural Connections

There is No Erasing in the Brain


Constraints on connectionist models

Constraints on Connectionist Models

100 Step Rule

Human reaction times ~ 100 milliseconds

Neural signaling time ~ 1 millisecond

Simple messages between neurons

Long connections are rare

No new connections during learning

Developmentally plausible


Connectionist models in cognitive science

Connectionist Models in Cognitive Science

Structured

PDP

Hybrid

Neural

Conceptual

Existence

Data Fitting

Fast MappingSkill Learning

Not discussed in meeting


Triangle nodes and mccullough pitts neurons

A

B

C

Triangle nodes and McCullough-Pitts Neurons?

A

B

C


Representing concepts using triangle nodes

Representing concepts using triangle nodes


The icsi berkeley neural theory of language project

Functionalism

In fact, the belief that neurophysiology is even relevant to the functioning of the mind is just a hypothesis. Who knows if we’re looking at the right aspects of the brain at all. Maybe there are other aspects of the brain that nobody has even dreamt of looking at yet. That’s often happened in the history of science. When people say that the mental is just the neurophysiological at a higher level, they’re being radically unscientific. We know a lot about the mental from a scientific point of view. We have explanatory theories that account for a lot of things. The belief that neurophysiology is implicated in these things could be true, but we have very little evidence for it. So, it’s just a kind of hope; look around and you see neurons: maybe they’re implicated.

Noam Chomsky 1993, p.85


Embodiment

Embodiment

Of all of these fields, the learning of languages would be the most impressive, since it is the most human of these activities. This field, however, seems to depend rather too much on the sense organs and locomotion to be feasible.

Alan Turing (Intelligent Machines,1948)

Continuity Principle of the American Pragmatists


The icsi berkeley neural theory of language project1

Learning early constructions (Chang, Mok)

The ICSI/BerkeleyNeural Theory of Language Project

ECG


Ideas from cognitive linguistics

Ideas from Cognitive Linguistics

  • Embodied Semantics (Lakoff, Johnson, Sweetser, Talmy

  • Radial categories (Rosch 1973, 1978; Lakoff 1985)

    • mother: birth / adoptive / surrogate / genetic, …

  • Profiling (Langacker 1989, 1991; cf. Fillmore XX)

    • hypotenuse, buy/sell (Commercial Event frame)

  • Metaphor and metonymy (Lakoff & Johnson 1980, …)

    • ARGUMENT IS WAR, MORE IS UP

    • The ham sandwich wants his check.

  • Mental spaces (Fauconnier 1994)

    • The girl with blue eyes in the painting really has green eyes.

  • Conceptual blending (Fauconnier & Turner 2002, inter alia)

    • workaholic, information highway, fake guns

    • “Does the name Pavlov ring a bell?” (from a talk on ‘dognition’!)


Simulation based language understanding

Cafe

Simulation-based language understanding

Utterance

“Harry walked to the cafe.”

Constructions

Analysis Process

General Knowledge

Simulation Specification

SchemaTrajectorGoal

walkHarrycafe

Belief State

Simulation


Psycholinguistic evidence

Psycholinguistic evidence

  • Embodied language impairs action/perception

    • Sentences with visual components to their meaning can interfere with performance of visual tasks (Richardson et al. 2003)

    • Sentences describing motion can interfere with performance of incompatible motor actions (Glenberg and Kashak 2002)

    • Sentences describing incompatible visual imagery impedes decision task (Zwaan et al. 2002)

  • Simulation effects from fictive motion sentences

    • Fictive motion sentences describing paths that require longer time, span a greater distance, or involve more obstacles impede decision task (Matlock 2000, Matlock et al. 2003)


Neural evidence mirror neurons

Neural evidence: Mirror neurons

  • Gallese et al. (1996) found “mirror” neurons in the monkey motor cortex, activated when

    • an action was carried out

    • the same action (or a similar one) was seen.

  • Mirror neuron circuits found in humans (Porro et al. 1996)

  • Mirror neurons activated when someone:

    • imagines an action being carried out (Wheeler et al. 2000)

    • watches an action being carried out (with or without object) (Buccino et al. 2000)


Active representations

walker at goal

energy

walker=Harry

goal=home

Active representations

  • Many inferences about actions derive from what we know about executing them

  • Representation based on stochastic Petri nets captures dynamic, parameterized nature of actions

  • Used for acting, recognition, planning, and language

  • Walking:

  • bound to a specific walker with a direction or goal

  • consumes resources (e.g., energy)

  • may have termination condition(e.g., walker at goal)

  • ongoing, iterative action


Learning verb meanings david bailey

Learning Verb MeaningsDavid Bailey

A model of children learning their first verbs.

Assumes parent labels child’s actions.

Child knows parameters of action, associates with word

Program learns well enough to:

1) Label novel actions correctly

2) Obey commands using new words (simulation)

System works across languages

Mechanisms are neurally plausible.


The icsi berkeley neural theory of language project

System Overview


The icsi berkeley neural theory of language project

Learning Two Senses of PUSH

Model merging based on Bayesian MDL


Ntl manifesto

NTL Manifesto

  • Basic Concepts are Grounded in Experience

    • Sensory, Motor, Emotional, Social,

  • Abstract and Technical Concepts map by Metaphor to more Basic Concepts

  • Neural Computation models all levels


Simulation based language understanding1

Analyzer:

Discourse & Situational Context

Simulation based Language Understanding

Constructions

Utterance

incremental,

competition-based, psycholinguistically plausible

Semantic Specification:

image schemas, frames, action schemas

Simulation


The icsi berkeley neural theory of language project

Pragmatics

Semantics

Syntax

Morphology

Phonology

Phonetics

“Harry walked into the cafe.”


The icsi berkeley neural theory of language project

Pragmatics

UTTERANCE

Semantics

Syntax

Morphology

Phonology

Phonetics

“Harry walked into the cafe.”


Embodied construction grammar

Embodied Construction Grammar

  • Embodied representations

    • active perceptual and motor schemas(image schemas, x-schemas, frames, etc.)

    • situational and discourse context

  • Construction Grammar

    • Linguistic units relate form and meaning/function.

    • Both constituency and (lexical) dependencies allowed.

  • Constraint-based

    • based on feature unification (as in LFG, HPSG)

    • Diverse factors can flexibly interact.


Embodiment and grammar learning

Embodiment and Grammar Learning

Paradigm problem for Nature vs. Nurture

The poverty of the stimulus


Embodiment and grammar learning1

Embodiment and Grammar Learning

Paradigm problem for Nature vs. Nurture

The poverty of the stimulus

The opulence of the substrate

Intricate interplay of genetic and environmental, including social, factors.


Embodied construction grammar ecg formalizing cognitive linguisitcs

Embodied Construction GrammarECG(Formalizing Cognitive Linguisitcs)

  • Linguistic Analysis

  • Computational Implementation

    • Test Grammars

    • Applied Projects – Question Answering

  • Map to Connectionist Models, Brain

  • Models of Grammar Acquisition


  • Login