The ICSI/Berkeley Neural Theory of Language Project

1. Learning early constructions (Chang, Mok) The ICSI/Berkeley Neural Theory of Language Project ECG

3. Connectionist Model of Word Recognition (Rumelhart and McClelland)

4. Constraints on Connectionist Models 100 Step Rule Human reaction times ~ 100 milliseconds Neural signaling time ~ 1 millisecond Simple messages between neurons Long connections are rare No new connections during learning Developmentally plausible

5. Can we formalize/model these intuitions • What is a neurally plausible computational model of spreading activation that captures these features. • What does semantics mean in neurally embodied terms • What are the neural substrates of concepts that underlie verbs, nouns, spatial predicates?

6. o u t p u t y { 1 if net > 0 0 otherwise w0 I0 = 1 w1 w2 wn . . . i1 i2 in i n p u t i Abstract Neuron

7. Computing with Abstract Neurons • McCollough-Pitts Neurons were initially used to model • pattern classification • size = small AND shape = round AND color = green AND location = on_tree => unripe • linking classified patterns to behavior • size = large OR motion = approaching => move_away • size = small AND direction = above => move_above • McCollough-Pitts Neurons can compute logical functions. • AND, NOT, OR

8. What are the drawbacks of each representation? Distributed vs Localist Rep’n

9. What happens if you want to represent a group? How many persons can you represent with n bits? 2^n What happens if one neuron dies? How many persons can you represent with n bits? n Distributed vs Localist Rep’n

10. Sparse Distributed Representation

11. … … Visual System • 1000 x 1000 visual map • For each location, encode: • orientation • direction of motion • speed • size • color • depth • Blows up combinatorically!

12. Coarse Coding info you can encode with one fine resolution unit = info you can encode with a few coarse resolution units Now as long as we need fewer coarse units total, we’re good

13. Feature 1e.g. Orientation Y-Orientation Y X X-Orientation Y-Dir X-Dir Coarse-Fine Coding Coarse in F2, Fine in F1 but we can run into ghost “images” G G Coarse in F1, Fine in F2 Feature 2e.g. Direction of Motion

14. Connectionist Models in Cognitive Science Structured PDP Hybrid Neural Conceptual Existence Data Fitting

15. Computing other relations • The 2/3 node is a useful function that activates its outputs (3) if any (2) of its 3 inputs are active • Such a node is also called a triangle node and will be useful for lots of representations.

16. A B C Triangle nodes and McCullough-Pitts Neurons? A B C

17. “They all rose” triangle nodes: when two of the neurons fire, the third also fires model of spreading activation

18. Spreading activation and feature structures • Parallel activation streams. • Top down and bottom up activation combine to determine the best matching structure. • Triangle nodes bind features of objects to values • Mutual inhibition and competition between structures • Mental connections are active neural connections

19. Representing concepts using triangle nodes

20. Feature Structures in Four Domains

22. Categories and concepts- introduction CS182/Ling109/CogSci110 Spring 2008

23. Lecture Outline • Categories • Basic Level • Prototype Effects • Neural Evidence for Category Structure • Aspects of a Neural Theory of concepts • Image Schemas • Description and types • Behavioral Experiment on Image Schemas • Event Structure and Motor Schemas

24. The WCS Color Chips • Basic color terms: • Single word (not blue-green) • Frequently used (not mauve) • Refers primarily to colors (not lime) • Applies to any object (not blonde)

25. Concepts • What Concepts Are: Basic Constraints • Concepts are the elements of reason, and • constitute the meanings of words and linguistic expressions.

26. Concepts Are: • Universal: they characterize all particular instances; e.g., the concept of grasping is the same no matter who the agent is or what the patient is or how it is done. • Stable. • Internally structured. • Compositional. • Inferential. They interact to give rise to inferences. • Relational. They may be related by hyponymy, antonymy, etc. • Meaningful. • Not tied to the specific word forms used to express them.

27. Concepts: Traditional Theory • The Traditional Theory • Reason and language are what distinguish human beings from other animals. • Concepts therefore use only human-specific brain mechanisms. • Reason is separate from perception and action, and does not make direct use of the sensory-motor system. • Concepts must be “disembodied” in this sense.

28. The neural theory Human concepts are embodied. Many concepts make direct use of sensory-motor, emotional, and social cognition capacities of our body-brain system. • Many of these capacities are also present in non-human primates. • Continuity Principle of Am. Pragmatists

29. Classical vs prototype model of categorization • Classical model • Category membership determined on basis of essential features • Categories have clear boundaries • Category features are binary • Prototype model • Features that frequently co-occur lead to establishment of category • Categories are formed through experience with exemplars

30. Prototype theory • Certain members of a category are prototypical – or instantiate the prototype • Categories form around prototypes; new members added on basis of resemblance to prototype • No requirement that a property or set of properties be shared by all members • Features/attributes generally gradable • Category membership a matter of degree • Categories do not have clear boundaries

31. Prototype theory • Certain members of a category are prototypical – or instantiate the prototype Category members are not all equal a robin is a prototypical bird, but we may not want to say it is the prototype, rather it instantiates (manifests) the prototype or ideal -- it exhibits many of the features that the abstract prototype does “It is conceivable that the prototype for dog will be unspecified for sex; yet each exemplar is necessarily either male or female.” (Taylor)

32. Prototype theory 3. No requirement that a property or set of properties be shared by all members -- no criterial attributes • Category where a set of necessary and sufficient attributes can be found is the exception rather than the rule • Labov household dishes experiment • Necessary that cups be containers, not sufficient since many things are containers • Cups can’t be defined by material used, shape, presence of handles or function

33. Prototype theory • Wittgenstein’s examination of game • Generally necessary that all games be amusing, not sufficient since many things are amusing • Board games, ball games, card games, etc. have different objectives, call on different skills and motor routines - categories normally not definable in terms of necessary and sufficient features

34. Prototype theory • What about mathematical categories like odd or even numbers? Aren’t these sharply defined? • (Armstrong et al.) Subjects asked to assign numbers a degree of membership to the categories odd number or even number  3 had a high degree of membership, 447 and 91 had a lower degree (all were rated at least ‘moderately good’)

35. Categories - who decides? • Embodied theory of meaning- categories are not pre-formed and waiting for us to behold them. Our need for categories drives what categories we will have • Basic level categories - not all categories have equal status. The basic level category has demonstrably greater psychological significance.

36. Basic-level categories

37. chair desk chair easy chair rocking chair furniture lamp desk lamp floor lamp table dining room table coffee table Superordinate Basic Subordinate

38. Categories & Prototypes: Overview Furniture Superordinate • Three ways of examining the categories we form: • relations between categories (e.g. basic-level category) • internal category structure (e.g. radial category) • instances of category members (e.g. prototypes) Sofa Desk Basic-Level Category leathersofa fabricsofa L-shapeddesk Receptiondisk Subordinate

39. Basic-level -- Criteria • Perception – • overall perceived shape • single mental image • fast identification

40. Basic-level -- Criteria • Perception • Function – motor program for interaction

41. Basic-level -- Criteria • Perception • Function • Words – • shortest • first learned by children • first to enter lexicon

42. Basic-level -- Criteria • Perception • Function • Communication • Knowledge organization – • most attributes are stored at this level

43. Perception: similar overall perceived shape single mental image (gestalt perception) fast identification Function: general motor program Communication: shortest most commonly used contextually neutral first to be learned by children first to enter the lexicon Knowledge Organization: most attributes of category members stored at this level Basic-Level Category What constitutes a basic-level category?

44. Other Basic-level categories • Objects • Colors • Motor-routines

45. Concepts are not categorical

46. Mother • The birth model The person who gives birth is the mother • The genetic model The female who contributes the genetic material is the mother • The nurturance model The female adult who nurtures and raises a child is the mother of the child • The marital model The wife of the father is the mother • The genealogical model The closest female ancestor is the mother (WFDT Ch.4, p.74, p.83)

47. Radial Structure of Mother Geneticmother Stepmother The radial structure of this category is defined with respect to the different models Unwedmother Adoptivemother CentralCase Surrogatemother Birthmother Biologicalmother Naturalmother Fostermother

48. Marriage • What is a marriage? • What are the frames (or models) that go into defining a marriage? • What are prototypes of marriage? • What metaphors do we use to talk about marriages? • Why is this a contested concept right now?

49. Concepts and radial categories Concepts can get to be the "prototype" of their category in various ways. • Central subcategory (others relate to this) • Amble and swagger relate to WALK • Shove relates to PUSH • Essential (meets a folk definition: birds have feathers, beaks, lay eggs) • Move involves change of location. • Typical case (most are like this: "sparrow") • Going to a conference involves air travel. • Ideal/anti-ideal case (positive social standard: "parent"); anti-ideal case (negative social standard: "terrorist") • Stereotype (set of attributes assumed in a culture: "Arab") • Salient exemplar (individual chosen as example)

50. Category Structure • Classical Category: • necessary and sufficient conditions • Radial Category: • a central member branching out to less-central and non-central cases • degrees of membership, with extendable boundary • Family Resemblance: • every family member looks like some other family member(s) • there is no one property common across all members (e.g. polysemy) • Prototype-Based Category • Essentially-Contested Category (Gallie, 1956) (e.g. democracy) • Ad-hoc Category (e.g. things you can fit inside a shopping bag)