1 / 21

Emergence of Semantics from Experience

Emergence of Semantics from Experience. Jay McClelland Department of Psychology and Center for Mind, Brain, and Computation Stanford University. 1. language. The Parallel Distributed Processing Approach to Semantic Cognition.

guinevere-mckinney
Download Presentation

Emergence of Semantics from Experience

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Emergence of Semantics from Experience Jay McClelland Department of Psychology andCenter for Mind, Brain, and ComputationStanford University 1

  2. language The Parallel Distributed Processing Approach to Semantic Cognition • Representation is a pattern of activation distributed over neurons within and across brain areas. • Bidirectional propagation of activation underlies the ability to bring these representations to mind from given inputs. • The knowledge underlying propagation of activation is in the connections. • Experience affects our knowledge representations through a gradual connection adjustment process 2

  3. dog goat hammer Distributed Representations:and Overlapping Patterns for Related Concepts dog goat hammer 3

  4. Emergence of Meaning in Learned Distributed Representations • Learned distributed representations that capture important aspects of meaning emerge through a gradual learning process in simple connectionist networks • The progression of learning captures several aspects of cognitive development: • Differentiation of Concepts • Illusory Correlations • Overgeneralization • And many other things 4

  5. 5

  6. The Rumelhart Model 6

  7. The Training Data: All propositions true of items at the bottom levelof the tree, e.g.: Robin can {grow, move, fly} 7

  8. Target output for ‘robin can’ input 8

  9. aj wij ai neti=Sajwij wki Forward Propagation of Activation 9

  10. Back Propagation of Error (d) aj wij ai di ~ Sdkwki wki dk ~ (tk-ak) Error-correcting learning: At the output layer: Dwki = edkai At the prior layer: Dwij = edjaj … 10

  11. 11

  12. 12

  13. Early Later LaterStill Experie nce 13

  14. Learning is sensitive to patterns of coherent covariation Coherent Covariation: The tendency for properties of objects to co-vary in clusters Figure shows attribute loadings on the principal dimensions of covariation. These capture: 1. Plants vs. animals 2. Birds vs. fish 3. Trees vs. flowers Same color = features that covary Diff color = anti-covarying features Why Does the Model Show Progressive Differentiation? 14

  15. Trajectories of Concept Representations During Differentiation 15

  16. Illusory Correlations 16

  17. A typical property thata particular object lacks e.g., pine has leaves An infrequent, atypical property 17

  18. Overgeneralization of Frequent Names to Similar Objects “goat” “tree” “dog” 18

  19. 19

  20. Other Applications of the Model • Expertise effects • Conceptual reorganization • Effects of language and culture • Effects of brain damage: • Loss of differentiation • Overgeneralization inobject naming • Illusory correlations camel swan 20

  21. Conclusion • We represent objects using patterns of activity over neuron-like processing units • These patterns depend on connection weights learned through experience • Differences in experience lead to differences in conceptual representations. 21

More Related