1 / 26

Brain related semantics

Brain related semantics. CSCTR – Session 8 Dana Retov á. Grounded cognition. Rejects the standard view that amodal symbols represent knowledge in semantic memory Cognition shares the same mechanisms with perception, action and introspection Simulation

tia
Download Presentation

Brain related semantics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Brain related semantics CSCTR – Session 8 Dana Retová

  2. Grounded cognition • Rejects the standard view that amodal symbols represent knowledge in semantic memory • Cognition shares the same mechanisms with perception, action and introspection • Simulation • A core form of computation in the brain • Reenactment of perceptual, motor and introspective states acquired during experience • As experience occurs, the brain captures the states across modalities and integrates them with a multimodal representation stored in memory

  3. Mental imagery vs. amodal symbols • Modal representation and imagery representing knowledge • Epicurus, Kant, Reid • Behaviorists • Imagery not sufficiently scientific • Cognitivists • Amodal representation (feature lists, semantic networks, frames) • Elegant and powerful formalisms for representing knowledge • Could be implemented in AI

  4. Problem with traditional AI • No evidence supports the presence of amodal symbols in cognition • Grounding problem • Traditional theories fail to explain how cognition interfaces with perception and action • Problem where the brain stores amodal symbols • How is it consistent with neural principles of computation?

  5. Grounded cognition theories • Simulation • Situated action • Bodily states • Modal representations are central to knowledge

  6. Theories of Grounded Cognition • Cognitive Linguistics Theories • Lakoff & Johnson (1980, 1999) • Abstract concepts are grounded metaphorically in embodied and situated knowledge • Theories of situated action • Gibson (1979) • Role of environment in shaping cognitive mechanisms • Coupling of perception and action during goal achievement • Social interaction • Research in robotics • Dynamic systems as preferred architecture • Fixed representations do not exist

  7. Cognitive Simulation Theories • Memory theories • Glenberg (1997) • Memory is not just passive storage of information • Perception of relevant objects triggers affordances for action stored in memory • Reasoning about future actions relies on remembering affordances while suppressing perception of the environment • Social simulation theories • How we represent the mental states of other people • We use simulations of our own minds • To feel someone else’s pain we simulate our own pain • Mirror neurons • Empathy, imitation, social coordination

  8. Cognitive Simulation Theories • Perceptual Symbol Systems • Synthetic approach • Implements standard symbolic functions • Type-token binding, inference, productivity, recursion, propositions • A single multimodal representation system in the brain supports diverse forms of simulation across different cognitive processes • High-level perception • Working memory • long-term memory • conceptual knowledge • Convergence zone architecture (Damasio 1989, Simmons & Barsalou 2003) • Single representation system controlled by multiple simulation mechanisms

  9. Convergence zone • ensemble of neurons within which many feedforward/feedback loops make contact. • It 1) receives forward projections from cortical regions located in the connectional level immediately below • 2) Sends reciprocal backward projections to the originating cortices • 3) Sends forward projections to cortical regions in the next connectional level; and • 4) Receives projections from heterarchically placed cortices and from subcortical nuclei in thalamus, basal forebrain, and brainstem.

  10. Empirical Evidence for Grounded Theories • Perceptual Inference • Perception-action coordination • Perception of space • Memory • Implicit memory • Explicit memory • Working memory • Conceptual processing

  11. Perceptual inference • Vision and motion • Goldstone (1995) • Association between shape and color • Hansen et al. (2006) • Object’s natural color distort achromatic perception of the object toward the opponent color • Motion (Freyd 1987, Shiffrar & Freyd 1990,1993) • Subjects simulate the visual trajectory beyond its actual trajectory • Also during apparent motion, simulation of possible action shapes the perception of motion • Speech (Warren 1970) : http://www.youtube.com/watch?v=UlJs24j3i8E • Lexical knowledge produces simulation in speech perception – missing phoneme simulation

  12. Perception-action coordination • Simulations of potential actions • Viewing an object grasped with a precision or power grip (grape vs. hammer) produces a simulation of the appropriate action (Tucker & Ellis 1998) • This is affected by • object’s orientation (Symes et al. 2007) • Size (Glover et al. 2004) • Simulations of both grasping and functional actions (Bub et al 2007) • Also name triggers simulation (Tucker & Ellis 2004) • Hearing a word activates the articulatory action associated with producing it (Pulvermuller 2006) • Perceived effort affects visual perception (Proffitt 2006) • Being tired from a run makes a hill look steeper • Carrying a heavy pack makes a path look longer

  13. Perception-action coordination • Motor simulations • Motor system constructs a feed-forward simulation of the action to guide and correct it (Grush 2004, Wolpert et al. 1999) • Generating visual inferences about the anticipated actions of perceived agents (Wilson & Knoblich 2005)

  14. Perception of space • The perception of space is shaped by the body and it’s relation to the environment • Locating objects has various difficulty along different axes • Vertical • easiest • Front-back • Left-right • Most difficult – bodily cues are lacking • Perception of near space extends with arm length (Longo & Laurenco 2007)

  15. Implicit memory • Results form simulation of perceptual memories • Repetition priming is strongest when the modalities of the memory and stimulus match (e.g. auditory) (Kirsner et al., 1989) • Repetition priming is strongest when perceptual details of the memory and stimulus match (e.g. orientation, size,…) (Jacoby&Hayman, 1987) • Imagining produces repetition priming similar to actual perception (Roediger&Blaxton, 1987)

  16. Explicit memory • Multimodal simulations of previous episodes • Important for constructing future events • The retrieval of a word stimulates the modal operations performed at encoding (Wheeler et al. 2000) • Visual areas become active during retrieval following visual study while auditory areas become active following auditory study • Greater activation in modal areas when remembering something that really occurred than false memories (Slotnick & Schacter 2004)

  17. Working memory • Absent stimulus is stored in working memory (Levy & GoldmanRakic 2000) • To maintain working memory, neurons in the frontal lobes maintain a simulation of the absent stimulus in the modal system that processed it originally. • Some frontal regions maintain working memories of objects, other spatial locations, motion, textures, etc. • They are highly selective for the specific features • Visual imagery in working memory simulates visual processing (Finke 1989, Kosslyn 1980,…) • Analogously, motor imagery, auditory imagery, etc. • Mental rotation of visual objects -> motor simulations of making them turn (Richter et al. 2000)

  18. Conceptual processing • Behavioral evidence • When asked whether an property belongs to an objects subjects simulate properties to verify them (Solomon & Barsalou 2004) • Lesion evidence • Lesions in one modality – losing categories that rely on it for processing (Damasio 1994, …) • E.g. damage to visual areas – losing of ability to categorize animals (visual processing is dominant) • Damage to motor areas – categorization of tools • Neuroimaging evidence (Martin 2001, 2007) • When processing conceptual knowledge, brain areas representing properties are active

  19. Language comprehension • Perceptual simulation • Motor simulation • Affective simulation

  20. Language comprehension • Situation models • Evidence of modal representations in language comprehension • Spatial representation (Bower & Morrow 1990) • People confused pictures with text (Intraub & Hoffman 1992) • Replacing words with pictures did not disrupt sentence processing (Potter 1986)

  21. Perceptual simulation • Subjects read a sentence and then processed a picture that either matched or mismatched something implied by the sentence • “The ranger saw the eagle in the sky” • Picture of an eagle – wings outstretched or folded • Visual irrelevant information interferes with spatial inferences (Fincher-Keifer 2001)

  22. Motor simulation • Verbs for head, arm and leg actions produce head, arm and leg simulation in the respective areas of the motor system (Pulvermuller 2005) • When action to make a response is consistent with text meaning, the response is quicker (Glenberg & Kaschak 2003) • Subjects simulate corresponding motion through space (Richardson et al. 2003) • Positive/negative valence (Meier & Robinson) • High/low power (Schubert 2005)

  23. Affective simulation • Subjects’ faces configured according to sentences with emotional content (Havas 2007) • When facial emotion matched the content comprehension was better • Gesture • Producing gestures helps speakers retrieve words whose meaning are related to the gestures (Krauss 1998) • Also help listeners comprehend what speaker says • Children can gesture before speaking

  24. Thought • Physical reasoning • Gear, pulleys • Driven by spatial simulation • Sketchy, not holistic and detailed • Abstract reasoning • Content effects • Reasoning about time using space domain - metaphors

  25. Social Cognition • Embodiment effects • Activating elderly stereotype causes people to walk slowly and to perform lexical decision slowly (Dijksterhuis & Bargh 2001) • Engaging the smiling musculature produces positive affect (Strack et al. 1988) • Social mirroring • Individual differences in the ability to simulate other people’s mental states correlate with rated empathy (Jackson et al. 2005) • Development • Mirroring, object permanence…

  26. Questions?

More Related