1 / 51

Can We Simulate Negation? The Simulation Effects of Negation in English Intransitive Sentences

Meylysa Tseng Jung-Hee Kim Benjamin Bergen University of Hawai ’ i at Manoa BLS 2006. Can We Simulate Negation? The Simulation Effects of Negation in English Intransitive Sentences. Thanks to. Amy Schafer William O ’ Grady Kathryn Wheeler Cognitive Linguistics Research Group

bonner
Download Presentation

Can We Simulate Negation? The Simulation Effects of Negation in English Intransitive Sentences

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Meylysa Tseng Jung-Hee Kim Benjamin Bergen University of Hawai’i at Manoa BLS 2006 Can We Simulate Negation?The Simulation Effects of Negation in English Intransitive Sentences

  2. Thanks to • Amy Schafer • William O’Grady • Kathryn Wheeler • Cognitive Linguistics Research Group at University of Hawaii

  3. Mental imagery • When you hear The dolphin soared, You may be mentally picturing (simulating) a visual scene • Mental imagery comes from our perceptual or motor experience • Mental imagery should resemble our perceptual or motor experience

  4. Imagery in Language • Imagery is crucial in language use (Zwaan 1999, Barsalou 1999, Glenberg & Robertson 2000, Bergen & Chang 2005) • The idea: • Language is associated with concrete perceptual or motor experiences • These experiences are recreated through imagery during language understanding • Such perceptual or motor imagery is activated in language understanding • The dolphin soared (visual imagery) • I pushed the door (motor imagery)

  5. Dimensions of Visual imagery used in language processing • Orientation of objects(Stanfield & Zwaan 2001) • Shape of objects(Zwaan et al. 2002) • Spatial location(Richardson et al. 2003, Lindsay 2003) • Upwards or downwards motion (Bergen 2005)

  6. Bergen 2005: Up or Down visual imagery • What parts of the sentences yield the simulation effects? • The mule climbed (Up-Verb) • The glass fell (Down-Verb) • The sky darkened (Up-Noun) • The ground shook (Down-Noun) • It was concrete Subject Nouns and Verbs denoting up/down motion or location • Understanding sentences with subject nouns or verbs that entail upness or downness visual imagery selectively interfered with the visual processing of objects in the same parts of the visual field

  7. Today: Simulating Negation • Negation presents an especially difficult case for a simulation-based approach • You can’t imagine what’s not happening • How can negation be captured or represented in a simulation-based account?

  8. Questions remain • Understanding negated sentences also engages mental simulation? • Up/down visual imagery is activated in comprehending ‘The mule didn’t climb’or ‘The sky didn’t darken’?

  9. Simulation-based account of negation comprehension • Negation can be understood as a process “Negation is implicitly encoded in the simulation processes that are undertaken when comprehending a negative sentence” (Kaup and Lüdtke 2005) • Negation involves two simulations • Simulation of the counterfactual situation • Suppression • Simulation of the factual situation

  10. This study: Negation & Up/Down Visual Imagery • This study replicated Bergen 2005 • We hypothesize: mental simulation is engaged in understanding negation • We test: Up/Down visual imagery is activated by such up/down-associated verbs and nouns even when embedded in the negated sentences • Stimuli: up/down-associated negative sentences • We negated Bergen’s literal sentences with up or down-associated nouns or verbs The mule didn’t climb(Up-Verb) The glass didn’t fall (Down-Verb) The sky didn’t darken (Up-Noun) The ground didn’t shake (Down-Noun)

  11. Predictions • In processing negated sentences, if up/down associated noun and verbs induce up/down visual imagery especially in the counterfactual simulation stage, there should be an interaction effect between noun/verb location and object location in a visual object categorization task (2) Consistent with the proposed stages of negation simulation, we would see a shift of simulation from counterfactual to factual over time

  12. Time course – 200ms, 500ms • 200ms – the initial counterfactual simulation • Up/Down visual imagery was activated at 200ms time delay in processing up/down-associated ‘positive’ sentences (Bergen 2005) • 500ms– the later factual simulation • ‘Negation seemed to effect some time between 500 and 1,000 msec from the time the negative sentence had been read’ (Hasson and Gluksberg 2004:20) • Prediction: • in processing up/down-associated ‘negative’ sentences, up/down visual imagery would be activated at 200ms and there would be a change in the simulation at 500ms

  13. Method: replicate Bergen 2005 • fixation cross (1s) • Ss heard a sentence • Either 200 ms or 500 ms delay • Ss saw a circle or square on the screen: up, down, right, or left (for 200 msec) • Critical trials: up or down. • The same number of (non-critical) sentences followed by an object on the left or right • Ss categorize visual object • Subjects press circle (z) or square (x) • A comprehension question was interleaved, to ensure listening for content

  14. Example

  15. + + +

  16. “The mule didn’t climb.” (heard over the headphone)

  17. + + +

  18. “The glass didn’t fall.” (heard through headphone)

  19. Result: Negated Verb Sentences • Up: The mule didn’t climb • Down: The glass didn’t fall

  20. Negated Verb Sentences (200 ms) *

  21. Negated Verb Sentences (500 ms) *

  22. Result: Negated Noun Sentences • Up: The rainbow didn’t fade • Down: The ground didn’t shake • “The sky didn’t darken” was eliminated due to having average means higher than 2.5 SD from the mean

  23. Negated Noun Sentences (200 ms) * *

  24. Negated Noun Sentences (500 ms) * * *

  25. Summary of Result * * *

  26. Summary of Effect

  27. Findings: Prediction 1 • There was a significant interaction effect between up/down noun/verb location and object location • Up/down nouns or verbs induced visual simulation even when embedded in negated sentences • This thus supports our main hypothesis that negation induces mental simulation

  28. Findings: Prediction 2 • Same interaction effect at both 200ms and 500ms, which results from the counterfactual simulation • We found an evidence for a simulation of the counterfactual situation shortly after processing of negative sentences • We found no evidence for factual simulation • We found no evidence of a shift in the simulation in negation processing

  29. No evidence for factual simulation • If the glass didn’t fall,… • We didn’t give clear factual situation • Without context, factual situation is relatively underspecified • This lead to not clear factual simulation stage

  30. Positive Negative 200 ms interference facilitation 500 ms facilitation Bergen 2005 vs This study • These two studies showed that up/down visual • imagery is activated in processing negated • sentences as well as positive sentences with • up/down nouns or verbs • Interestingly, • our interaction was in the opposite direction.

  31. Bergen 2005 vs. This study(Interference vs. Facilitation) * * * * *

  32. Discussion: Interference vs. Facilitation • Interference and facilitation effects in imagery studies are both simulation effects. • The two effects basically result from the use of the same neural structures to understand language and perform a perception • Two factors (Kaschak et al. 2004) • Extent of Time overlap between the two processing involved (sentence processing and visual processing) • Extent of Integratability of the two tasks “the extent to which the perceptual stimulus (circle or square) can be integrated into the simulation constructed of the content of the sentence”

  33. How the two factors work • If sentence processing and visual processing are, • Sequential– Facilitation as a form ofsemantic priming • Simultaneous– Interference or Facilitation depending on Integratability  Integratable– Facilitation  Non-integratable– Interference • Interference – perform two different (non-integratable) tasks at the same time • Facilitation – perform two related (integratable) tasks at the same time

  34. Interference in Bergen’s positive sentences • Simultaneous processing (200ms time overlap) • Less integratability in simulating positive sentences • The sky darkened • Positive sentence tells us exactly what happened. It is fully informative about the location to be simulated • Such detailed image construction is not readily integratable with a circle or a square visual processing • Such specifically simulated image blocks visual processing • Therefore, interference effect

  35. Facilitation in our negative sentences • Simultaneous processing (200ms overlap) • Relatively more integratability involved in simulating negative sentence • The sky didn’t darken • A single negative sentence with no context is less informative (less specific or less detailed) in its conveyed meaning • Perceivers would simulate less well-defined location imagery and thus construct a less-detailed simulation • This less-detailed image constructionanda circle or a squarevisual processing can be relatively integratable • Thus, this less-detailed simulated image doesn’t block visual processing • Therefore, facilitation effect

  36. Summary of discussion • Positive sentence processing • Fully informative • Well-defined and detailed image construction • Low integratability • Interference • Negative sentence processing • Less informative • Less well-defined and less-detailed image construction • Makes the two tasks relatively integratable • Facilitation

  37. Discussion: Verb effects significant only at 200ms * * *

  38. Weaker simulation effect for verbs at 500ms • Scope of negation • The sky [didn’t darken] (Up-noun) • The glass [didn’t fall] (Down-verb) • For nouns, the direction is not changed under negation: less subject variability • For verbs, the direction is changed under negation: more subject variability • This would make verbs exhibit relatively weaker simulation at 500ms

  39. More variability in Verbs (200ms)

  40. Summary • Found an interaction effect of noun/verb direction with visual object recognition in comprehending negated intransitive sentences • Negated sentences involve a less detailed simulation, causing facilitation • Imagery is involved in the comprehension of negated sentences • Found an evidence for a simulation of the counterfactual situation • Found no evidence for factual simulation • Found no evidence for a shift in the simulation of negation

  41. Future Study • How to capture a shift in the process of negation simulation? • To capture the further clear counterfactual simulation, test sentences at an earlier time course (0ms) • To capture the clear factual simulation, give a clear factual situation, since without context, the factual situation is relatively underspecified • The glass didn’t fall. It rose.

  42. Conclusions • Spatial imagery of varying degrees of detail is automatically or unconsciously engaged in language understanding • Language about a visual scene induces visual simulation, even in negated sentences • How we actually process negated sentences crucially depends upon the detailed semantics of sentential constituents, such as subject nouns and verbs • Embodied view of meaning • “[…] the core of our conceptual systems is directly grounded in perception, body movement, and experience of a physical and social character.” (Lakoff 1987, p.xiv)

  43. Thank you

  44. References • BARSALOU, LAWRENCE W. 1999. Perceptual symbol systems. Behavioral and Brain Sciences, 22.577-660. • BERGEN, BENJAMIN K., NARAYAN, SHWETA and FELDMAN, JEROME. 2003. Embodied verbal semantics: evidence from an image-verb matching task. Paper presented at Cognitive Science Conference. . • GLENBERG, ARTHUR M. and KASCHAK, MICHAEL P. 2002. Grounding Language in Action. Psychonomic Bulletin & Review, 9.558-65. • Hasson, Uri and Sam Glucksberg. 2004. Does Understanding Negation Entail Affirmation? An Examination of • Negated Metaphors. manuscript. University of Chicago. • Just, M. A., and P. A. Carpenter. 1976. Eye fixations and cognitive processes. Cognitive Psychology, 8: 441-480. • Kaschak, Michael P, Rolf A. Zwaan, Mary Aveyard and Richard H. Yaxley. (in press). Perception of auditory motion affects language processing. Cognitive Science. • Kaschak, Michael P., Carol J. Madden, David J. Therriault, Richard H. Yaxley, Mark Aveyard, Adrienne A. Blanchard & Rolf A. Zwaan. 2004. Perception of motion affects language processing. Cognition 94:B79-B89. • Kaup, Barbara. 2001 Negation and its impact on the accessibility of text information. Memory & Cognition, 29, 860-967. • KAUP, BARBARA and ZWANN, ROLF A. 2003. Effects of negation and situational presence on the accessibility of text information. Journal of Experimental Psychology: Learning, Memory, and Cognition, 29.439-46. • Lindsay, Shane (2003) Visual priming of language comprehension. Masters Thesis. Department of Cognitive and Computing Sciences. University of Susses. • MacDonald, Maryellen C and Just, Marcel A. 1989. Changes in activation levels with negation. Journal of Experimental Psychology: Learning, Memory and Cognition. Jul;15(4):633-42. • NARAYAN, SHWETA, BERGEN, BENJAMIN K. and WEINBERG, ZACH. 2004. Embodied verbal semantics: Evidence from a lexical matching task. Paper presented at The Thirtieth Berkeley Linguistics Society Annual Meeting. • RICHARDSON, DANIEL C., SPIVEY, MICHAEL J., BARSALOU, LAWRENCE W. and MCRAE, KEN. 2003. Spatial representations activated during real-time comprehension of verbs. Cognitive Science.767-80. • ZWANN, ROLF A., STANFIELD, ROBERT A. and YAXLEY, RICHARD H. 2002. Language comprehenders mentally represent the shapes of objects. Psychological Science, 13.168-71. • ZWANN, ROLF A. 1999. Embodied cognition, perceptual symbols, and situation models. Discourse Processes, 28.81-88.

  45. Down-Verb The stone didn’t sink The chair didn’t topple The pipe didn’t drop The glass didn’t fall The cat didn’t descend Up-Verb The mule didn’t climb The lizard didn’t ascend The cork didn’t rocket The patient didn’t rise The dolphin didn’t soar Down-Noun The grass didn’t glisten The submarine didn’t fire The cellar didn’t flood The ground didn’t shake The shoe didn’t smell Up-Noun The sky didn’t darken The rainbow didn’t fade The roof didn’t creak The tree didn’t sway The ceiling didn’t crack Stimuli

  46. NOUN RTs

  47. VERB RTs

More Related