The Embodied Cognition Literature

1. 16th International Summer School in Cognitive Science, New Bulgarian University, Sofia, Bulgaria, 2009 The Embodied Cognition Literature …cont’d Michael J. Spivey Department of Cognitive Science University of California, Merced

2. Embodied Cognition Summer School in Cognitive Science Sensorimotor Influences in Cognition • Visual Properties Influence Concepts • Motor Properties Influence Vision • Language Influences Visual Properties • • • Language and Motor Influence Each Other • Your Entire Body Participates in Thought •

3. Embodied Cognition Summer School in Cognitive Science Sensorimotor Influences in Cognition Scheerer & Lyons (1957)

4. Skeletal motor Oro- Facial Oculo motor Haptic Concepts Visual Auditory Memory Linguistic Embodied Cognition Summer School in Cognitive Science Sensorimotor Influences in Cognition

5. Embodied Cognition Summer School in Cognitive Science Sensorimotor Influences in Cognition • Visual Properties Influence Concepts • Motor Properties Influence Vision • Language Influences Visual Properties • • • Language and Motor Influence Each Other • Your Entire Body Participates in Thought •

6. Embodied Cognition Summer School in Cognitive Science Barsalou, Solomon, & Wu (1999) List the features of a watermelon List the features of half a watermelon Large Green Sweet Red Sweet Seeds

7. Skeletal motor Oro- Facial Oculo motor Haptic Concepts Visual Auditory Memory Linguistic Embodied Cognition Summer School in Cognitive Science Sensorimotor Influences in Cognition

8. Embodied Cognition Summer School in Cognitive Science Sensorimotor Influences in Cognition • Visual Properties Influence Concepts • Motor Properties Influence Vision • Language Influences Visual Properties • • • Language and Motor Influence Each Other • Your Entire Body Participates in Thought •

9. Vision Coextensive with Action Summer School in Cognitive Science Stimulus-Response Compatibility Tucker & Ellis (1998)

10. Vision Coextensive with Action Summer School in Cognitive Science Stimulus-Response Compatibility Tucker & Ellis (1998)

11. Cognition Coextensive with Action Summer School in Cognitive Science Stimulus-Response Compatibility Tucker & Ellis (1998)

12. Cognition Coextensive with Action Summer School in Cognitive Science Stimulus-Response Compatibility Tucker & Ellis (1998)

13. Cognition Coextensive with Action Summer School in Cognitive Science The Importance of Being Somatosensory Input Bosbach, Cole, Prinz, Knoblich (2005)

14. Skeletal motor Oro- Facial Oculo motor Haptic Concepts Visual Auditory Memory Linguistic Embodied Cognition Summer School in Cognitive Science Sensorimotor Influences in Cognition

15. Embodied Cognition Summer School in Cognitive Science Sensorimotor Influences in Cognition • Visual Properties Influence Concepts • Motor Properties Influence Vision • Language Influences Visual Properties • • • Language and Motor Influence Each Other • Your Entire Body Participates in Thought •

16. Language Coextensive with Imagery Summer School in Cognitive Science Mental Simulations Stanfield & Zwaan (2001) John put the pencil in the cup. John put the pencil in the drawer.

17. Language Coextensive with Imagery Summer School in Cognitive Science Mental Simulations Zwaan, Stanfield & Yaxley (2001) The ranger saw the eagle in the sky. The ranger saw the eagle in the nest.

18. Language Coextensive with Imagery Summer School in Cognitive Science Mental Simulations Zwaan, Madden, Yaxley, & Aveyard (2004) The pitcher hurled the softball at you. You hurled the softball at the pitcher.

19. Language Coextensive with Imagery Summer School in Cognitive Science Mental Simulations Meteyard, Bahrami, & Vigliocco (2007) Downward: “bomb, drop, fall…” Control: “cook, sew, write…” Upward: “climb, rise, fly…”

20. Language Coextensive with Imagery Summer School in Cognitive Science Mental Simulations Meteyard, Bahrami, & Vigliocco (2007) Downward: “bomb, drop, fall…” Control: “cook, sew, write…” Upward: “climb, rise, fly…”

21. Language Coextensive with Imagery Summer School in Cognitive Science Mental Simulations Meteyard, Bahrami, & Vigliocco (2007) Downward: “bomb, drop, fall…” Control: “cook, sew, write…” Upward: “climb, rise, fly…”

22. Language Coextensive with Imagery Summer School in Cognitive Science Verb Image Schemas • An intimate relationship between language and space • Language is embodied and metaphorical (Gibbs, 2006; Lakoff, 1999) • Spatial registration hypothesis (Chatterjee, 2001; Coslett, 1999) • Language is not describable without reference to cognitive processes such as attentional mechanisms (Langacker, 1991; Talmy, 1983) • Perceptual symbol systems: cognitive representations are schematics, or simulations of perceptual experiences (Barsalou, 1999) Richardson, Spivey, Barsalou, & McRae (2003)

23. Language Coextensive with Imagery Summer School in Cognitive Science Verb Image Schemas - Offline Data Richardson, Spivey, Barsalou, & McRae (2003) • Forced Choice Norming Study

24. Language Coextensive with Imagery Summer School in Cognitive Science Verb Image Schemas Richardson, Spivey, Barsalou, & McRae (2003) • Free Drawing Norming Study

25. Language Coextensive with Imagery Summer School in Cognitive Science

26. Language Coextensive with Imagery Summer School in Cognitive Science

27. Language Coextensive with Imagery Summer School in Cognitive Science

29. Language Coextensive with Imagery Summer School in Cognitive Science Verb Image Schemas • Did subjects agree with each other? • 70% agreement in forced choice; Std errors about 10˚ in freeform. • Did the tasks tap the same representation? • Axis/aspect angle correlation, R = 0.71 • Did they agree with linguistic intuitions? • Significant effect of expected axis in both studies

30. Language Coextensive with Imagery Summer School in Cognitive Science Verb Image Schemas - Online Data Richardson, Spivey, Barsalou, & McRae (2003) • Experiment 1 - Visual discrimination • Hypothesis: comprehending a verb will activate a form of spatial representation that will be extended along either a horizontal or vertical axis, as shown by norming data. • Non-compatible mental imagery will interfere with the detection or discrimination of overlapping visual stimuli (Perky, 1910; Craver & Arterberry, 2001) • Participants presented with a spoken sentence containing a horizontal or vertical verb, and identified a rapidly flashed visual stimulus.

31. Language Coextensive with Imagery Summer School in Cognitive Science Verb Image Schemas - Online Data Richardson, Spivey, Barsalou, & McRae (2003) • Performance on the visual task was impaired when the visual stimulus appeared along the axis of the verb’s image schema

32. Language Coextensive with Imagery Summer School in Cognitive Science Verb Image Schemas - Online Data Richardson, Spivey, Barsalou, & McRae (2003) • Experiment 2 - Memory • Hypothesis: the spatial structure associated with a verb will influence the encoding of visual stimuli • Imagery improves memory (Paivio, 1969), and visual stimuli are remembered better when they are presented in the same spatial locations at presentation and test (Santa, 1977; Zimmer 1998) • Participants heard a sentence and saw pictures of the agent and patient of the sentence, presented centrally. At a later test stage, they were shown pairs of pictures and asked if they had been previously been presented together. At test, the pictures were presented in a horizontal or vertical alignment.

33. Language Coextensive with Imagery Summer School in Cognitive Science Verb Image Schemas - Online Data Richardson, Spivey, Barsalou, & McRae (2003)

34. Language Coextensive with Imagery Summer School in Cognitive Science Verb Image Schemas - Online Data Richardson, Spivey, Barsalou, & McRae (2003)

35. Language Coextensive with Imagery Summer School in Cognitive Science Verb Image Schemas - Online Data Richardson, Spivey, Barsalou, & McRae (2003) • At test, picture pairs were recognized faster if they were presented in the same orientation as the associated verb’s image schema

36. Language Coextensive with Imagery Summer School in Cognitive Science Verb Image Schemas Richardson, Spivey, Barsalou, & McRae (2003) • In two offline tasks, naïve participants categorized verbs as having a horizontal or vertical image schema • There was a good degree of consistency between tasks and between participants • The offline tasks provided norms for experiments involving online verb comprehension • In both a visual discrimination and a memory task, reaction times showed an interaction between the horizontal/vertical nature of the verb’s image schema, and the horizontal/vertical position of the visual stimuli • Such spatial effects of verb comprehension provide evidence for the perceptual-motor character of linguistic representations (see also Bergen, Lindsay, Matlock & Narayanan, 2007).

37. Skeletal motor Oro- Facial Oculo motor Haptic Concepts Visual Auditory Memory Linguistic Embodied Language Summer School in Cognitive Science Sensorimotor Influences in Cognition

38. Language Coextensive with Imagery Summer School in Cognitive Science Iconicity Affects Semantic Relatedness Judgments (Zwaan & Yaxley, 2003) attic basement

39. Language Coextensive with Imagery Summer School in Cognitive Science Iconicity Affects Semantic Relatedness Judgments (Zwaan & Yaxley, 2003) basement attic

40. Skeletal motor Oro- Facial Oculo motor Haptic Concepts Visual Auditory Memory Linguistic Embodied Language Summer School in Cognitive Science Sensorimotor Influences in Cognition

41. Embodied Language Summer School in Cognitive Science Sensorimotor Influences in Cognition • Visual Properties Influence Concepts • Motor Properties Influence Vision • Language Influences Visual Properties • • • Language and Motor Influence Each Other • Your Entire Body Participates in Thought •

42. Embodied Language Summer School in Cognitive Science (Spivey, Tanenhaus, Eberhard, & Sedivy, 2002) Ambiguous: “Put the apple on the towel in the box” “Put the apple that’s on the towel in the box” Unambiguous: One-Referent Context Two-Referent Context

43. Embodied Language Summer School in Cognitive Science Motor Constraints During Language (Chambers, Tanenhaus, & Magnuson, 2004) Pour the egg in the bowl onto the flour. Pour the egg that’s in the bowl onto the flour. Compatible Incompatible

44. Embodied Language Summer School in Cognitive Science Motor Constraints During Language (Chambers, Tanenhaus, & Magnuson, 2004)

45. Embodied Language Summer School in Cognitive Science Motor Facilitation During Language The Action-Sentence Compatibility Effect (Glenberg & Kaschak, 2002)

46. Embodied Language Summer School in Cognitive Science Motor Facilitation During Language The Action-Sentence Compatibility Effect (Glenberg & Kaschak, 2002) Close the drawer. Open the drawer.

47. Embodied Language Summer School in Cognitive Science Motor Facilitation During Language The Action-Sentence Compatibility Effect (Glenberg & Kaschak, 2002) Andy delivered the pizza to you. You delivered the pizza to Andy.

48. Embodied Language Summer School in Cognitive Science Motor Facilitation During Language The Action-Sentence Compatibility Effect (Glenberg & Kaschak, 2002) Liz told you the story. You told Liz the story.

49. Embodied Language Summer School in Cognitive Science Motor Facilitation During Language The Action-Sentence Compatibility Effect (Glenberg & Kaschak, 2002) (For variation in the temporal dynamics, see Kaschak & Borregine, 2008)

50. Embodied Language Summer School in Cognitive Science Motor Activation During Language (Pulvermüller, 1999)