How gestures embody meaning
Download
1 / 45

How Gestures Embody Meaning - PowerPoint PPT Presentation


  • 689 Views
  • Updated On :

How Gestures Embody Meaning. David McNeill Departments of Psychology & Linguistics University of Chicago Presented by Jim Goss. Midwest Faculty Seminar April 24-6, 2008. What is the reason for looking at speech-synchronized gestures?.

Related searches for How Gestures Embody Meaning

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'How Gestures Embody Meaning' - KeelyKia


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
How gestures embody meaning l.jpg

How Gestures Embody Meaning

David McNeill

Departments of Psychology & Linguistics

University of Chicago

Presented by Jim Goss

Midwest Faculty Seminar April 24-6, 2008


What is the reason for looking at speech synchronized gestures l.jpg
What is the reason for looking at speech-synchronized gestures?

  • It provides us with a way to examine how language, thought, and communication interface

    • Speech-synch’d gestures are spontaneously created by the speaker as she/he speaks.

    • The shape and space of gestures reflect speaker’s thought processes.

    • Gestures are timed with speech to create thought processes that are integrated with speech

  • Gesture/speech synchrony provides a window onto language and mind.


The gesture continuum categories from kendon l.jpg
The Gesture Continuum gestures?(categories from Kendon)

Spontaneous Gesticulation (Mode 1) —> Language-slotted (Mode 2) —> Pantomime (Mode 2) —> Emblems (Mode 2) —> Signs

Mode 1=unwitting gestures, Mode 2=gestures intended as symbols (due to S. Duncan)

Gestures differ in timing and convention. We focus on gesticulation. As one goes from left to right on the continuum we find that:

  • The obligatory presence of speech declines

  • Language like properties increase

  • Individually defined (idiosyncratic) form-meaning pairs are replaced by socially regulated signs


Types of gestures l.jpg
Types of Gestures gestures?

  • Beats

  • Deictic

  • Iconic

  • Metaphoric


Opposing modes of creating meaning l.jpg
Opposing Modes of Creating Meaning gestures?

  • Properties of gesture

    • Imagistic

    • Synthetic

      • Any gesture can be a combination of the four types of gestures

    • Global/Holistic

      • The parts only make sense in reference to the whole

    • Instantaneous

    • Largely unconscious

  • Gestures can convey complementary or supplementary information to speech


  • Opposing modes of creating meaning6 l.jpg
    Opposing Modes of Creating Meaning gestures?

    • Properties of formal language

      • Analytic

        • English is a analytic language that depends a great deal on word order, but all languages are analytic to some degree

      • Combinatoric-

        • rules for combination (grammar) can produce an infinite variety of utterances

      • Linear

      • Based on socially-conventionalized rules and forms

      • Form/meaning duality


    The cartoon stimulus l.jpg
    The cartoon stimulus gestures?

    • Two scenes from “Canary Row”, a WB Tweety and Sylvester cartoon from about 1950.

    • In each scene, Sylvester tries to reach Tweety by means of a drainpipe (Tweety is perched in a window some floors up from the street) - Sylvester climbs on the outside of the pipe first, then on the inside. Things don’t work out so well for Sylvester.


    Gesture phases regular and slow motion l.jpg
    Gesture Phases - Regular and Slow Motion gestures?

    “and Tweety runs and gets a bowling

    prep hold stroke hold

    ba[ll and dropsit down

    retraction

    the drain pipe]”

    Viv. drops it down

    prestr. poststr.

    regular

    vivrunslow.mpg

    slow


    Meanings of gesture phases in gets a bowling b all and drop s it do wn the drain pipe l.jpg
    Meanings of Gesture Phases in gestures?“gets a bowling b[all and dropsit down the drain pipe]”

    • Preparation - onset of idea (during “ball”)

    • Prestroke hold - targeting the stroke

    • Stroke - the core content of the utterance in which two representational modes co-exist(“it do-”)

    • Poststroke hold - completes stroke content (“-wn”)

    • Holds ensure the integrity of the idea and maintain the synchrony of co-expressive speech and gesture


    Timing l.jpg
    Timing gestures?

    • The key phase is the stroke - the meaningful part of the gesture.

    • Strokes coincide with the co-expressive linguistic segment(s) more than 90% of the time.

    • The remainder of strokes anticipate these segments, often because of speech disfluency.

    • The preparation is the onset of the gesture and routinely leads the co-expressive speech.


    Thus at the instant of the stroke two simultaneous forms of thinking l.jpg
    Thus, at the Instant of the Stroke, Two Simultaneous Forms of Thinking

    With the combination of speech and gesture, thought is both:

    • Instantaneous - distributed

    • Global - analytic

    • Ephemeral - unchanging

      Also, they are co-expressive which is essential

      The semiotic differences for the same idea fuel dynamic thinking-for-speaking through instability- the dialectic need to work itself out

      In this dialectic, properties of gesture images combine with simultaneous linguistic properties; this is a ‘Growth Point’.


    Speech and gesture unbreakable bond l.jpg
    SPEECH AND GESTURE - Unbreakable bond of Thinking

    The tight combination of gesture and speech is a psycholinguistic fact of some importance. It creates the conditions of an imagery-language dialectic. It is shown in a variety of phenomena:

    • DAF (Delayed Auditory Feedback)

    • Memory

    • Stuttering

    • Gestures of the blind

    • Deaffrentation- loss of proprioception


    Gestures and speech behave as single tightly coupled entity l.jpg
    Gestures and Speech Behave as Single Tightly Coupled Entity of Thinking

    Non-DAF: this speaker doesn’t gesture at all, possibly because of hyper-elaboration of speech - repeated embeddings:

    • Then he tries, then he proceeds-->to clamber up-->inside a drainspout--> which eventually-->will lead up to

      DAF (delayed auditory feedback): suddenly we find gestures and speech and gesture in synch despite the interference

    Non DAF

    DAF


    This speech gesture mismatch is preserved between interlocuters l.jpg
    This Speech/Gesture Mismatch is Preserved Between Interlocuters

    • Memory doesn’t differentiate between speech and gesture

      Experimenter Subject

    Says “bounces,” gestures path only

    Says “rolls” but gestures “bounce”


    Speech gesture mismatches l.jpg
    Speech/Gesture Mismatches Interlocuters

    • Gesture blends during deception (Amy Franklin)

    • Speech/gesture mismatches when solving problems (Susan Goldin-Meadow)


    Gestures and speech are tightly coupled gestures inoculate for clinical stuttering r mayberry l.jpg
    Gestures and Speech are Tightly Coupled -Gestures Inoculate for Clinical Stuttering(R. Mayberry)

    Stroke interrupted if underway

    No stuttering at stroke onset

    Gestures are held (interrupted) during stuttering, but synchrony is preserved


    Gestures and speech are tightly coupled the blind iverson goldin meadow l.jpg
    Gestures and Speech are Tightly Coupled - The Blind for Clinical Stuttering(Iverson & Goldin-Meadow)

    • Congenitally blind speakers also produce gestures with speech

    • Have never seen gestures.

    • Produce gestures regardless of whether listener can see or not.


    Slide18 l.jpg
    IW for Clinical Stuttering

    • Lost his proprioception at age 19

    • He must plan out every movement ahead of time and monitor it visually

    • When he cannot see his hands his instrumental and co-speech gestures

      • Maintain morphokinetic accuracy

      • Poor topokinetic control

  • Without vision speech/gesture synchrony and co-expressiveness are preserved


  • Language as prosthesis l.jpg
    Language as Prosthesis for Clinical Stuttering

    Gestures Embodying Meaning


    Closing the loop l.jpg
    Closing the Loop for Clinical Stuttering

    • Mead’s Loop

      “Gestures become significant symbols when they implicitly arouse in the individual making them the same response which they explicitly arouse in other individuals”

      - G.H. Mead

    • Thought-Language-Hand Link

      • With mirror neurons Mead’s loop co-opts areas of the brain where actions are orchestrated- BA 44, BA 45

      • Significances other than the significances of the actions themselves co-opt the system


    The combination of speech and gesture favors the formation of gesture linguistic units l.jpg
    The combination of speech and gesture favors the formation of gesture-linguistic units

    • The word ‘unit’ is used in Vygotsky’s sense:

      • “By a unit we mean a product of analysis which, in distinction from elements, possesses all the basic properties of a whole. Further, these properties must be a living portion of the unified whole which cannot be broken down further...”

    • We will consider a theory - the ‘growth point’ theory - of what comprises a unit in this field that retains the basic properties of a whole with “living properties”


    The growth point hypothesis l.jpg
    The Growth Point Hypothesis of gesture-linguistic units

    • The growth point is proposed as the minimal unit of the dialectic between imagery and language.

    • A growth point is a package that has both linguistic categorial and imagistic components.

    • Growth points are inferred from the totality of communicative events with special focus on speech-gesture synchrony and co-expressivity.

    • By focusing on these properties we bring out the duality of cognition


    Why it is called a growth point l.jpg
    Why it is called a “growth point” of gesture-linguistic units

    • A GP is meant to be the initial idea unit out of which speech-gesture organization emerges

    • The GP is a theoretical unit in which utterance formulation is viewed as a process of mental change over very short time spans

    • The GP addresses the concept that there is a definite starting point for an idea unit


    To identify a gp l.jpg
    To identify a GP of gesture-linguistic units

    • We must combine three kinds of information:

      • Co-expressive speech and gesture

      • Synchronous speech and gesture

      • The field of oppositions that makes this co-expressive, synchronous combination the point of newsworthy differentiation in context

    • Only by combining all three can we hypothesize the existence of a GP


    Case study of a gp l.jpg
    Case Study of a GP of gesture-linguistic units

    • Earlier “it down” example:

      • “and Tweety runs and gets a bowling bal[l and dropsit down the drain] pipe”

    • Thanks to the holds, the stroke coincides exaclty with “it down”, which was co-expressive with it.

    • But this is odd from a grammatical vantage point, because “it down” is not a constituent of the sentence.

    • Yet it was the core idea unit, the GP.

    Viv. drops it down

    regular

    slow

    Vivianslow.mpg


    Gp properties l.jpg
    GP Properties of gesture-linguistic units

    • The inferred GP is the image of downward movement plus the linguistic content of the “it” (i.e., the bowling ball) and the path particle “down”.

    • In the GP, the downward thrusting image is categorized as a bowling ball on a downward path.

    • Does not include the verb “drops” - preparation phase continued through it and ended with a prestroke hold until the verb was safely over.


    Gp properties27 l.jpg
    GP Properties of gesture-linguistic units

    • Both image and linguistic categorial contentare important

    • Imagery is important:

      • Grounds the linguistic categories in a specific visuospatial context.

      • Provides the GP with the property of ‘chunking’- speaking is a chunk of linguistic output organized around the presentation of an image. This chunking is realized in a phonologically centered, gestured ‘pulse’.


    Gp properties28 l.jpg
    GP Properties of gesture-linguistic units

    • Linguistic categorization is also important:

      • Brings the imageinto the system of categories of the language

      • GP is an image with a foot in the door of language - visuospatial thinking (instantaneous, global, imagistic) is tied to the categorial content of language (inherently segmented with combinatoric potential)


    How the gp is formed differentiation l.jpg
    How the GP is Formed -Differentiation of gesture-linguistic units

    • The GP is formed by differentiation

      • It is an emergent point or “figure” differentiated from a context or (back)ground.

      • The GP is the point of contextual weight and newsworthiness, the significant departure in the immediate context of speaking.

      • It corresponds to the ‘psychological predicate’ in Vygotsky and is the point of maximum ‘communicative dynamism’


    The psychological predicate key to explaining differentiation and context l.jpg
    The Psychological Predicate - Key to explaining differentiation and context

    • Psychological predicate - not necessarily a grammatical predicate.

    • Marks a significant departure from the immediate context. Vygotsky examples:

    • Implies this context as background.

    • The speaker shapes the background in a certain way, in order to make possible the intended significant contrast within it.


    Timing reflects differentiation two climbs up examples s duncan l.jpg
    Timing Reflects Differentiation differentiation and contextTWO ‘CLIMBS UP’ EXAMPLES - S. Duncan

    • Shows that timing is differentiated relative to context

    • Climbs 1 “[he climbs upthe…]” Climbs 2 “climbs [up in through the]”


    Context fields of opposition l.jpg
    ‘Context’ = ‘Fields Of Opposition’ differentiation and context

    • Background is, in part, a mental construction – the speaker's effort to construct a meaning

    • This is a model of meaningfulness – the joint product of context and GP is a new meaning which exists only in relation to a background.

    • Use the “field of oppositions” and “significant (‘newsworthy’) contrast” to refer to this constructed meaning.


    It down plus image is a psychological predicate l.jpg
    ‘It down’ Plus Image is a Psychological Predicate differentiation and context

    • Pre- and poststroke holds show that “it down” was targeted by the gesture- not an accidental combination:

      • The continued preparation and then prestroke hold ‘drops it d..’ shows that the stroke was withheld until “it”

      • The poststroke hold ‘…it down’ shows that the stroke was actively preserved through “down” even though movement had ceased.

    • What was the field of oppositions? This question can be approached empirically via the concept of a catchment


    The catchment gestural access to field of oppositions l.jpg
    The Catchment - Gestural Access to Field of Oppositions differentiation and context

    • The catchment gives us a way to observe the effective context (field of oppositions).

    • A catchment is a thematic segment in gesture form.

    • Catchments suggest the current fields of oppositions during ongoing discourse.

      • A possible communicative role of gesture at the catchment level = guiding the listener to the current field of oppositions


    Definition l.jpg
    Definition differentiation and context

    • A catchment is recognized from recurrences of gesture form features over a stretch of discourse

    • It’s a kind of thread of consistent dynamic visuospatial imagery running through the discourse

    • The logic is that discourse themes produce gestures with recurring features; these recurrences give rise to the catchment.

    • Thus, working backwards, the catchment offers clues to the discourse themes in the text with which it co-occurs.


    Viv s battle plan catchments l.jpg
    Viv.’s Battle Plan Catchments differentiation and context

    Identified from hand use:

    • C1 - 1 handed = Sylvester as a solo force

    • C2- 2 similar handed = the bowling ball as an antagonistic force

    • C3 - 2 different handed = the relative spatial positions of the bowling ball and Sylvester inside the pipe


    Context in the case study l.jpg
    Context in the Case Study differentiation and context

    Viv’s Battle Plan (full scene)

    (1) he tries going [up the insid][e of the drainpipe and]

    (2)Tweety Bird runs and gets a bowling ba[ll and dropsit down the drai]npipe

    (3) [and as he's coming up]

    (4) [and the bowling ball's coming d][own

    (5) he ssswallows it]

    (6) [and he comes out the bottom of the drai][npipe

    (7) and he's got this big bowling ball inside h][im

    (8) [and he rolls on down] [into a bowling all][ey

    (9) and then you hear a stri]ke


    Catchment interpretations 1 l.jpg

    C1 differentiation and contextOne-handed gestures- items (1)and(6) - ties together references to Sylvester as a solo force.

    C2Two-handed symmetrical gestures- items (2), (7), (8) and (9) - groups descriptions where the bowling ball is the antagonist, the dominant force. The 2-handed symmetric gesture form highlights the shape of the bowling ball.

    C3Two-handed asymmetrical gestures - items (3), (4) and (5) - groups items in which the bowling ball and Sylvester are equals differing only in their direction of motion.

    (1) he tries going [[up the insid][e of the drainpipe and]]- (6) [and he comes out the bottom of the drai][npipe

    (2) Tweety Bird runs and gets a bowling ba[ll and dropsit down the drai]npipe

    (7) and he's got this big bowling ball inside h][im

    (8) [[and he rolls on down] [into a bowling all]][ey

    (9) and then you hear a sstri]ke

    (3) [[and as he's coming up]

    (4) [[and the bowling ball's coming d]][own

    (5) he ssswallows it]

    Catchment Interpretations- 1


    Catchment interpretations 2 l.jpg
    Catchment Interpretations -2 differentiation and context

    • The verb “drops”, therefore, was excluded from the GP. This despite the fact that the gesture showed ‘dropping.’

    • We can explain this as follows. The verb describes what Tweety did, not what the bowling ball did (it went down). “Drops” was not part of significant contrast involving the bowling ball - the core idea at (2) was the bowling ball and its action, not Tweety and his.

      • The detailed synchrony of speech and gesture thus incorporated the context at the moment of speaking.


    Unpacking rest of the story l.jpg
    Unpacking - Rest of the Story differentiation and context

    • Unpacking is resolution of an imagery-language dialectic with further mental growth over short intervals.

    • The term describes extracting the implications of the GP through further meaning generation.

    • Unpacking is not just mechanical translation into language.

    • It requires its own generation of meanings possibly with its own context - beyond the GP’s context.


    Some implications l.jpg
    Some Implications differentiation and context

    • The sentence didn’t start from the verb - “drops” emerged in the unpacking as a result of new meaning - a caused-motion construction.

      • The verb is not necessarily the core of the dynamic process of evolving meaning and utterance.

    • The original GP and the unpacking were both guided by contextual contrasts - each by its own.

    • Meaning develops during unpacking - the opposite concept from an ‘input’ - keep going until a well-formed stopping point is reached.


    Morals l.jpg
    Morals differentiation and context

    • An utterance, even though seemingly self-contained, contains content from outside of its own structure.

      • This other content ties the utterance to the context at the level of thinking.

      • It is this fact - by no means unique or peculiar - that conflicts with the axioms of modularity yet fits the schema outlined in the GP.

    • That multiple contexts collaborate to form one grammatical structure implies that a sense of grammatical form enters into utterances in piecemeal and oblique ways.


    Living properties from coordinative structures l.jpg
    ‘Living properties’ from coordinative structures differentiation and context

    • The question is: does the idea of the bowling ball as an antagonistic force moving downwards automatically take care of features such as size (largish), placement (upper), direction (down), and motive force (agenthood)?

    • The key: ideas or significances can be attractors of coordinative structures; coordinative structures zero in on meaning. Features of gesture arise during the action itself. Once a gesture has been created usually we can identify features of form that carry meanings, but these are the outcomes of the gesture, not sources.


    General conclusions l.jpg
    General conclusions differentiation and context

    • Language is inseparable from imagery.

    • It has a dual reality, as Wundt (and Saussure, about the same time) saw - both instantaneous (here: image) and successive (here: linguistic and social) - combined in a dialectic that fuels change.

    • This observation has a number of implications:

      • Speaking is in part imagery

      • Imagery and language jointly comprise the growth point

      • This GP itself isn’t speakable - it is an idea unit - unpacked by finding further meanings that are true to the point of high communicative dynamism and context, and situate the GP in a construction

    • Across many languages, imagery and catchments differ, but GPs and unpacking–embodiments–seem universal.


    Thank you l.jpg
    Thank You! differentiation and context

    Clockwise: Werner, Husserl, Merleau-Ponty, Bakhtin, Jakobson, IW, Saussure, Vygotsky, and Wundt – in different ways, the GP hypothesis has drawn on all these individuals (and more).


    ad