1 / 55

How Speakers' Eye Movements Reflect Spoken Language Generation

How Speakers' Eye Movements Reflect Spoken Language Generation. Zenzi M. Griffin Department of Psychology. Intro to Cognitive Science University of Texas at Austin October 14, 2011. ?. M E S S A G E. TIME. “ blah blah blah… ”. What the eyes tell us about speaking:.

maik
Download Presentation

How Speakers' Eye Movements Reflect Spoken Language Generation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. How Speakers' Eye Movements Reflect Spoken Language Generation Zenzi M. Griffin Department of Psychology Intro to Cognitive Science University of Texas at Austin October 14, 2011

  2. ? M E S S A G E TIME “blah blah blah…”

  3. What the eyes tell us about speaking: • when speakers prepare names for objects • which object they intend to name • possibly when they commit to an order of mention or syntactic structure

  4. Griffin & Bock (2000) Psych Science

  5. green  current fixation blue  earlier fixation size  fixation duration Movie Legend

  6. Speakers gazed at referents in the second before naming them The girl is kicking the boy Griffin & Bock (2000) Psych Science

  7. Long gazes followed order of mention • Not left-right • or salient-less salient • or agent-patient • or big-small • or human-nonhuman

  8. Difficulty in selecting a name varies with number & strength of competing candidates Word selection "TV" or "television" "baby"

  9. FrequencyLow High Codability Medium High

  10. SE Isolated object naming • Latencies from a large norming study (Griffin & Huitema, 1999) • Huge, additive effects of codability & frequency.

  11. SE Object categorization task • Is it bigger than a piece of paper? • Codability effect ns • Frequency effect ns • Interaction ns NO YES

  12. “The clock and the TV are above the needle” Griffin (2001) Cognition

  13. Griffin (2001) Cognition

  14. “The|clockand theTVare above theneedle.” Griffin (2001) Cognition

  15. Gaze duration correlated with name difficulty Griffin (2001) Cognition

  16. Gaze duration correlated with name difficulty Griffin (2001) Cognition

  17. 1st noun Gaze time on B Freq: Codability: MEDIUM HIGH Effects of preparing 2nd name only during speech, not before See also work by Antje Meyer & colleagues on effects of word frequency, phonological priming, image degradation, etc. Griffin (2001) Cognition

  18. Prepare then speak Until ready to describe The crib & the limo are above the needle time

  19. Gaze reflects difficulty when preparing words in advance Disfluent, Multiple names Fluent, Multiple names Fluent, One name Griffin (in prep)

  20. Gaze time on B Freq: Codability: MEDIUM HIGH MEDIUM HIGH Extemporaneous vs. Prepared speech 1st noun Griffin (in prep)

  21. Not much looking at locations while articulating prepared speech “The A and the B are above the C”

  22. Gazes Reflect Word Preparation • When describing scenes, speakers gaze at objects just before naming them. • Time spent gazing at an object reflects how difficult it is to prepare a name. • Gazes reflect when words are prepared, not articulated.

  23. Why look?

  24. Gazing at intended object could facilitate name retrieval leaf

  25. Gazing could prevent interference from other objects uh leaf

  26. No eye movements Fixation point

  27. "finger" for "thumb" (half speed then real time)

  28. Timing of gaze: before error = before correct Griffin (2004) Psych Science

  29. (real time)

  30. Speakers gazed longer at agents before calling them by inaccurate, similar names even when only fluent trials considered  Griffin & Oppenheimer (2006) JEPLMC

  31. Timing of gaze offsets was similar Griffin & Oppenheimer (2006) JEPLMC

  32. Speakers even gazed at referent when a better match to name was visible (real time) Griffin & Oppenheimer (2006) JEPLMC

  33. Speakers also gazed at referent when using a novel word "blick" (half speed then real time) Griffin & Oppenheimer (2006) JEPLMC

  34. Name-related eye movements aren't necessary & do not seem to facilitate name preparation • Speakers can describe simple scenes without moving their eyes. • Gazes before speech errors are similar to those before correct names. • Speakers gaze at referents before lying about them.

  35. How early must speakers commit to order-of-mention or syntactic structure?

  36. “The A and the B are above the C” A B C

  37. “The A and the B are above the C”

  38. “The|womanis throwing adogabone.” Referent for

  39. Order of mention/ structure choice a bone to a dog." "A woman throws theme-recipient a doga bone." recipient-theme

  40. Director/Confederate Matcher/Participant Is a man drinking coffee? No match. 1st presentation

  41. Director/Confederate Matcher/Participant Is a tall woman throwing a bone to a dog? Yes, a tall woman is throwing a bone to a dog. 2nd presentation: Biasing Question

  42. Biasing question like having pre-planned order: no uncertainty about order of mention "Yes, a woman is throwing a bone to a dog"

  43. (real time)

  44. Director/Confederate Matcher/Participant Is a tall woman swimming? No, a tall woman is throwing a bone to a dog. 2nd presentation: Neutral Question

  45. Emergent argument ordering: "Yes, a woman is throwing a bone to a dog"

  46. Dependent measure: gaze shifts "Yes, a woman is throwing a bone to a dog"

  47. (real time, half speed)

  48. More shifts between theme & recipient during articulation of 1st noun phrase shifts + SE Match, a woman is throwing a bone to a dog Griffin, Garton, & Mouzon (in prep)

More Related