PSY 369: Psycholinguistics

PSY 369: Psycholinguistics Language Comprehension: Meaning beyond the word

Announcements • Homework 5 (a or b) deadline extended to April 5

Comprehension roadmap • This week: • Comprehension of Sentence Meaning • Traditional view: Propositions • New view: Embodied representations • Comprehension in Discourse

Propositions • How do we represent sentence meaning? • Propositions • Two or more concepts (arguments) with a relationship (relations) between them • Arguments – particular times, places, people, objects, etc. (nouns) • Relations - May be used for any kind (e.g., actions, attributes, positions, class memberships) • Smallest unit of knowledge that can be judged as true or false • Complex sentences consist of combinations of smaller propositional units

mouse agent cat patient relation bit Propositions A mouse bit a cat bit (mouse, cat) • How do we represent sentence meaning? • Propositions • Two or more concepts with a relationship between them • Can represent this within a network framework

Past Eat subject subject relation relation relation time Bread Slow Children Cold Deriving Propositions • More complex example: • Children who are slow eat bread that is cold • Slow children • Children eat bread • Bread is cold

Evidence for propositions • Bransford and Franks (1971, 1972) • Ratcliff and McKoon (1978)

Evidence for propositions • Bransford and Franks (1971, 1972) • Study-Recognition Test Task • Read sets of sentences, answered a question about each, later presented sentences and asked whether they were new (not previously presented) or old (previously presented) The girl broke the window on the porch. Broke what? The hill was steep. What was? The cat, running from the barking dog, jumped on the table. From what? The tree was tall. Was what? The old car climbed the hill. Did what? The cat running from the dog jumped on the table. Where? The girl who lives next door broke the window on the porch. Lives where? …

Evidence for propositions • Bransford and Franks (1971, 1972) • Study-Recognition Test Task All of the sentence came from 4 complex sentences. The full complex sentences were not presented at study. e.g., The girl who lives next door broke the large window on the porch … The girl lives next door. The girl broke the window. The window was on the porch. The window was large.

Evidence for propositions • Bransford and Franks (1971, 1972) • Study-Recognition Test Task Test: Old - same sentences that were presented at study New - based on the propositions in the complex sentence, but not presented at study (including the full complex sentences) Noncase- based on new propositions not based on the complex sentences (mixing of propositions across the different situations)

Evidence for propositions • Bransford and Franks (1971, 1972) • Study-Recognition Test Task Results: • False recognition of sentences that they were not previously presented with • Accurate rejections of noncases (different propositions) • Unable to distinguish between the old and new cases that came from the same complex sentences Yes 5 Yes 4   new  Yes 3   old  Yes 2 Yes 1  Recognition confidence  0 No 1  No 2  No 3 No 4 ★ No 5 fours threes twos ones noncases # of propositions

Evidence for propositions • Bransford and Franks (1971, 1972) • Study-Recognition Test Task Conclusions: • Participants remembered the basic meaning (propositions) • Participants spontaneously combined the propositions into larger units Yes 5 Yes 4   new  Yes 3   old  Yes 2 Yes 1  Recognition confidence  0 No 1  No 2  No 3 No 4 ★ No 5 fours threes twos ones noncases # of propositions

Evidence for propositions • Ratcliff and McKoon (1978) Tested 3 hypotheses: • Sentences stored as single unit • Sentences stored as connected propositions • Sentences stored verbatim

Evidence for propositions • Ratcliff and McKoon (1978) • Study-Recognition Test Task • Read sets of 4 unrelated sentences, then presented words (one at a time) and asked whether the words were in the preceding sentences • Dependent Measure: Priming - manipulated the order of the words at test The mausoleum that enshrined the tsar overlooked the square. The clutch failed to engage. The beggar forgave injustice but resented hunger. Satire hurt the incumbent. hunger Y Saturn N square Y mausoleum Y beetle N

Evidence for propositions • Ratcliff and McKoon (1978) • Involves two propositions: • P1 [OVERLOOK, MAUSOLEUM, SQUARE] • P2 [ENSHRINE, MAUSOLEUM, TSAR]. The mausoleum that enshrined the tsar overlooked the square. The clutch failed to engage. The beggar forgave injustice but resented hunger. Satire hurt the incumbent.

square mausoleum square clutch square tsar Within a single proposition Between two propositions in the same sentence Across sentences Evidence for propositions • Ratcliff and McKoon (1978) • Predictions (if Hypothesis 2: propositions are the memory representation): • If prime word from the same sentence, then should respond faster • If prime word from the same proposition, then should respond faster than if from a different proposition (within the same sentence) The mausoleum that enshrined the tsar overlooked the square. The clutch failed to engage.

**91 msec **111 msec Evidence for propositions • Ratcliff and McKoon (1978) • Predictions (if Hypothesis 2: propositions are the memory representation): • If prime word from the same sentence, then should respond faster • If prime word from the same proposition, then should respond faster than if from a different proposition (within the same sentence) • Results 671 580 560 square mausoleum square clutch square tsar Within a single proposition Between two propositions in the same sentence Across sentences

**20 msec Evidence for propositions • Ratcliff and McKoon (1978) • Predictions (if Hypothesis 2: propositions are the memory representation): • If prime word from the same sentence, then should respond faster • If prime word from the same proposition, then should respond faster than if from a different proposition (within the same sentence) • Results 671 580 560 square mausoleum square clutch square tsar Within a single proposition Between two propositions in the same sentence Across sentences

Evidence for propositions • Ratcliff and McKoon (1978) • Predictions (if Hypothesis 2: propositions are the memory representation): • If prime word from the same sentence, then should respond faster • If prime word from the same proposition, then should respond faster than if from a different proposition (within the same sentence) • Conclusions • Support the hypothesis that propositions are used to organize our memories of sentences

Inference in comprehension • Not all propositions come from the bottom-up • Elaboration - integration of new information with information from long term memory • Memory for the new information improves as it is integrated • Inferences - a proposition (or other representation) drawn by the comprehender • From LTM, not directly from the input

Inference in comprehension • Bransford, and colleagues (1972, 73) • We draw inferences in the course of understanding new events. • The inferences get encoded into our memory of the events. • e.g., drawing inferences of instruments

Inference in comprehension Saw (or heard): John was trying to fix the birdhouse. He was looking for the nail when his father came out to watch him and to help him do the work. • Bransford, and colleagues (1972, 73) Tested: John was using the hammer to fix the birdhouse when his father came out to watch him and to help him do the work. was not mentioned in the text, but was inferred Result: Participants falsely believed that they had heard this sentence So memory is not only of propositions in the original sentence, but may also include additional propositions that may have been inferred

Arguments against propositions • Propositions are symbolic and amodal • Referential problem: • Disconnected with outside world (symbols referring to other symbols) • Implementation problem: • Has been very difficult to develop a propositional parser • Lack of scientific productivity: • More work on what you can do with propositions than is there evidence of the psychological reality of propositions • Lack of a biological foundation: • How do biological (or neurological) data constrain propositions

More than propositions • Barclay (1973) • Subjects are presented with sequences of sentences that create a spatial array, like: • The bear is to the left of the moose. • The moose is to the right of the lion. • The moose is to the left of the cow. • The lion is to the left of the bear. • Array: lion < bear < moose < cow • Subjects are asked either to remember the sentences or to remember the order • Afterwards, people asked to remember the array also ‘remember’ sentences they didn’t actually hear, such as: • The bear is to the left of the cow (also faster to verify, Potts, 1974)

More than propositions • Bransford, Barclay, and Franks (1972) Hear: There is a tree with a box beside it, and a chair is on top of the box. The box is to the right of the tree. The tree is green and extremely tall. (a): The tree is to the left of the chair. (b): The chair is to the left of the tree. Recognition Task Result: correctly rejected (b) but accepted (a)

Mental model Mental Models Hear: There is a tree with a box beside it, and a chair is on top of the box. The box is to the right of the tree. The tree is green and extremely tall. • These experiments suggested that contexts are not simply lists of propositions, but that these propositions are somehow ‘merged’ to create `world-like’ representations • Johnson-Laird (1983): While processing, humans construct representations of worlds/situations related (identical with?) those built from perception

Embodiment in language • Embodied Representations • Many researchers assume that cognition is “embodied” (or “grounded”) rather than “abstract” (e.g., Barsalou, 2008) • Activates representations associated with the body and actions • Theoretical proposals from many disciplines • Linguistics: Lakoff, Langacker, Talmy • Neuroscience: Damasio, Edelman • Cognitive psychology: Barsalou, Gibbs, Glenberg, MacWhinney, Zwaan • Computer science: Steels, Feldman

Embodiment in language • Embodied Representations • Much of this work argues that language is embodied (e.g., Barsalou, 2008; Glenberg, 2008; Zwaan & Taylor, 2006) • Perceptual and motor systems play a central role in language production and comprehension (and meaning/concepts) • Words and sentences are usually grounded to perceptual, motoric, and emotional experiences. • In absence of immediate sensory-motor referents, words and sentences refer to mental models or simulations of experience

Embodiment in language • Embodied Representations • Simulation hypothesis (Gallese, 2008) • Simulation exploits some of the same neural structures activated during performance, perception, imagining, memory… • Language gives us enough information to simulate • Processing (producing or comprehending) walk involves the use of representations involved in the act of walking producing or comprehending “walk”

Embodiment in language • Evidence for Embodied Representations • Stanfied & Zwaan (2001) • Presented participants with sentences • John put the pencil in the cup. • John put the pencil in the drawer • See a picture and ask “does this describe what you read about?” • Results: faster at saying horizontal pencil with drawer and vertical pencil with cup

Embodiment in language • Evidence for Embodied Representations • Zwaan et al (2004) • Presented participants with a sentence • A: The pitcher hurled the softball at you. • B: You hurled the softball at the pitcher. • See two pictures and ask “are these pictures the same object” B A • Results: faster at saying ‘Yes’ when sentence matched the pictures (e.g., sentence A and pictures in A, if the ball is small and then gets big, it is coming towards you)

Embodiment in language • Evidence for Embodied Representations • Hauk et al (2004) • Do action words activate the motor cortex? fMRI study • 50 words from 3 semantic subcategories • (words matched for freq, length, imageability, etc.) • Rated for whether words reminded them of face, arm, or leg • Movement Comparison: moved their foot, finger, or tongue

Embodiment in language • Evidence for Embodied Representations • Hauk et al (2004) • Do action words activate the motor cortex? fMRI study • Action words did activate some of the same areas as the movements

Traditional Cognition = Computation Representation by propositions Propositions are abstract relations Embodiment of Meaning Cognition is serving perception and actions Representation = Patterns of possible bodily interactions with the world (lawfully related to the world) What an object, event, sentence means for you, is what you can do with the object, event, sentence. Summing up

Summing up • The results of sentence comprehension are meaning representations • Some debate over what these representations are • Whatever they are, they get integrated with each other and with existing knowledge from LTM

Discourse Psycholinguistics • Traditional Psycholinguistics • Determining what happens when we understand sentences • Broader View • How we resolve/understand sentences against the current discourse representation • Sentence comprehension is a process that anchors the interpretation of the sentence to the representation of the prior text

Processing Discourse • What is discourse? • The ways that we process (i.e., comprehend and remember) units of language larger than a sentence • Lectures, personal narratives, expository discourse • Units of analysis larger than a sentence • Applies to both spoken and written forms Discourse processing is sort of like syntactic processing – a way of organizing/connecting the different pieces in to larger chunks. Here the chunks are larger than sentences.

Bill and Ted traveled through time and space. Bill asked, “Dude, did you find him?” “Yeah, Bach was in the music store, but I haven’t found Abe yet. Dude, you should hear him play, he rocks!” “Excellent! Man, we’ve got to get these dudes back to school before we get there.”

Bill and Ted traveled through time and space. Bill asked, “Dude, did you find him?” “Yeah, Bach was in the music store, but I haven’t found Abe yet. Dude, you should hear him play, he rocks!” “Excellent! Man, we’ve got to get these dudes back to school before we get there.” • To whom does “him” refer to?

Bill and Ted traveled through time and space. Bill asked, “Dude, did you find him?” “Yeah, Bach was in the music store, but I haven’t found Abe yet. Dude, you should hear him play, he rocks!” “Excellent! Man, we’ve got to get these dudes back to school before we get there.” • To whom does “him” refer? Bach

Bill and Ted traveled through time and space. Bill asked, “Dude, did you find him?” “Yeah, Bach was in the music store, but I haven’t found Abe yet. Dude, you should hear him play, he rocks!” “Excellent! Man, we’ve got to get these dudes back to school before we get there.” • To whom does this “him” refer?

Bill and Ted traveled through time and space. Bill asked, “Dude, did you find him?” “Yeah, Bach was in the music store, but I haven’t found Abe yet. Dude, you should hear him play, he rocks!” “Excellent! Man, we’ve got to get these dudes back to school before we get there.” • To whom does this “him” refer? Bachagain

Bill and Ted traveled through time and space. Bill asked, “Dude, did you find him?” “Yeah, Bach was in the music store, but I haven’t found Abe yet. Dude, you should hear him play, he rocks!” “Excellent! Man, we’ve got to get these dudes back to school before we get there.” • To whom does this “him” refer? Bachagain Why not Abe?

Bill and Ted traveled through time and space. Bill asked, “Dude, did you find him?” “Yeah, Bach was in the music store, but I haven’t found Abe yet. Dude, you should hear him play, he rocks!” “Excellent! Man, we’ve got to get these dudes back to school before we get there.” • Huh!?

Bill and Ted traveled through time and space. Bill asked, “Dude, did you find him?” “Yeah, Bach was in the music store, but I haven’t found Abe yet. Dude, you should hear him play, he rocks!” “Excellent! Man, we’ve got to get these dudes back to school before we get there.” • Huh!? Oh yeah, they’re time travelers.

Characteristics of Discourse • Local Structure (microstructure): • The relationship between individual sentences • Cohesion • Coherence • Global Structure (macrostructure): • The relationship between the sentences and our knowledge of the world

Characteristics of Discourse • Local Structure (microstructure): • The relationship between individual sentences • Cohesion • Does the discourse “stick together”? • Interpretation of one sentence depends on other sentences? • Coherence • Does the passage make sense? • Logical consistency and semantic continuity?

Characteristics of Discourse • Cohesion: Interpretation of one sentence depends on other sentences • Referential Cohesion • “Dude, you should hear him play…” • Substitution Cohesion • “We’ve got to get these dudes back to …” • And many more • Ellipsis, conjunction, lexical cohesion (See pg 160 of textbook for examples) • The relationship between the referring expression and the antecedentcreate referential cohesion of discourse

PSY 369: Psycholinguistics