1 / 20

Statistical NLP Winter 2009

Statistical NLP Winter 2009. Lecture 12: Computational Psycholinguistics. Roger Levy. NLP techniques, human parsing. Our “ parsing” here is about Treebank parsing Now for a bit about human parsing! Techniques from NLP are still the foundation

teneil
Download Presentation

Statistical NLP Winter 2009

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Statistical NLPWinter 2009 Lecture12: Computational Psycholinguistics Roger Levy

  2. NLP techniques, human parsing • Our “parsing” here is about Treebank parsing • Now for a bit about human parsing! • Techniques from NLP are still the foundation • We’ll focus on rationalmodels of human sentence processing [rational = using all available information to make inferences] • incremental inference: understanding of and response to a partial utterance

  3. Incrementality and Rationality • Online sentence comprehension is hard • But lots of information sources can be usefully brought to bear to help with the task • Therefore, it would be rational for people to use all the information available, whenever possible • This is what incrementality is • We have lots of evidence that people do this often “Put the apple on the towel in the box.” (Tanenhaus et al., 1995)

  4. Anatomy of ye olde garden path sentence The horse raced past the barn fell. • It’s weird • People fail to understand it most of the time • People are more likely to misunderstand it than to understand it properly • “What’s a barn fell?” • The horse that raced past the barn fell • The horse raced past the barn and fell • Today I’m going to talk about three outstanding puzzles involving garden-path sentences

  5. Garden paths: What we do understand • We have decent models of how this sentence is not understood • Incremental probabilistic parsing with beam search (Jurafsky, 1996) • Surprisal (Hale, 2001; Levy, 2008): the disambiguating word fell is extremely low probability  alarm signal signals “this doesn’t make sense” to the parser • These models are based on rational use of evidential information (data-driven probabilistic inference) • Also compatible with gradations in garden-path difficulty (Garnsey et al., 1997; McRae et al., 1998)

  6. Hale, 2001; Levy, 2008; Smith & Levy, 2008: surprisal • Let the difficulty of a word be its surprisal given its context: • Captures the expectation intuition: the more we expect an event, the easier it is to process • Many probabilistic formalisms, including probabilistic context-free grammars,can give us word surprisals

  7. a man arrived yesterday 0.7 0.15 0.35 Total probability: 0.7*0.35*0.15*0.3*0.03*0.02*0.4*0.07= 1.8510-7 0.3 0.03 0.02 0.4 Algorithms by Lafferty and Jelinek (1992), Stolcke (1995) give usP(wi|context) from a PCFG 0.07 0.3 S  S CC S 0.15 VP  VBD ADVP 0.7 S  NP VP 0.4 ADVP  RB 0.35 NP  DT NN ...

  8. Surprisal and garden paths: theory • Revisiting the horse raced past the barn fell • After the horse raced past the barn, assume 2 parses: • Jurafsky 1996 estimated the probability ratio of these parses as 82:1 • The surprisal differential of fell in reduced versus unreduced conditions should thus be log2 83 = 6.4 bits *(assuming independence between RC reduction and main verb)

  9. Aside: These are way too high, but that’s because the grammar’s crude Surprisal and garden paths: practice • An unlexicalized PCFG (from Brown corpus) gets right monotonicity of surprisals at disambiguating word “fell” this is the key comparison; the difference is small, but in the right direction

  10. Garden Paths: What we don’t understand so well • How do people arrive at the misinterpretations they come up with? • What factors induce them to be more or less likely to come up with such a misinterpretation

  11. Outstanding puzzle:length effects • Try to read this: • Tom heard the gossip about the neighbors wasn’t true. • Compare it with this: • Tom heard the gossip wasn’t true. • Likewise: • While the man hunted the deer that was brown and graceful ran into the woods. • While the man hunted the deer ran into the woods. • The longer the ambiguous region, the harder it is to recover (Frazier & Rayner, 1987; Tabor & Hutchins, 2004) • Also problematic for rational models: effects of irrelevant information

  12. Memory constraints in human parsing • Sentence meaning is structured • The number of logically possible analyses for a sentence is at best exponential in sentence length • So we must be entertaining some limited subset of analyses at all times* *“Dynamic programming”, you say? Ask later.

  13. Dynamic programming • Exact probabilistic inference with context-free grammars can be done efficiently in O(n3) • But… • This inference requires strict probabilistic locality • Human parsing is linear—that is, O(n)—anyway • Here, we’ll explore an approach from the machine-learning literature: the particle filter

  14. The particle filter: general picture • Sequential Monte Carlo for incremental observations • Let xi be hidden data, zibe unobserved states • For parsing: xi are words, zi are structural analyses • Suppose that after n-1observations we have the distribution overinterpretationsP(zn-1|x1…n-1) • After obtaining the next word xn, represent the next distribution P(zn|x1…n) inductively: • Representing P(zi|x1…i) by samples makes it a Monte Carlo method

  15. Particle filter with probabilistic grammars S * S * * NP VP * * * NP VP * * * * N RRC V * * * * * * N * V Adv * * V Adv * * * * * * * horses raced quickly tired tired horses quickly raced 0.4 1.0 1.0 1.0 1.0 0.4 1.0

  16. Returning to the puzzle A-S Tom heard the gossip wasn’t true. A-L Tom heard the gossip about the neighbors wasn’t true. U-S Tom heard that the gossip wasn’t true. U-L Tom heard that the gossip about the neighbors wasn’t true. • Previous empirical finding: ambiguity induces difficulty… • …but so does the length of the ambiguous region • Our linking hypothesis: The proportion of parse failures at the disambiguating region should be monotonically related to the difficulty of the sentence Frazier & Rayner,1982; Tabor & Hutchins, 2004

  17. Model Results Ambiguity matters… But the length of the ambiguous region also matters!

  18. Human results (offline rating study)

  19. Rational comprehension’s other successes • Global disambiguation preferences (Jurafsky, 1996) The women discussed the dogs on the beach • Basic garden-path sentences (Hale, 2001) The horse raced past the barn fell • Garden-path gradience (Narayanan & Jurafsky, 2002) The crook arrested by the detective was guilty • Predictability in unambiguous contexts (Levy, 2008) The children went outside to… • Grounding in optimality/rational analysis (Norris, 2006; Smith & Levy, 2008) ? ? (not difficult) (that was) play chat

  20. Behavioral correlates (Tabor et al., 2004) • Also, Konieczny (2006, 2007) found compatible results in stops-making-sense and visual-world paradigms • These results are problematic for theories requiring global contextual consistency (Frazier, 1987; Gibson, 1991, 1998; Jurafsky, 1996; Hale, 2001, 2006) harder than thrown tossed

More Related