Context Free Grammars

1. Context Free Grammars

2. Syntax Syntax = rules describing how words can connect to each other • Goal of syntax is to model the knowledge people unconsciously have about the grammar of their native language

3. Syntax Why should you care? Grammar checkers Question answering Information extraction Machine translation

4. Constituency • The basic idea here is that groups of words within utterances can be shown to act as single units. • And in a given language, these units form coherent classes that can be shown to behave in similar ways • Constituency • How words group into units and how the various kinds of units behave

5. Constituency • Ordering: • What are the rules that govern the ordering of words and bigger units in the language?

6. Constituency • E.g., Noun phrases (NPs) • Three parties from Brooklyn • A high-class SUV such as Mindy’s • The Broadway coppers • They • Harry the Horse • The reason he comes into the Hot Box

7. How do we know these form a constituent? • They can all appear before a verb: • Three parties from Brooklyn arrive… • A high-class SUV such as Mindy’s attracts… • The Broadway coppers love… • They sit • But individual words can’t always appear before verbs: • *from arrive… • *as attracts… • *the is • *spot is… • Must be able to state generalizations like: • Noun phrases occur before verbs

8. Preposing and postposing: • On September 17th, I’d like to fly from Atlanta to Denver • I’d like to fly on September 17th from Atlanta to Denver • I’d like to fly from Atlanta to Denver on September 17th. • But not: • *On September, I’d like to fly 17th from Atlanta to Denver • *On I’d like to fly September 17th from Atlanta to Denver

9. Context-Free Grammars • Context-free grammars (CFGs) • Also known as • Phrase structure grammars • Backus-Naur form • Consist of • Rules • Terminals • Non-terminals

10. Context-Free Grammars • Terminals • We’ll take these to be words • Non-Terminals • The constituents in a language • Like noun phrase, verb phrase and sentence • Rules • Rules are equations that consist of a single non-terminal on the left and any number of terminals and non-terminals on the right.

11. CFG Example • S -> NP VP • NP -> Det NOMINAL • NOMINAL -> Noun • VP -> Verb • Det -> a • Noun -> flight • Verb -> left • these rules are defined independent of the context • where they might occur -> CFG

12. CFGs • S -> NP VP • This says that there are units called S, NP, and VP in this language • That an S consists of an NP followed immediately by a VP • Generativity • As with FSAs we can view these rules as either analysis or synthesis machines • Generate strings in the language • Reject strings not in the language • Impose structures (trees) on strings in the language

13. Parsing Parsing is the process of taking a string and a grammar and returning a (many?) parse tree(s) for that string

14. Context? • The non-terminal on the left-hand side of a rule is out there all by itself • A -> B C • Means that I can rewrite an A as a B followed by a C regardless of the context in which A is found

15. Key Constituents (English) • Sentences • Noun phrases • Verb phrases • Prepositional phrases

16. Some NP Rules • Here are some rules for the noun phrases • Together, these describe two kinds of NPs. • One that consists of a determiner followed by a nominal • And another that says that proper names are NPs. • The third rule illustrates two things • An explicit disjunction • Two kinds of nominals • A recursive definition • Same non-terminal on the right and left-side of the rule

17. Grammar

18. CFGs more formally • A context-free grammar has 4 parameters (“is a 4-tuple”) • A set of non-terminal symbols (“variables”) N • A set of terminal symbols  (disjoint from N) • A set of productions P, each of the form • A ->  • Where A is a non-terminal and  is a string of symbols from the infinite set of strings (  N)* • A designated start symbol S

19. Derivations • A derivation is a sequence of rules expansion • Common to represent derivation by a parse tree

20. Bracketed Notation • [S [NP [PRO I]] [VP [V prefer] [NP [Det a] [Nom [N morning] [N flight] ] ] ] ]

21. Sentence Types • Declaratives: A plane left. S  NP VP • Imperatives: Leave! S  VP • Yes-No Questions: Did the plane leave? S  Aux NP VP • WH Questions: When did the plane leave? S  WH-NP Aux NP VP

22. NPs • NP -> Pronoun • I came, you saw it, they conquered • NP -> Proper-Noun • Los Angeles is west of Texas • John Hennessy is the president of Stanford • NP -> Det Noun • The president • NP -> Nominal • Nominal -> Noun Nominal l Noun • A morning flight to Denver

23. PPs • PP -> Preposition NP • From LA • To the store • On Tuesday morning • With lunch

24. Recursion • We’ll have to deal with rules such as the following where the non-terminal on the left also appears somewhere on the right (directly) • NP -> NP PP [[The flight] [to Boston]] • VP -> VP PP [[departed Miami] [at noon]]

25. Recursion Example • flights from Denver • Flights from Denver to Miami • Flights from Denver to Miami in February • Flights from Denver to Miami in February on a Friday • Flights from Denver to Miami in February on a Friday under $300 • Flights from Denver to Miami in February on a Friday under $300 with lunch

26. Determiners • Noun phrases can start with determiners... • Determiners can be • Simple lexical items: the, this, a, an, etc. • A car • Or simple possessives • John’s car • Or complex recursive versions of that • John’s sister’s husband’s son’s car

27. Pre-determiners • Word classes that appear in the NP before the determiner are called predeterminers • Eg: all the flights • Word classes that appear in the NP between the determiner and the head noun are called post determiners ( they include cardinal numbers, ordinal numbers and quantifiers, adjective phrase) Eg: one stop The first one The nonstop flight the least expensive fare

28. Nominals • Contains the head and any pre- and post- modifiers of the head. • Pre- • Quantifiers, cardinals, ordinals... • Three cars • Adjectives • large cars • Ordering constraints • Three large cars • ?large three cars

29. NP->(det) (card) (ord) (quant) (AP) Nominal • After the head noun there can be post modifiers

30. Postmodifiers • Three kinds • Prepositional phrases • All flights from Seattle • Non-finite clauses • Any flights arriving before noon • Relative clauses • a flight that serves breakfast • Same general (recursive) rule to handle these • Nominal  Nominal PP • Nominal  Nominal GerundVP • Nominal  Nominal RelClause • GerundVP-> GerundV NP • GerundV PP • GerundV (being, preferring, arriving…) • GerundV NP PP • RelClause -> (who, that) VP

31. Coordination Constructions • Noun phrases and other units can be conjoined with conjunctions like and, or and but • S -> S and S • John went to NY and Mary followed him • NP -> NP and NP ( the flights and the costs) • VP -> VP and VP (leaving Denver and arriving in Sanfrancisco) • …

32. Practice: Another Recursion Example The following where the non-terminal on the left also appears somewhere on the right (directly). NP -> NP PP [[The flight] [to Boston]] VP -> VP PP [[departed Miami] [at noon]]

33. Recursion Example flights from Denver Flights from Denver to Miami Flights from Denver to Miami in February Flights from Denver to Miami in February on a Friday Flights from Denver to Miami in February on a Friday under $300 Flights from Denver to Miami in February on a Friday under $300 with lunch

34. Potential Problems in CFG Agreement Subcategorization Movement

35. Sentence Types Declaratives: A plane left. S  NP VP Imperatives: Leave! S  VP Yes-No Questions: Did the plane leave? S  Aux NP VP WH Questions: When did the plane leave? S  WH-NP Aux NP VP

36. Noun Phrases Let’s consider the following rule in more detail... NP Det Nominal Most of the complexity of English noun phrases is hidden in this rule. Consider the derivation for the following example All the morning flights from Denver to Tampa leaving before 10

37. Noun Phrases

38. NP Structure Clearly this NP is really about flights. That’s the central criticial noun in this NP. It is the head. We can dissect this kind of NP into the stuff that can come before the head, and the stuff that can come after it.

39. Determiners Noun phrases can start with determiners... Determiners can be Simple lexical items: the, this, a, an, etc. A car Or simple possessives John’s car Or complex recursive versions of that John’s sister’s husband’s son’s car

40. Nominals Contains the head and any pre- and post- modifiers of the head. Pre- Quantifiers, cardinals, ordinals... Three cars Adjectives large cars Ordering constraints Three large cars ?large three cars

41. Postmodifiers Three kinds Prepositional phrases From Seattle Non-finite clauses Arriving before noon Relative clauses That serve breakfast Same general (recursive) rule to handle these Nominal  Nominal PP Nominal  Nominal GerundVP Nominal  Nominal RelClause

42. Agreement This dog Those dogs This dog eats Those dogs eat *This dogs *Those dog *This dog eat *Those dogs eats

43. Verb Phrases English VPs consist of a head verb along with 0 or more following constituents which we’ll call arguments.

44. Subcategorization Sneeze: John sneezed Find: Please find [a flight to NY]NP Give: Give [me]NP[a cheaper fare]NP Help: Can you help [me]NP[with a flight]PP Prefer: I prefer [to leave earlier]TO-VP Told: I was told [United has a flight]S … *John sneezed the book *I prefer United has a flight *Give with a flight Subcat expresses the constraints that a predicate places on the number and type of the argument it wants to take

45. Overgeneration The various rules for VPs overgenerate. They permit the presence of strings containing verbs and arguments that don’t go together For example VP -> V NP therefore Sneezed the book is a VP since “sneeze” is a verb and “the book” is a valid NP In lecture: go over the grammar for assignment 3

46. Possible CFG Solution Possible solution for agreement. Can use the same trick for all the verb/VP classes. SgS -> SgNP SgVP PlS -> PlNp PlVP SgNP -> SgDet SgNom PlNP -> PlDet PlNom PlVP -> PlV NP SgVP ->SgV Np …

47. Movement • Core example • [[My travel agent]NP [booked [the flight]NP]VP]S • I.e. “book” is a straightforward transitive verb. It expects a single NP arg within the VP as an argument, and a single NP arg as the subject.

48. Movement • What about? • Which flight do you want me to have the travel agent book? • The direct object argument to “book” isn’t appearing in the right place. It is in fact a long way from where its supposed to appear. • And note that it’s separated from its verb by 2 other verbs.

49. Formally… To put all previous discussions/examples in a formal definition for CFG: A context free grammar has four parameters: • A set of non-terminal symbols N • A set of terminal symbols T • A set of production rules P, each of the form A  a, where A is a non-terminal, and a is a string of symbols from the infinite set of strings (T  N)* • A designated start symbol S

50. Grammar equivalence and normal form Strong equivalence: • two grammars are strongly equivalent if: • they generate/accept the same set of strings • they assign the same phrase structure to each sentence • two grammars are weakly equivalent if: • they generate/accept the same set of strings • they do not assign the same phrase structure to each sentence Normal form • Restrict the form of productions • Chomsky Normal Form (CNF) • Right hand side of the productions has either one or two terminals or non-terminals • e.g. A -> BC A -> a • Any grammar can be translated into a weakly equivalent CNF • A -> B C D <=> A-> B X X -> C D