Semantic Inference for Question Answering

1. Semantic Inference for Question Answering Sanda Harabagiu Department of Computer Science University of Texas at Dallas Srini Narayanan International Computer Science Institute Berkeley, CA and

2. Outline • Part I. Introduction: The need for Semantic Inference in QA • Current State-of-the-art in QA • Parsing with Predicate Argument Structures • Parsing with Semantic Frames • Special Text Relations • Part II. Extracting Semantic Relations from Questions and Texts • Knowledge-intensive techniques • Supervised and unsupervised techniques

3. Outline • Part III. Knowledge representation and inference • Representing the semantics of answers • Extended WordNet and abductive inference • Intentional Structure and Probabilistic Metonymy • An example of Event Structure • Modeling relations, uncertainty and dynamics • Inference methods and their mapping to answer types

4. Outline • Part IV. From Ontologies to Inference • From OWL to CPRM • FrameNet in OWL • FrameNet to CPRM mapping • Part V. Results of Event Structure Inference for QA • AnswerBank examples • Current results for Inference Type • Current results for Answer Structure

5. The need for Semantic Inference in QA • Some questions are complex! • Example: • How can a biological weapons program be detected ? • Answer: In recent months, Milton Leitenberg, an expert on biological weapons, has been looking at this murkiest and most dangerous corner of Saddam Hussein's armory. He says a series of reports add up to indications that Iraq may be trying to develop a new viral agent, possibly in underground laboratories at a military complex near Baghdad where Iraqis first chased away inspectors six years ago. A new assessment by the United Nations suggests Iraq still has chemical and biological weapons - as well as the rockets to deliver them to targets in other countries. The UN document says Iraq may have hidden a number of Scud missiles, as well as launchers and stocks of fuel. US intelligence believes Iraq still has stockpiles of chemical and biological weapons and guided missiles, which it hid from the UN inspectors.

6. Complex questions • Example: • How can a biological weapons program be detected ? • This question is complex because: • It is a manner question • All other manner questions that were evaluated in TREC were asking about 3 things: • Manners to die, e.g. “How did Cleopatra die?”, “How did Einstein die?” • Manners to get a new name, e.g. “How did Cincinnati get its name?” • Manners to say something in another language, e.g. “How do you say house in Spanish?” • The answer does not contain any explicit manner of detection information, instead it talks about reports that give indications that Iraq may be trying to develop a new viral agent and assessments by the United Nations suggesting that Iraq still has chemical and biological weapons

7. Complex questions andsemantic information • Complex questions are not characterized only by a question class (e.g. manner questions) • Example: How can a biological weapons program be detected ? • Associated with the pattern “How can X be detected?” • And the topic X = “biological weapons program” • Processing complex questions is also based on access to the semantics of the question topic • The topic is modeled by a set of discriminating relations, e.g. Develop(program); Produce(biological weapons); Acquire(biological weapons) or stockpile(biological weapons) • Such relations are extracted from topic-relevant texts

8. Alternative semantic representations • Using PropBank to access a 1 million word corpus annotated with predicate-argument structures.(www.cis.upenn.edu/~ace) • We can train a generative model for recognizing the arguments of each predicate in questions and in the candidate answers. • Example: How can a biological weapons program be detected ? Predicate: detect Argument 0 = detector : Answer(1) Argument 1 = detected: biological weapons Argument 2 = instrument : Answer(2) Expected Answer Type

9. More predicate-argument structures for questions • Example: From which country did North Korea import its missile launch pad metals? • Example:What stimulated India’s missile programs? Predicate: import Argument 0: (role = importer): North Korea Argument 1: (role = commodity): missile launch pad metals Argument 2 (role = exporter): ANSWER Predicate: stimulate Argument 0: (role = agent): ANSWER (part 1) Argument 1: (role = thing increasing): India’s missile programs Argument 2 (role = instrument): ANSWER (part 2)

10. Additional semantic resources • Using FrameNet • frame-semanticdescriptions of several thousand English lexical items with semantically annotated attestations (www.icsi.berkeley.edu/~framenet) • Example:What stimulated India’s missile programs? Frame: STIMULATE Frame Element: CIRCUMSTANCES: ANSWER (part 1) Frame Element: EXPERIENCER: India’s missile programs Frame Element: STIMULUS: ANSWER (part 2) Frame: SUBJECT STIMULUS Frame Element: CIRCUMSTANCES: ANSWER (part 3) Frame Element: COMPARISON SET: ANSWER (part 4) Frame Element: EXPERIENCER: India’s missile programs Frame Element: PARAMETER: nuclear/biological proliferation

11. Semantic inference for Q/A • The problem of classifying questions • E.g. “manner questions”: • Example“How did Hitler die?” • The problem of recognizing answer types/structures • Should “manner of death” by considered an answer type? • What other manner of event/action should be considered as answer types? • The problem of extracting/justifying/ generating answers to complex questions • Should we learn to extract “manner” relations? • What other types of relations should we consider? • Is relation recognition sufficient for answering complex questions? Is it necessary?

12. Manner-of-death In previous TREC evaluations 31 questions asked about manner of death: • “How did Adolf Hitler die?” • State-of-the-art solution (LCC): • We considered “Manner-of-Death” as an answer type, pointing to a variety of verbs and nominalizations encoded in WordNet • We developed text mining techniques for identifying such information based on lexico-semantic patterns from WordNet • Example: • [kill #sense1 (verb) – CAUSE  die #sense1 (verb)] • Source of the troponyms of the [kill #sense1 (verb)] concept are candidates for the MANNER-OF-DEATH hierarchy • e.g., drown, poison, strangle, assassinate, shoot

13. X DIE in ACCIDENT seed: train, accident, be killed (ACCIDENT) car wreck X DIE {from|of} DISEASE seed: cancer be killed (DISEASE) AIDS X DIE after suffering MEDICAL seed: stroke, suffering of CONDITION (ACCIDENT) complications caused by diabetes Practical Hurdle • Not all MANNER-OF-DEATH concepts are lexicalized as a verb  we set out to determine additional patterns that capture such cases • Goal: (1) set of patterns (2) dictionaries corresponding to such patterns  well known IE technique: (IJCAI’99, Riloff&Jones) • Results: 100 patterns were discovered

14. Outline • Part I. Introduction: The need for Semantic Inference in QA • Current State-of-the-art in QA • Parsing with Predicate Argument Structures • Parsing with Semantic Frames • Special Text Relations

15. Answer types in State-of-the-art QA systems Ranked set of passages Docs Question Answer Question Expansion Answer Selection IR answer type Answer Type Prediction Answer Type Hierarchy Features • Answer type • Labels questions with answer type based on a taxonomy • Classifies questions (e.g. by using a maximum entropy model)

16. In Question Answering two heads are better than one • The idea originated in the IBM’s PIQUANT project • Traditional Q/A systems employ a pipeline approach: • Questions analysis • Document/passage retrieval • Answer selection • Questions are classified based on the expected answer type • Answers are also selected based on the expected answer type, regardless of the question class Motivated by the success of ensemble methods in machine learning, use multiple classifiers to produce the final output for the ensemble made of multiple QA agents • A multi-strategy, multi-source approach.

17. Question Analysis Answer Classification Multiple sources, multiple agents Knowledge Source Portal QGoals WordNet Q-Frame Answer Type Answering Agents QPlan Generator Cyk QUESTION Predictive Annot. Answering Agents Web Statistical Answering Agents Semantic Search Definitional Q. Answering Agents QPlan Executor KSP-Based Answering Agents Keyword Search Pattern-Based Answering Agents AQUAINT CNS TREC Answer Resolution Answers ANSWER

18. Multiple Strategies • In PIQUANT, the answer resolution strategies consider that different combinations of the questions processing, passage retrieval and answer selection from different agents is ideal. • This entails the fact that all questions are processed depending on the questions class, not the question type • There are multiple question classes, e.g. “What” questions asking about people, “What” questions asking about products, etc. • There are only three types of questions that have been evaluated yet in systematic ways: • Factoid questions • Definition questions • List questions • Another options is to build an architecture in which question types are processed differently, and the semantic representations and inference mechanisms are adapted for each question type.

19. Document Processing Question Processing Factoid Answer Processing Single Factoid Passages Question Parse Answer Extraction Factoid Question Multiple List Passages Answer Justification Semantic Transformation Factoid Answer Answer Reranking Recognition of Expected Answer Type List Question Theorem Prover Axiomatic Knowledge Base Passage Retrieval Keyword Extraction Named Entity Recognition (CICERO LITE) Answer Type Hierarchy (WordNet) Answer Extraction List Answer List Answer Processing Document Index Threshold Cutoff Question Processing AQUAINT Document Collection Definition Answer Processing Definition Question Question Parse Answer Extraction Pattern Matching Pattern Repository Pattern Matching Keyword Extraction The Architecture of LCC’s QA System Multiple Definition Passages Definition Answer

20. Extracting Answers for Factoid Questions • In TREC 2003 the LCC QA system extracted 289 correct answers for factoid questions • The Name Entity Recognizer was responsible for 234 of them

21. Special Case of Names Questions asking for names of authored works

22. NE-driven QA • The results of the past 5 TREC evaluations of QA systems indicate that current state-of-the-art QA is determined by the recognition of Named Entities: • Precision of recognition • Coverage of name classes • Mapping into concept hierarchies • Participation into semantic relations (e.g. predicate-argument structures or frame semantics)

23. Concept Taxonomies • For 29% of questions the QA system relied on an off-line taxonomy with semantic classes such as: • Disease • Drugs • Colors • Insects • Games • The majority of these semantic classes are also associated with patterns that enable their identification

24. Definition Questions • They asked about: • PEOPLE (most of them starting with “Who”) • other types of NAMES • general concepts • People questions • Many use the PERSON name in the format [First name, Last name] • examples: Aaron Copland, Allen Iverson, Albert Ghiorso • Some names had the PERSON name in format [First name, Last name1, Last name2] • example: Antonia Coello Novello • Other names had the name as a single word very well known person • examples: Nostradamus, Absalom, Abraham • Some questions referred to names of kings or princes: • examples: Vlad the Impaler, Akbar the Great

25. Answering definition questions • Most QA systems use between 30-60 patterns • The most popular patterns:

26. Complex questions • Characterized by the need of domain knowledge • There is no single answer type that can be identified, but rather an answer structure needs to be recognized • Answer selection becomes more complicated, since inference based on the semantics of the answer type needs to be activated • Complex questions need to be decomposed into a set of simpler questions

27. Example of Complex Question How have thefts impacted on the safety of Russia’s nuclear navy, and has the theft problem been increased or reduced over time? Need of domain knowledge To what degree do different thefts put nuclear or radioactive materials at risk? Question decomposition • Definition questions: • What is meant by nuclear navy? • What does ‘impact’ mean? • How does one define the increase or decrease of a problem? Factoid questions: • What is the number of thefts that are likely to be reported? • What sort of items have been stolen? Alternative questions: • What is meant by Russia? Only Russia, or also former Soviet facilities in non-Russian republics?

28. The answer structure • For complex questions, the answer structure has a compositional semantics, comprising all the answer structures of each simpler question in which it is decomposed. • Example: Q-Sem: How can a biological weapons program be detected? Question pattern: How can X be detected? X = Biological Weapons Program Conceptual Schemas INSPECTION Schema Inspect, Scrutinize, Monitor, Detect, Evasion, Hide, Obfuscate POSSESSION Schema Acquire, Possess, Develop, Deliver Structure of Complex Answer Type:EVIDENCE CONTENT SOURCE QUALITY JUDGE RELIABILITY

29. Conceptual Schemas INSPECTION Schema Inspect, Scrutinize, Monitor, Detect, Evasion, Hide, Obfuscate POSSESSION Schema Acquire, Possess, Develop, Deliver Answer Selection • Based on the answer structure • Example: • The CONTENT is selected based on: • Conceptual schemas are instantiated when predicate-argument structures or semantic frames are recognized in the text passages • The SOURCE is recognized when the content source is identified • The Quality of the Judgements, the Reliability of the judgements and the Judgements themselves are produced by an inference mechanism Structure of Complex Answer Type:EVIDENCE CONTENT SOURCE QUALITY JUDGE RELIABILITY

30. ANSWER: Evidence-Combined:Pointer to Text Source: A1: In recent months, Milton Leitenberg, an expert on biological weapons, has been looking at this murkiest and most dangerous corner of Saddam Hussein's armory. A2: He says a series of reports add up to indications that Iraq may be trying to develop a new viral agent, possibly in underground laboratories at a military complex near Baghdad where Iraqis first chased away inspectors six years ago. Answer Structure A3: A new assessment by the United Nations suggests Iraq still has chemical and biological weapons - as well as the rockets to deliver them to targets in other countries. A4:The UN document says Iraq may have hidden a number of Scud missiles, as well as launchers and stocks of fuel. A5: US intelligence believes Iraq still has stockpiles of chemical and biological weapons and guided missiles, which it hid from the UN inspectors Content: Biological Weapons Program: develop(Iraq, Viral_Agent(instance_of:new)) Justification: POSSESSION Schema Previous (Intent and Ability): Prevent(ability, Inspection); Inspection terminated; Status: Attempt ongoing Likelihood: Medium Confirmability: difficult, obtuse, hidden possess(Iraq, Chemical and Biological Weapons) Justification: POSSESSION SchemaPrevious (Intent and Ability): Prevent(ability, Inspection); Status: Hidden from Inspectors Likelihood: Medium possess(Iraq, delivery systems(type : rockets; target: other countries)) Justification: POSSESSION SchemaPrevious (Intent and Ability): Hidden from Inspectors; Status: Ongoing Likelihood: Medium

31. Answer Structure (continued) ANSWER: Evidence-Combined:Pointer to Text Source: A1: In recent months, Milton Leitenberg, an expert on biological weapons, has been looking at this murkiest and most dangerous corner of Saddam Hussein's armory. A2: He says a series of reports add up to indications that Iraq may be trying to develop a new viral agent, possibly in underground laboratories at a military complex near Baghdad where Iraqis first chased away inspectors six years ago. A3: A new assessment by the United Nations suggests Iraq still has chemical and biological weapons - as well as the rockets to deliver them to targets in other countries. A4:The UN document says Iraq may have hidden a number of Scud missiles, as well as launchers and stocks of fuel. A5: US intelligence believes Iraq still has stockpiles of chemical and biological weapons and guided missiles, which it hid from the UN inspectors Content: Biological Weapons Program: possess(Iraq, delivery systems(type : scud missiles; launchers; target: other countries)) Justification: POSSESSION SchemaPrevious (Intent and Ability): Hidden from Inspectors; Status: Ongoing Likelihood: Medium possess(Iraq, fuel stock(purpose: power launchers)) Justification: POSSESSION SchemaPrevious (Intent and Ability): Hidden from Inspectors; Status: Ongoing Likelihood: Medium hide(Iraq, Seeker: UN Inspectors; Hidden: CBW stockpiles & guided missiles) Justification: DETECTION SchemaInspection status: Past; Likelihood: Medium

32. Answer Structure (continued) ANSWER: Evidence-Combined:Pointer to Text Source: A1: In recent months, Milton Leitenberg, an expert on biological weapons, has been looking at this murkiest and most dangerous corner of Saddam Hussein's armory. A2: He says a series of reports add up to indications that Iraq may be trying to develop a new viral agent, possibly in underground laboratories at a military complex near Baghdad where Iraqis first chased away inspectors six years ago. A3: A new assessment by the United Nations suggests Iraq still has chemical and biological weapons - as well as the rockets to deliver them to targets in other countries. A4:The UN document says Iraq may have hidden a number of Scud missiles, as well as launchers and stocks of fuel. A5: US intelligence believes Iraq still has stockpiles of chemical and biological weapons and guided missiles, which it hid from the UN inspectors Source: UN documents, US intelligence SOURCE.Type: Assesment reports; Source.Reliability: Med-high Likelihood: Medium Judge: UN, US intelligence, Milton Leitenberg (Biological Weapons expert) JUDGE.Type: mixed; Judge.manner; Judge.stage: ongoing Quality: low-medium; Reliability: low-medium;

33. State-of-the-art QA:Learning surface text patterns • Pioneered by Ravichandran and Hovy (ACL-2002) • The idea is that given a specific answer type (e.g. Birth-Date), learn all surface patterns that enable the extraction of the answer from any text passage • Patterns are learned by two algorithms: • Relies on Web redundancy Algorithm 1 (Generates Patterns) Step 1: Select an answer type AT and a question Q(AT) Step 2: Generate a query (Q(AT) & AT) and submit it to search engine (google, altavista) Step 3: Download the first 1000 documents Step 4: Select only those sentences that contain the question content words and the AT Step 5: Pass the sentences through a suffix tree constructor Step 6: Extract only the longest matching sub-strings that contain the AT and the question word it is syntactically connected with. Algorithm 2 (Measures the Precision of Patterns) Step 1: Query by using only question Q(AT) Step 2: Download the first 1000 documents Step 3: Select only those sentences that contain the question word connected to the AT Step 4: Compute C(a)= #patterns matched by the correct answer; C(0)=#patterns matched by any word Step 6: The precision of a pattern is given by: C(a)/C(0) Step 7: Retain only patterns matching >5 examples

34. Results and Problems • Some results: • Limitations: • Cannot handle long-distance dependencies • Cannot recognize paraphrases – since no semantic knowledge is associated with these patterns (unlike patterns used in Information Extraction) • Cannot recognize a paraphrased questions Answer Type=INVENTOR: <ANSWER> invents <NAME> the <NAME> was invented by <ANSWER> <ANSWER>’s invention of the <NAME> <ANSWER>’s <NAME> was <NAME>, invented by <ANSWER> That <ANSWER>’s <NAME> Answer Type=BIRTH-YEAR: <NAME> (<ANSWER>- ) <NAME> was born on <ANSWER> <NAME> was born in <ANSWER> born in <ANSWER>, <NAME> Of <NAME>, (<ANSWER>

35. Shallow semantic parsing • Part of the problems can be solved by using shallow semantic parsers • Parsers that use shallow semantics encoded as either predicate-argument structures or semantic frames • Long-distance dependencies are captured • Paraphrases can be recognized by mapping on IE architectures • In the past 4 years, several models for training such parsers have emerged • Lexico-Semantic resources are available (e.g PropBank, FrameNet) • Several evaluations measure the performance of such parsers (e.g. SENSEVAL, CoNNL)

36. Outline • Part I. Introduction: The need for Semantic Inference in QA • Current State-of-the-art in QA • Parsing with Predicate Argument Structures • Parsing with Semantic Frames • Special Text Relations

37. S NP VP VP PP NP The futures halt was assailed by Big Board floor traders ARG1 = entity assailed PRED ARG0 = agent Proposition Bank Overview • A one million word corpus annotated with predicate argument structures [Kingsbury, 2002]. Currently only predicates lexicalized by verbs. • Numbered arguments from 0 to 5. Typically ARG0 = agent, ARG1 = direct object or theme, ARG2 = indirect object, benefactive, or instrument. • Functional tags: ARMG-LOC = locative, ARGM-TMP = temporal, ARGM-DIR = direction.

38. S NP VP VP PP NP Task 1 The futures halt was assailed by Big Board floor traders PRED ARG1 ARG0 Task 2 The Model • Consists of two tasks: (1) identifying parse tree constituents corresponding to predicate arguments, and (2) assigning a role to each argument constituent. • Both tasks modeled using C5.0 decision tree learning, and two sets of features: Feature Set 1 adapted from [Gildea and Jurafsky, 2002], and Feature Set 2, novel set of semantic and syntactic features [Surdeanu, Harabagiu et al, 2003].

39. PHRASE TYPE (pt): type of the syntactic phrase as argument. E.g. NP for ARG1. PARSE TREE PATH (path): path between argument and predicate. E.g. NP  S  VP  VP for ARG1. PATH LENGTH (pathLen): number of labels stored in the predicate-argument path. E.g. 4 for ARG1. POSITION (pos): indicates if constituent appears before predicate in sentence. E.g. true for ARG1 and false for ARG2. VOICE (voice): predicate voice (active or passive). E.g. passive for PRED. HEAD WORD (hw): head word of the evaluated phrase. E.g. “halt” for ARG1. GOVERNING CATEGORY (gov): indicates if an NP is dominated by a S phrase or a VP phrase. E.g. S for ARG1, VP for ARG0. PREDICATE WORD: the verb with morphological information preserved (verb), and the verb normalized to lower case and infinitive form (lemma). E.g. for PRED verb is “assailed”, lemma is “assail”. S NP VP VP PP NP The futures halt was assailed by Big Board floor traders ARG1 PRED ARG0 Feature Set 1

40. PP in NP last June SBAR that S VP occurred NP yesterday VP to VP be VP declared Observations about Feature Set 1 • Because most of the argument constituents are prepositional attachments (PP) and relative clauses (SBAR), often the head word (hw) is not the most informative word in the phrase. • Due to its strong lexicalization, the model suffers from data sparsity. E.g. hw used < 3%. The problem can be addressed with a back-off model from words to part of speech tags. • The features in set 1 capture only syntactic information, even though semantic information like named-entity tags should help. For example, ARGM-TMP typically contains DATE entities, and ARGM-LOC includes LOCATION named entities. • Feature set 1 does not capture predicates lexicalized by phrasal verbs, e.g. “put up”.

41. Feature Set 2 (1/2) • CONTENT WORD (cw): lexicalized feature that selects an informative word from the constituent, other than the head. Selection heuristics available in the paper. E.g. “June” for the phrase “in last June”. • PART OF SPEECH OF CONTENT WORD (cPos): part of speech tag of the content word. E.g. NNP for the phrase “in last June”. • PART OF SPEECH OF HEAD WORD (hPos): part of speech tag of the head word. E.g. NN for the phrase “the futures halt”. • NAMED ENTITY CLASS OF CONTENT WORD (cNE): The class of the named entity that includes the content word. 7 named entity classes (from the MUC-7 specification) covered. E.g. DATE for “in last June”.

42. Feature Set 2 (2/2) • BOOLEAN NAMED ENTITY FLAGS: set of features that indicate if a named entity is included at any position in the phrase: • neOrganization: set to true if an organization name is recognized in the phrase. • neLocation: set to true if a location name is recognized in the phrase. • nePerson: set to true if a person name is recognized in the phrase. • neMoney: set to true if a currency expression is recognized in the phrase. • nePercent: set to true if a percentage expression is recognized in the phrase. • neTime: set to true if a time of day expression is recognized in the phrase. • neDate: set to true if a date temporal expression is recognized in the phrase. • PHRASAL VERB COLLOCATIONS: set of two features that capture information about phrasal verbs: • pvcSum: the frequency with which a verb is immediately followed by any preposition or particle. • pvcMax: the frequency with which a verb is followed by its predominant preposition or particle.

43. Results

44. Other parsers based on PropBank • Pradhan, Ward et al, 2004 (HLT/NAACL+J of ML) report on a parser trained with SVMs which obtains F1-score=90.4% for Argument classification and 80.8% for detecting the boundaries and classifying the arguments, when only the first set of features is used. • Gildea and Hockenmaier (2003) use features extracted from Combinatory Categorial Grammar (CCG). The F1-measure obtained is 80% • Chen and Rambow (2003) use syntactic and semantic features extracted from a Tree Adjoining Grammar (TAG) and report an F1-measure of 93.5% for the core arguments • Pradhan, Ward et al, use a set of 12 new features and obtain and F1-score of 93.8% for argument classification and 86.7 for argument detection and classification

45. Q: What kind of materials were stolen from the Russian navy? PAS(Q): What [Arg1: kind of nuclear materials] were [Predicate:stolen] [Arg2: from the Russian Navy]? Applying Predicate-Argument Structures to QA • Parsing Questions • Parsing Answers • Result: exact answer= “approximately 7 kg of HEU” A(Q): Russia’s Pacific Fleet has also fallen prey to nuclear theft; in 1/96, approximately 7 kg of HEU was reportedly stolen from a naval base in Sovetskaya Gavan. PAS(A(Q)): [Arg1(P1redicate 1): Russia’s Pacific Fleet] has [ArgM-Dis(Predicate 1) also] [Predicate 1: fallen] [Arg1(Predicate 1): prey to nuclear theft]; [ArgM-TMP(Predicate 2): in 1/96], [Arg1(Predicate 2): approximately 7 kg of HEU] was [ArgM-ADV(Predicate 2) reportedly] [Predicate 2: stolen] [Arg2(Predicate 2): from a naval base] [Arg3(Predicate 2): in Sovetskawa Gavan]

46. Outline • Part I. Introduction: The need for Semantic Inference in QA • Current State-of-the-art in QA • Parsing with Predicate Argument Structures • Parsing with Semantic Frames • Special Text Relations

47. The Model • Consists of two tasks: (1) identifying parse tree constituents corresponding to frame elements, and (2) assigning a semantic role to each frame element. • Both tasks introduced for the first time by Gildea and Jurafsky in 2000. It uses the Feature Set 1 , which later Gildea and Palmer used for parsing based on PropBank. S NP VP NP PP Task 1 She clapped her hands in inspiration PRED Agent Body Part Cause Task 2

48. Extensions • Fleischman et al extend the model in 2003 in three ways: • Adopt a maximum entropy framework for learning a more accurate classification model. • Include features that look at previous tags and use previous tag information to find the highest probability for the semantic role sequence of any given sentence. • Examine sentence-level patterns that exploit more global information in order to classify frame elements.

49. Q: What kind of materials were stolen from the Russian navy? FS(Q): What [GOODS: kind of nuclear materials] were [Target-Predicate:stolen] [VICTIM: from the Russian Navy]? Applying Frame Structures to QA • Parsing Questions • Parsing Answers • Result: exact answer= “approximately 7 kg of HEU” A(Q): Russia’s Pacific Fleet has also fallen prey to nuclear theft; in 1/96, approximately 7 kg of HEU was reportedly stolen from a naval base in Sovetskaya Gavan. FS(A(Q)): [VICTIM(P1): Russia’s Pacific Fleet] has also fallen prey to [Goods(P1): nuclear ] [Target-Predicate(P1): theft]; in 1/96, [GOODS(P2): approximately 7 kg of HEU] was reportedly [Target-Predicate (P2): stolen] [VICTIM (P2): from a naval base] [SOURCE(P2): in Sovetskawa Gavan]

50. Outline • Part I. Introduction: The need for Semantic Inference in QA • Current State-of-the-art in QA • Parsing with Predicate Argument Structures • Parsing with Semantic Frames • Special Text Relations