- 117 Views
- Uploaded on
- Presentation posted in: General

COGEX at the Second RTE

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

COGEX at the Second RTE

Marta Tatu, Brandon Iles, John Slavick, Adrian Novischi, Dan Moldovan

Language Computer Corporation

April 10th, 2006

- Linear combination of three entailment scores
- COGEX with constituency parse tree-derived logic forms
- COGEX with dependency parse tree-derived logic forms
- Lexical alignment between T and H
For each pair i (Ti,Hi)

If

then Ti entails Hi

- Lambda(λ) parameters learned on the development data for each task (IE, IR, QA, SUM)

@2006 Language Computer Corporation

- Textual Entailment
- Task definition: T entails H, denoted by T → H, if the meaning of H can be inferred from the meaning of T
- inferred » logic (theorem prover + axioms)
- meaning » semantics (semantic-enhanced representation)

@2006 Language Computer Corporation

- Transform the two text fragments into 3-layered logic forms
- Syntactic
- Semantic
- Temporal

- Automatically create axioms to be used during the proof
- Lexical Chains axioms
- World Knowledge axioms
- Linguistic transformation axioms

- Load COGEX’s SOS with T and H and its USABLE list of clauses with the generated axioms,
- Search for a proof by iteratively removing clauses from SOS and searching the USABLE for possible inferences until a refutation is found
- If no contradiction is detected
- Relax arguments
- Drop entire predicates from H

- If no contradiction is detected
- Compute proof score

semantic and temporal axioms

@2006 Language Computer Corporation

- Logic Form Transformation
- Negations
- not_RB(x1,e1) & walk_VB(e1,x2,x3) » -walk_VB(e1,x2,x3)
- not_RB(x1,e1) & walk_VB(e1,x2,x3) & fast_RB(x4,e1) » -fast_RB(x4,e1)
- no/DT case_NN(x1) & confirm_VB(e1,x2,x1) » -confirm_VB(e1,x2,x1)

- Negations

@2006 Language Computer Corporation

- Logic Form Transformation
- Temporal normalization of date/time predicates
- 13th of January 1990 vs. January 13th, 1990
- 13th_of_January_1990_NN(x1) vs. January_13th_1990_NN(x1)

- time_TMP(BeginFN(x1), year, month, day, hour, minute, second) & time_TMP(EndFN(x1), year, month, day, hour, minute, second)
- time_TMP(BeginFN(x1), 1990, 1, 13, 0, 0, 0) & time_TMP(EndFN(x1), 1990, 1, 13, 23, 59, 59)

- 13th of January 1990 vs. January 13th, 1990

- Temporal normalization of date/time predicates

@2006 Language Computer Corporation

- Logic Form Transformation
- Temporal context SUMO predicates (Clark et al., 2005)
- (S,E1,E2) : S is the temporal signal linking two events E1 and E2
- during_TMP(e1,x1), earlier_TMP(e1,x1), …

- Temporal context SUMO predicates (Clark et al., 2005)

@2006 Language Computer Corporation

- Generate LF from two different sources
- Constituency parse of the data
- Dependency parse trees (data provided by the challenge organizers)

@2006 Language Computer Corporation

- Gilda Flores was kidnapped on the 13th of January 1990.
- Constituency: Gilda_NN(x1) & Flores_NN(x2) & nn_NNC(x3,x1,x2) & _human_NE(x3) & kidnap_VB(e1,x9,x3) & on_IN(e1,x8) & 13th_NN(x4) & of_NN(x5) & January_NN(x6) & 1990_NN(x7) & nn_ NNC(x8,x4,x5,x6,x7) & _date_NE(x8) & THM_SR(x3,e1) & TMP_SR(x8,e1) & time_TMP(BeginFN(x1), 1990, 1, 13, 0, 0, 0) & time_TMP(EndFN(x1), 1990, 1, 13, 23, 59, 59) & during_TMP(e1,x8)
- Dependency: Gilda_Flores_NN(x2) & _human_NE(x2) & kidnap_VB(e1,x4,x2) & on_IN(e1,x3) & 13th_NN(x3) & of_IN(x3,x1) & January_1990_NN(x1)

@2006 Language Computer Corporation

- Axioms on Demand
- Lexical Chains
- Consider the first k=3 senses for each word
- Maximum length of a lexical chain = 3
- DERIVATIONAL WordNet relation is ambiguous with respect to the role of the noun
- Derivation-ACT: employ_VB(e1,x1,x2) → employment_NN(e1)
- Derivation-AGENT: employ_VB(e1,x1,x2) → employer_NN(x1)
- Derivation-THEME: employ_VB(e1,x1,x2) → employee_NN(x2)

- Morphological derivations between adjectives and verbs

- Lexical Chains

@2006 Language Computer Corporation

- Axioms on Demand
- Lexical Chains
- Augment with the NE predicate for NE target concepts
- nicaraguan_JJ(x1,x2) → Nicaragua_NN(x1) & _country_NE(x1)

- Discard lexical chains
- with more than 2 HYPONYMY relations (H too specific)
- with a HYPONYMY followed by an ISA
- Chicago_NN(x1) → Detroit_NN(x1)

- which include general concepts: object/NN, act/VB, be/VB
- ni= number of hyponyms of concept ci
- N = number of concepts in ci’s hierarchy

- Augment with the NE predicate for NE target concepts

- Lexical Chains

@2006 Language Computer Corporation

- Another 73 World Knowledge axioms
- Semantic Calculus – combinations of two semantic relations (82 axioms)
- ISA, KINSHIP, CAUSE are transitive relations
- ISA_SR(x1,x2) & PAH_SR(x3,x2) → PAH_SR(x3,x2)
- Mike is a rich man → Mike is rich

- Temporal Reasoning Axioms (Clark et al., 2005) (65 axioms)
- Dates entail more general times
- October 2000 → year 2000

- during_TMP(e1,e2) & during_TMP(e2,e3) → during_TMP(e1,e3)

- Dates entail more general times

@2006 Language Computer Corporation

- Proof Re-Scoring
- (T) smart people → people (H)
- (T) people → smart people (H)
- Entities mentioned in T and H are existentially quantified

- Universally quantified T and H entities
- (T) people → smart people (H)
- (T) smart people → people (H)

@2006 Language Computer Corporation

- Compute the edit distance between T and H
- Cost (deletion of a word from T) = 0
- Cost (replace of a word from T with another in H) = ∞
- Cost (insert a word from H) =
- Edit distance between synonyms = 0

@2006 Language Computer Corporation

- IE: score given by COGEXC with some correction from COGEXD
- IR: the highest contribution is made by LexAlign (~62%)
- COGEXD better on IE, IR, QA (~69% accuracy)
- COGEXC better on SUM (~66% accuracy)
- Three-way combination outperforms any individual results and any two-system combination

Learned parameters:

@2006 Language Computer Corporation

- Higher accuracy on the SUM task
- SUM is the highest accuracy task for all systems (false entailment pairs had H completely unrelated with the texts T)

- IE: highest number of false positives
- Need more World Knowledge
- (QA task) 15 safety violations → numerous safety violations

- Upper bound (human performance) for RTE2 test
- 97% proportional agreement
- Kappa agreement: K = 0.94 (good agreement)
- Fewer controversial examples in this year’s test

- Performance on RTE1 test: 69% accuracy

@2006 Language Computer Corporation

- Other types of context: report, planning, etc.
- Pairs (T:X said Y, H:Y) labeled as both TRUE and FALSE

- Need for more axioms
- Paraphrase acquisition (phrase1→ phrase2)
- Automatic gathering of semantic axioms
- Lexical chains link only concepts
- WordNet gloss axioms link a concept to a phrase

@2006 Language Computer Corporation

Thank You !

Questions?