Cogex at the second rte
This presentation is the property of its rightful owner.
Sponsored Links
1 / 18

COGEX at the Second RTE PowerPoint PPT Presentation


  • 104 Views
  • Uploaded on
  • Presentation posted in: General

COGEX at the Second RTE. Marta Tatu, Brandon Iles, John Slavick, Adrian Novischi, Dan Moldovan Language Computer Corporation April 10 th , 2006. LCC’s Submission to RTE2. Linear combination of three entailment scores COGEX with constituency parse tree-derived logic forms

Download Presentation

COGEX at the Second RTE

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Cogex at the second rte

COGEX at the Second RTE

Marta Tatu, Brandon Iles, John Slavick, Adrian Novischi, Dan Moldovan

Language Computer Corporation

April 10th, 2006


Lcc s submission to rte2

LCC’s Submission to RTE2

  • Linear combination of three entailment scores

    • COGEX with constituency parse tree-derived logic forms

    • COGEX with dependency parse tree-derived logic forms

    • Lexical alignment between T and H

      For each pair i (Ti,Hi)

      If

      then Ti entails Hi

  • Lambda(λ) parameters learned on the development data for each task (IE, IR, QA, SUM)

@2006 Language Computer Corporation


Semantic based logic approach

Semantic-based Logic Approach

  • Textual Entailment

    • Task definition: T entails H, denoted by T → H, if the meaning of H can be inferred from the meaning of T

    • inferred » logic (theorem prover + axioms)

    • meaning » semantics (semantic-enhanced representation)

@2006 Language Computer Corporation


Approach to rte with cogex

Approach to RTE with COGEX

  • Transform the two text fragments into 3-layered logic forms

    • Syntactic

    • Semantic

    • Temporal

  • Automatically create axioms to be used during the proof

    • Lexical Chains axioms

    • World Knowledge axioms

    • Linguistic transformation axioms

  • Load COGEX’s SOS with T and H and its USABLE list of clauses with the generated axioms,

  • Search for a proof by iteratively removing clauses from SOS and searching the USABLE for possible inferences until a refutation is found

    • If no contradiction is detected

      • Relax arguments

      • Drop entire predicates from H

  • Compute proof score

semantic and temporal axioms

@2006 Language Computer Corporation


Cogex enhancements 1 3

COGEX Enhancements (1/3)

  • Logic Form Transformation

    • Negations

      • not_RB(x1,e1) & walk_VB(e1,x2,x3) » -walk_VB(e1,x2,x3)

      • not_RB(x1,e1) & walk_VB(e1,x2,x3) & fast_RB(x4,e1) » -fast_RB(x4,e1)

      • no/DT case_NN(x1) & confirm_VB(e1,x2,x1) » -confirm_VB(e1,x2,x1)

@2006 Language Computer Corporation


Cogex enhancements 1 31

COGEX Enhancements (1/3)

  • Logic Form Transformation

    • Temporal normalization of date/time predicates

      • 13th of January 1990 vs. January 13th, 1990

        • 13th_of_January_1990_NN(x1) vs. January_13th_1990_NN(x1)

      • time_TMP(BeginFN(x1), year, month, day, hour, minute, second) & time_TMP(EndFN(x1), year, month, day, hour, minute, second)

        • time_TMP(BeginFN(x1), 1990, 1, 13, 0, 0, 0) & time_TMP(EndFN(x1), 1990, 1, 13, 23, 59, 59)

@2006 Language Computer Corporation


Cogex enhancements 1 32

COGEX Enhancements (1/3)

  • Logic Form Transformation

    • Temporal context SUMO predicates (Clark et al., 2005)

      • (S,E1,E2) : S is the temporal signal linking two events E1 and E2

      • during_TMP(e1,x1), earlier_TMP(e1,x1), …

@2006 Language Computer Corporation


Logic forms differences

Logic Forms Differences

  • Generate LF from two different sources

    • Constituency parse of the data

    • Dependency parse trees (data provided by the challenge organizers)

@2006 Language Computer Corporation


Logic forms differences1

Logic Forms Differences

  • Gilda Flores was kidnapped on the 13th of January 1990.

  • Constituency: Gilda_NN(x1) & Flores_NN(x2) & nn_NNC(x3,x1,x2) & _human_NE(x3) & kidnap_VB(e1,x9,x3) & on_IN(e1,x8) & 13th_NN(x4) & of_NN(x5) & January_NN(x6) & 1990_NN(x7) & nn_ NNC(x8,x4,x5,x6,x7) & _date_NE(x8) & THM_SR(x3,e1) & TMP_SR(x8,e1) & time_TMP(BeginFN(x1), 1990, 1, 13, 0, 0, 0) & time_TMP(EndFN(x1), 1990, 1, 13, 23, 59, 59) & during_TMP(e1,x8)

  • Dependency: Gilda_Flores_NN(x2) & _human_NE(x2) & kidnap_VB(e1,x4,x2) & on_IN(e1,x3) & 13th_NN(x3) & of_IN(x3,x1) & January_1990_NN(x1)

@2006 Language Computer Corporation


Cogex enhancements 2 3

COGEX Enhancements (2/3)

  • Axioms on Demand

    • Lexical Chains

      • Consider the first k=3 senses for each word

      • Maximum length of a lexical chain = 3

      • DERIVATIONAL WordNet relation is ambiguous with respect to the role of the noun

        • Derivation-ACT: employ_VB(e1,x1,x2) → employment_NN(e1)

        • Derivation-AGENT: employ_VB(e1,x1,x2) → employer_NN(x1)

        • Derivation-THEME: employ_VB(e1,x1,x2) → employee_NN(x2)

      • Morphological derivations between adjectives and verbs

@2006 Language Computer Corporation


Cogex enhancements 2 31

COGEX Enhancements (2/3)

  • Axioms on Demand

    • Lexical Chains

      • Augment with the NE predicate for NE target concepts

        • nicaraguan_JJ(x1,x2) → Nicaragua_NN(x1) & _country_NE(x1)

      • Discard lexical chains

        • with more than 2 HYPONYMY relations (H too specific)

        • with a HYPONYMY followed by an ISA

          • Chicago_NN(x1) → Detroit_NN(x1)

        • which include general concepts: object/NN, act/VB, be/VB

          • ni= number of hyponyms of concept ci

          • N = number of concepts in ci’s hierarchy

@2006 Language Computer Corporation


More axioms

More Axioms

  • Another 73 World Knowledge axioms

  • Semantic Calculus – combinations of two semantic relations (82 axioms)

    • ISA, KINSHIP, CAUSE are transitive relations

    • ISA_SR(x1,x2) & PAH_SR(x3,x2) → PAH_SR(x3,x2)

      • Mike is a rich man → Mike is rich

  • Temporal Reasoning Axioms (Clark et al., 2005) (65 axioms)

    • Dates entail more general times

      • October 2000 → year 2000

    • during_TMP(e1,e2) & during_TMP(e2,e3) → during_TMP(e1,e3)

@2006 Language Computer Corporation


Cogex enhancements 3 3

COGEX Enhancements (3/3)

  • Proof Re-Scoring

    • (T)  smart people →  people (H)

    • (T)  people →  smart people (H)

      • Entities mentioned in T and H are existentially quantified

    • Universally quantified T and H entities

      • (T)  people →  smart people (H)

      • (T)  smart people →  people (H)

@2006 Language Computer Corporation


Shallow lexical alignment

Shallow Lexical Alignment

  • Compute the edit distance between T and H

    • Cost (deletion of a word from T) = 0

    • Cost (replace of a word from T with another in H) = ∞

    • Cost (insert a word from H) =

    • Edit distance between synonyms = 0

@2006 Language Computer Corporation


Results

Results

  • IE: score given by COGEXC with some correction from COGEXD

  • IR: the highest contribution is made by LexAlign (~62%)

  • COGEXD better on IE, IR, QA (~69% accuracy)

  • COGEXC better on SUM (~66% accuracy)

  • Three-way combination outperforms any individual results and any two-system combination

Learned parameters:

@2006 Language Computer Corporation


Results1

Results

  • Higher accuracy on the SUM task

    • SUM is the highest accuracy task for all systems (false entailment pairs had H completely unrelated with the texts T)

  • IE: highest number of false positives

  • Need more World Knowledge

    • (QA task) 15 safety violations → numerous safety violations

  • Upper bound (human performance) for RTE2 test

    • 97% proportional agreement

    • Kappa agreement: K = 0.94 (good agreement)

    • Fewer controversial examples in this year’s test

  • Performance on RTE1 test: 69% accuracy

@2006 Language Computer Corporation


Future work

Future Work

  • Other types of context: report, planning, etc.

    • Pairs (T:X said Y, H:Y) labeled as both TRUE and FALSE

  • Need for more axioms

    • Paraphrase acquisition (phrase1→ phrase2)

    • Automatic gathering of semantic axioms

      • Lexical chains link only concepts

      • WordNet gloss axioms link a concept to a phrase

@2006 Language Computer Corporation


Thank you

Thank You !

Questions?


  • Login