Employing two question answering systems in trec 2005
1 / 18

employing two question answering systems in - PowerPoint PPT Presentation

  • Updated On :

Employing Two Question Answering Systems in TREC 2005. Harabagiu, Moldovan, et al 2005 Language Computer Corporation. Highlights. Two Systems PowerAnswer-2 : factoids (main task) PALANTIR : relationships Bells and whistles Web-boosting strategy Abductive logic prover

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'employing two question answering systems in' - Rita

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Employing two question answering systems in trec 2005 l.jpg

Employing Two Question Answering Systems inTREC 2005

Harabagiu, Moldovan, et al 2005

Language Computer Corporation

Highlights l.jpg

  • Two Systems

    PowerAnswer-2 : factoids (main task)

    PALANTIR : relationships

  • Bells and whistles

    • Web-boosting strategy

    • Abductive logic prover

    • World-knowledge axioms: XWN, SUMO,…

  • Results : “above median for all groups”

    • 53.4% Main task, 20.4% Relationships task

Trec 2005 l.jpg
TREC 2005

  • Tasks: Main (factoids), Relationships

  • What’s new

    • Question types: “Other”

    • Answer types: Events

  • Challenges

    • More complex coreference resolution

    • Temporal and other event-like constraints

    • Discovering info nuggets for “Other” questions

Challenges coreference resolution l.jpg
Challenges:Coreference resolution

  • TREC 2004: single antecedent for anaphora

  • TREC 2005: more candidate antecedents…

Challenges inter question constraints l.jpg
Challenges:Inter-Question constraints

  • A question and its answer constrain the subsequent questions

    • Correct answer to Q136.5 depends on

      • correct coreference resolution with previous Q’s

      • correct answer to Q136.4

  • Event answer types

    • Nominal answer types act as topics of subsequent questions; Events constrain subsequent questions with event-like properties: time, participants…

The lcc solution two systems l.jpg
The LCC Solution:Two Systems

  • PowerAnswer-2

    • Factoid questions

    • Includes: Abductive logic, temporal reasoner, world-knowledge axioms

    • Bonus: discover interesting and novel nuggets for “Other” questions


    • Relationship questions

    • Includes: keyword expansion, topic representation, automatic lexicon generation

Poweranswer 2 architecture l.jpg

Poweranswer 2 components l.jpg

  • Standard modules: QP, PR, AP

    • Question Processor, Passage Retrieval, Answer Processor

  • Sneaky module: WebBooster

  • Fancy module: COGEX Logic Prover

    • World-knowledge: SUMO, eXtended WordNet, JAGUAR

    • Linguistic knowledge: WordNet, manual ellipses and coreference axioms

    • “Prove” correct answers with abductive logic

    • Temporal inference from “advanced textual inference techniques”

Webbooster l.jpg

  • Exploit redundancy on web for answer ranking

    • Construct series of search engine queries

      • from “linguistic patterns” (morph/lex alternations?)

    • Extract most redundant answers from web documents

    • “Boost” (ie, increase weight of) answers from TREC collection that most closely match answers from web collection

  • Justification: the larger the set, the easier it is to pinpoint answers that more closely resemble surface form of question

  • Results: 20.8 % increase in factoid score

Cogex logic prover l.jpg
COGEX:Logic Prover

  • Convert Question  QLF, Answer  ALF

  • Perform “proof” on question over candidate answers

  • Rank answers by semantic similarity to question

    • Semantic similarity: WordNet!

      • Ex: similarity of “buy” and “own” judged by length of connecting path in WordNet

  • Results: 12.4 % increase in factoid score

Cogex temporal context reasoner l.jpg
COGEX:Temporal Context Reasoner

  • Document processing: index by dates

  • Q and A processing: represent temporal relations as triples (S, E1, E2)

    • S is temporal signal (“during”, “after”), Es are events

  • Reasoning:

    • Prefer passages that match detected temporal constraints in Q

    • Discover events related by temporal signals in the Q and candidate As

    • Perform temporal unification btw the Q and candidate As, boosting As that match Q times

  • Results: 2 % increase in factoid score

Other questions l.jpg
“Other” Questions

  • Generic definition-pattern based nuggets

    “...Russian submarine Kursk, which is lying on the sea bed in the Barents Sea...”

  • Answer-type based nuggets

    • Nugget-patterns pecific to properties of answer type

    • 33 target classes generated by Naïve Bayes classifier on WordNet synsets

      Bing Crosby  musican_person: band, singer, born, …

  • Entity-relationship based nuggets

    • Nugget patterns are based on relations to other NEs

      Akira Kurosawa AND _date

      Akira Kurosawa AND _location …

Palantir keyword selection l.jpg
PALANTIR:Keyword Selection

  • Collocation detection

    • identify complete phrases that aren’t just bags of keywords (Organization of African States)

  • Keyword Ranking

    • detect overall importance of keyword in query

    • Use keyword-density strategy for doc ranking

  • Keyword Expansion

    • Synonyms, alternate forms for keywords

Palanir topic representation l.jpg
PALANIR:Topic Representation

  • Harvest “topic signatures” from text

    • ??

  • Find relationships between topic signatures

    • Use syntax- and semantic-based relations between verbs and arguments

    • Use context-based relations that exist between entities

Palantir lexicon generation l.jpg
PALANTIR:Lexicon Generation

  • Q: Relationship questions have no single semantic answer type; how to identify appropriate answers from passages?

  • A: By generating set-types on the fly, of course!

    • Use weakly-supervised learning approach to identify semantic sets in question, then keywords relevant to that set (South American countries)

    • Automatically generate a large db of syntactic frames that represent semantic relations

Results l.jpg



Summary l.jpg

  • WebBooster – 20% increase

  • COGEX – 12% increase

  • Temporal Reasoner – 2% increase

  • Nugget-pattern discovery – 22.8% f-measure

  • PALANTIR strategies: