A confidence model for syntactically motivated entailment proofs
Download
1 / 19

A Confidence Model for Syntactically-Motivated Entailment Proofs - PowerPoint PPT Presentation


  • 84 Views
  • Uploaded on

A Confidence Model for Syntactically-Motivated Entailment Proofs. Asher Stern & Ido Dagan ISCOL June 2011, Israel. Recognizing Textual Entailment (RTE). Given a text, T , and a hypothesis, H Does T entail H. Example. T : An explosion caused by gas took place at a Taba hotel

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'A Confidence Model for Syntactically-Motivated Entailment Proofs' - nijole


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
A confidence model for syntactically motivated entailment proofs

A Confidence Model for Syntactically-Motivated Entailment Proofs

Asher Stern & Ido Dagan

ISCOL

June 2011, Israel


Recognizing textual entailment rte
Recognizing Textual Entailment (RTE) Proofs

  • Given a text, T, and a hypothesis, H

  • Does T entail H

Example

  • T: An explosion caused by gas took place at a Taba hotel

  • H: A blast occurred at a hotel in Taba.


Proof over p arse t rees
Proof Over ProofsParse Trees

T = T0→ T1→ T2→ ... →Tn = H


Bar ilan proof system entailment rules
Bar ProofsIlan Proof System - Entailment Rules

Generic Syntactic

Lexical Syntactic

Lexical

explosion

blast


Bar ilan proof system
Bar ProofsIlan Proof System

H: A blast occurred at a hotel in Taba.

An explosion caused by gas took place at a Taba hotel

A blast caused by gas took place at a Taba hotel

A blast took place at a Taba hotel

A blast occurred at a Taba hotel

A blast occurred at a hotel in Taba.

Lexical

Lexical syntactic

Syntactic


Tree edit distance
Tree-Edit-Distance Proofs

Insurgents attacked soldiers -> Soldiers were attacked by insurgents


Proof over parse trees
Proof Proofsover parse trees

Which steps?

How to classify?

Decide “yes” if and only if a proof was found

Almost always “no”

Cannot handle knowledge inaccuracies

Estimate a confidence to the proof correctness

  • Tree-Edits

    • Regular or custom

  • Entailment Rules


Proof systems
Proof systems Proofs

TED based

Entailment Rules based

Linguistically motivated

Rich knowledge

No estimation of proof correctness

Incomplete proofs

Mixed system with ad-hoc approximate match criteria

  • Estimate the cost of a proof

  • Complete proofs

  • Arbitrary operations

  • Limited knowledge

Our System

  • The benefits of both worlds, and more!

    • Linguistically motivated complete proofs

    • Confidence model


Our method
Our Method Proofs

  • Complete proofs

    • On the fly operations

  • Cost model

  • Learning model parameters


On the fly operations
On the fly Operations Proofs

  • “On the fly” operations

    • Insert node on the fly

    • Move node / move sub-tree on the fly

    • Flip part of speech

    • Etc.

  • More syntactically motivated than Tree Edits

  • Not justified, but:

  • Their impact on the proof correctness can be estimated by the cost model.


Cost model
Cost Model Proofs

The Idea:

Represent the proof as a feature-vector

Use the vector in a learning algorithm


Cost model1
Cost Model Proofs

  • Represent a proof as F(P) = (F1, F2 … FD)

  • Define weight vector w=(w1,w2,…,wD)

  • Define proof cost

  • Classify a proof

    • b is a threshold

  • Learn the parameters (w,b)


Search algorithm
Search Algorithm Proofs

  • Need to find the “best” proof

  • “Best Proof” = proof with lowest cost

    • Assuming a weight vector is given

  • Search space is exponential

    • pruning


Parameter estimation
Parameter Estimation Proofs

  • Goal: find good weight vector and threshold (w,b)

  • Use a standard machine learning algorithm (logistic regression or linear SVM)

  • But: Training samples are not given as feature vectors

    • Learning algorithm requires training samples

    • Training samples construction requires weight vector

    • Learning weight vector done by learning algorithm

  • Iterative learning



Parameter estimation2
Parameter Estimation Proofs

  • Start with w0, a reasonable guess for weight vector

  • i=0

  • Repeat until convergence

    • Find the best proofs and construct vectors, using wi

    • Use a linear ML algorithm to find a new weight vector, wi+1

    • i = i+1


Results
Results Proofs


Conclusions
Conclusions Proofs

  • Linguistically motivated proofs

    • Complete proofs

  • Cost model

    • Estimation of proof correctness

  • Search best proof

  • Learning parameters

  • Results

    • Reasonable behavior of learning scheme


Thank you

Thank you Proofs

Q & A