Recognizing partial textual entailment
This presentation is the property of its rightful owner.
Sponsored Links
1 / 72

Recognizing Partial Textual Entailment PowerPoint PPT Presentation


  • 145 Views
  • Uploaded on
  • Presentation posted in: General

Recognizing Partial Textual Entailment. Omer Levy Ido Dagan Natural Language Processing Lab Bar- Ilan University. Torsten Zesch Iryna Gurevych Ubiquitous Knowledge Processing Lab Technische Universität Darmstadt. Textual Entailment. T: muscles move bones.

Download Presentation

Recognizing Partial Textual Entailment

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Recognizing Partial Textual Entailment

Omer LevyIdo Dagan

Natural Language Processing Lab

Bar-Ilan University

TorstenZeschIrynaGurevych

Ubiquitous Knowledge Processing Lab

TechnischeUniversität Darmstadt


Textual Entailment


T: muscles move bones


T: muscles move bonesH: muscles generate movement


T: muscles move bonesH: muscles generate movement


T: muscles move bonesH: the muscles’ main job is to move bones

?


T: muscles move bonesH: the muscles’ main job is to move bones


Complete Textual Entailment


Complete Textual Entailment


Complete Textual Entailment

.Partial.


Complete Textual Entailment

.Partial.

(Nielsen et al, 2009)


T: muscles move bonesH: the muscles’ main job is to move bones


T: muscles move bones(main job of muscles) (muscles move) (bones are moved)H: the muscles’ main job is to move bones


T: muscles move bones(main job of muscles) (muscles move) (bones are moved)H: the muscles’ main job is to move bones


T: muscles move bones(main job of muscles) (muscles move) (bones are moved)H: the muscles’ main job is to move bones

Student Response Analysis


T: muscles move bones(main job of muscles) (muscles move) (bones are moved)H: the muscles’ main job is to move bones

Student Response Analysis

Summarization

Information Extraction


T: muscles move bones(main job of muscles) (muscles move) (bones are moved)H: the muscles’ main job is to move bones

Facets


T: muscles move bones(main job of muscles) (muscles move) (bones are moved)H: the muscles’ main job is to move bones(main job, muscles)(muscles, move) (move, bones)

Facets


T: muscles move bones(main job of muscles) (muscles move) (bones are moved)H: the muscles’ main job is to move bones(main job, muscles) (muscles, move) (move, bones)

Facets


T: muscles move bones(main job of muscles) (muscles move) (bones are moved)H: the muscles’ main job is to move bones(main job, muscles) (muscles, move) (move, bones)

Facets


Facet: a pair of terms and their implied semantic relation.


Facet: a pair of termsand their implied semantic relation.

(move, bones)


Facet: a pair of termsand their implied semantic relation.

(move, bones)

theme

… is to move bones


Recognizing Faceted Entailment


T: muscles move bonesH: the muscles’ main job is to move bones

RecognizingFacetedEntailment


T: muscles move bonesH: the muscles’ main job is to move bones

RecognizingFacetedEntailment


T: muscles move bonesH: the muscles’ main job is to move bones

RecognizingFacetedEntailment


T: muscles move bonesH: the muscles’ main job is to move bones

RecognizingFacetedEntailment


What did we do?


Approach: LeverageExisting Methods


Approach: Leverage Existing Methods


Exact MatchP: 96%, R: 30%penicillincurespneumonia(penicillin, cure)Lexical Inference(penicillin, cure) pneumonia is treated by penicillinLexical-Syntactic Inference

Approach: Leverage Existing Methods


Exact MatchP: 96%, R: 30%penicillincurespneumonia(penicillin, cure)Lexical Inference(penicillin, cure) pneumonia is treated by penicillinLexical-Syntactic Inference

Approach: Leverage Existing Methods


Exact Matchpenicillin cures pneumonia (penicillin, cure)Lexical InferenceWordNetpneumonia is treated by penicillin(penicillin, cure)Lexical-Syntactic Inference

Approach: Leverage Existing Methods


Exact Matchpenicillin cures pneumonia (penicillin, cure)Lexical InferenceWordNetpneumonia is treated by penicillin(penicillin, cure)Lexical-Syntactic Inference

Approach: Leverage Existing Methods


Exact Matchpenicillin cures pneumonia (penicillin, cure)Lexical Inferencepneumonia is treated by penicillin (penicillin, cure)Lexical-Syntactic Inference

Approach: Leverage Existing Methods

penicillin cures pneumonia

pneumonia is treated by penicillin


Bar-Ilan University Textual Entailment Engine(Stern and Dagan, 2011)


BIUTEE


BIUTEE

T H

cures

penicillin

pneumonia


BIUTEE

T H

cures

treated

penicillin

pneumonia

pneumonia

is

by

penicillin


BIUTEE

T H

cures

treated

penicillin

pneumonia

pneumonia

is

by

penicillin


cures

penicillin

pneumonia


cure treat

(lexical)

cures

treats

penicillin

penicillin

pneumonia

pneumonia


cure treat

(lexical)

cures

treats

treated

active passive

(syntactic)

penicillin

penicillin

pneumonia

pneumonia

pneumonia

is

by

penicillin


Facets to Dependencies

treated

pneumonia

is

by

penicillin


Facets to Dependencies

treated

pneumonia

is

by

penicillin


BaselineBaseLexBaseSynMajority

Decision Mechanisms

Exact Match

Exact Match OR Lexical Inference

Exact Match OR Syntactic Inference

Lexical Inference

Exact Match OR AND

Syntactic Inference


BaselineBaseLexBaseSynMajority

Decision Mechanisms

Exact Match

Exact Match OR Lexical Inference

Exact Match OR Syntactic Inference

Lexical Inference

Exact Match OR AND

Syntactic Inference


BaselineBaseLexBaseSynMajority

Decision Mechanisms

Exact Match

Exact Match OR WordNet

Exact Match OR BIUTEE

Lexical Inference

Exact Match OR AND

Syntactic Inference


BaselineBaseLexBaseSynMajority

Decision Mechanisms

Exact Match

Exact Match OR WordNet

Exact Match OR BIUTEE

WordNet

Exact Match OR AND

BIUTEE


SemEval 2013 Task 7


SciEntsBank(Djikovska et al, 2012)15 Domains5,106 Student Responses16,263 Facets

Dataset


SciEntsBank(Djikovska et al, 2012)15 Domains5,106 Student Responses16,263 Facets

Dataset


SciEntsBank(Djikovska et al, 2012)15 Domains5,106 Student Responses16,263 Facets

Dataset


SciEntsBank(Djikovska et al, 2012)15 Domains13,145 Train Facets16,263 Test Facets

Dataset


SciEntsBank(Djikovska et al, 2012)15 Domains13,145 Train Facets16,263 Test Facets

Dataset


Unseen Answers9%Unseen Questions12%Unseen Domains79%

Scenarios


Unseen Answers9%Unseen Questions12%Unseen Domains79%

Scenarios


Unseen Answers9%Unseen Questions12%Unseen Domains79%

Scenarios


Unseen Answers9%Unseen Questions12%Unseen Domains79%

Scenarios


Results


Performance (Weighted F1)


Summary


Problem: Need fine-grained entailment Solution: Partial Textual Entailment Approach: Leverage existing methods Results: Significant improvement


Problem: Need fine-grained entailment Solution: Partial Textual Entailment Approach: Leverage existing methods Results: Significant improvement


Problem: Need fine-grained entailment Solution: Partial Textual Entailment Approach:Leverage existing methods Results: Significant improvement


Problem: Need fine-grained entailment Solution: Partial Textual Entailment Approach:Leverage existing methodsResults: Significant improvement


Partial Textual Entailment is interesting


Partial Textual Entailment is useful


Partial Textual Entailment is practical


Partial Textual Entailment is interesting


Questions?


  • Login