Recognizing partial textual entailment
This presentation is the property of its rightful owner.
Sponsored Links
1 / 72

Recognizing Partial Textual Entailment PowerPoint PPT Presentation


  • 139 Views
  • Uploaded on
  • Presentation posted in: General

Recognizing Partial Textual Entailment. Omer Levy Ido Dagan Natural Language Processing Lab Bar- Ilan University. Torsten Zesch Iryna Gurevych Ubiquitous Knowledge Processing Lab Technische Universität Darmstadt. Textual Entailment. T: muscles move bones.

Download Presentation

Recognizing Partial Textual Entailment

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Recognizing partial textual entailment

Recognizing Partial Textual Entailment

Omer LevyIdo Dagan

Natural Language Processing Lab

Bar-Ilan University

TorstenZeschIrynaGurevych

Ubiquitous Knowledge Processing Lab

TechnischeUniversität Darmstadt


Textual entailment

Textual Entailment


T muscles move bones

T: muscles move bones


T muscles move bones h muscles generate movement

T: muscles move bonesH: muscles generate movement


T muscles move bones h muscles generate movement1

T: muscles move bonesH: muscles generate movement


T muscles move bones h the muscles main job is to move bones

T: muscles move bonesH: the muscles’ main job is to move bones

?


T muscles move bones h the muscles main job is to move bones1

T: muscles move bonesH: the muscles’ main job is to move bones


Complete textual entailment

Complete Textual Entailment


Complete textual entailment1

Complete Textual Entailment


Complete textual entailment2

Complete Textual Entailment

.Partial.


Complete textual entailment3

Complete Textual Entailment

.Partial.

(Nielsen et al, 2009)


T muscles move bones h the muscles main job is to move bones2

T: muscles move bonesH: the muscles’ main job is to move bones


Recognizing partial textual entailment

T: muscles move bones(main job of muscles) (muscles move) (bones are moved)H: the muscles’ main job is to move bones


Recognizing partial textual entailment

T: muscles move bones(main job of muscles) (muscles move) (bones are moved)H: the muscles’ main job is to move bones


Recognizing partial textual entailment

T: muscles move bones(main job of muscles) (muscles move) (bones are moved)H: the muscles’ main job is to move bones

Student Response Analysis


Recognizing partial textual entailment

T: muscles move bones(main job of muscles) (muscles move) (bones are moved)H: the muscles’ main job is to move bones

Student Response Analysis

Summarization

Information Extraction


Recognizing partial textual entailment

T: muscles move bones(main job of muscles) (muscles move) (bones are moved)H: the muscles’ main job is to move bones

Facets


Recognizing partial textual entailment

T: muscles move bones(main job of muscles) (muscles move) (bones are moved)H: the muscles’ main job is to move bones(main job, muscles)(muscles, move) (move, bones)

Facets


Recognizing partial textual entailment

T: muscles move bones(main job of muscles) (muscles move) (bones are moved)H: the muscles’ main job is to move bones(main job, muscles) (muscles, move) (move, bones)

Facets


Recognizing partial textual entailment

T: muscles move bones(main job of muscles) (muscles move) (bones are moved)H: the muscles’ main job is to move bones(main job, muscles) (muscles, move) (move, bones)

Facets


Facet a pair of terms and their implied semantic relation

Facet: a pair of terms and their implied semantic relation.


Facet a pair of terms and their implied semantic relation1

Facet: a pair of termsand their implied semantic relation.

(move, bones)


Facet a pair of terms and their implied semantic relation2

Facet: a pair of termsand their implied semantic relation.

(move, bones)

theme

… is to move bones


Recognizing partial textual entailment

Recognizing Faceted Entailment


T muscles move bones h the muscles main job is to move bones3

T: muscles move bonesH: the muscles’ main job is to move bones

RecognizingFacetedEntailment


T muscles move bones h the muscles main job is to move bones4

T: muscles move bonesH: the muscles’ main job is to move bones

RecognizingFacetedEntailment


T muscles move bones h the muscles main job is to move bones5

T: muscles move bonesH: the muscles’ main job is to move bones

RecognizingFacetedEntailment


T muscles move bones h the muscles main job is to move bones6

T: muscles move bonesH: the muscles’ main job is to move bones

RecognizingFacetedEntailment


What did we do

What did we do?


Recognizing partial textual entailment

Approach: LeverageExisting Methods


Recognizing partial textual entailment

Approach: Leverage Existing Methods


Recognizing partial textual entailment

Exact MatchP: 96%, R: 30%penicillincurespneumonia(penicillin, cure)Lexical Inference(penicillin, cure) pneumonia is treated by penicillinLexical-Syntactic Inference

Approach: Leverage Existing Methods


Recognizing partial textual entailment

Exact MatchP: 96%, R: 30%penicillincurespneumonia(penicillin, cure)Lexical Inference(penicillin, cure) pneumonia is treated by penicillinLexical-Syntactic Inference

Approach: Leverage Existing Methods


Recognizing partial textual entailment

Exact Matchpenicillin cures pneumonia (penicillin, cure)Lexical InferenceWordNetpneumonia is treated by penicillin(penicillin, cure)Lexical-Syntactic Inference

Approach: Leverage Existing Methods


Recognizing partial textual entailment

Exact Matchpenicillin cures pneumonia (penicillin, cure)Lexical InferenceWordNetpneumonia is treated by penicillin(penicillin, cure)Lexical-Syntactic Inference

Approach: Leverage Existing Methods


Recognizing partial textual entailment

Exact Matchpenicillin cures pneumonia (penicillin, cure)Lexical Inferencepneumonia is treated by penicillin (penicillin, cure)Lexical-Syntactic Inference

Approach: Leverage Existing Methods

penicillin cures pneumonia

pneumonia is treated by penicillin


Bar ilan university textual entailment engine stern and dagan 2011

Bar-Ilan University Textual Entailment Engine(Stern and Dagan, 2011)


Biutee

BIUTEE


Biutee1

BIUTEE

T H

cures

penicillin

pneumonia


Biutee2

BIUTEE

T H

cures

treated

penicillin

pneumonia

pneumonia

is

by

penicillin


Biutee3

BIUTEE

T H

cures

treated

penicillin

pneumonia

pneumonia

is

by

penicillin


Recognizing partial textual entailment

cures

penicillin

pneumonia


Recognizing partial textual entailment

cure treat

(lexical)

cures

treats

penicillin

penicillin

pneumonia

pneumonia


Recognizing partial textual entailment

cure treat

(lexical)

cures

treats

treated

active passive

(syntactic)

penicillin

penicillin

pneumonia

pneumonia

pneumonia

is

by

penicillin


Facets to dependencies

Facets to Dependencies

treated

pneumonia

is

by

penicillin


Facets to dependencies1

Facets to Dependencies

treated

pneumonia

is

by

penicillin


Baseline baselex basesyn majority

BaselineBaseLexBaseSynMajority

Decision Mechanisms

Exact Match

Exact Match OR Lexical Inference

Exact Match OR Syntactic Inference

Lexical Inference

Exact Match OR AND

Syntactic Inference


Baseline baselex basesyn majority1

BaselineBaseLexBaseSynMajority

Decision Mechanisms

Exact Match

Exact Match OR Lexical Inference

Exact Match OR Syntactic Inference

Lexical Inference

Exact Match OR AND

Syntactic Inference


Baseline baselex basesyn majority2

BaselineBaseLexBaseSynMajority

Decision Mechanisms

Exact Match

Exact Match OR WordNet

Exact Match OR BIUTEE

Lexical Inference

Exact Match OR AND

Syntactic Inference


Baseline baselex basesyn majority3

BaselineBaseLexBaseSynMajority

Decision Mechanisms

Exact Match

Exact Match OR WordNet

Exact Match OR BIUTEE

WordNet

Exact Match OR AND

BIUTEE


Semeval 2013 task 7

SemEval 2013 Task 7


Scientsbank djikovska et al 2012 15 domains 5 106 student responses 16 263 facets

SciEntsBank(Djikovska et al, 2012)15 Domains5,106 Student Responses16,263 Facets

Dataset


Scientsbank djikovska et al 2012 15 domains 5 106 student responses 16 263 facets1

SciEntsBank(Djikovska et al, 2012)15 Domains5,106 Student Responses16,263 Facets

Dataset


Scientsbank djikovska et al 2012 15 domains 5 106 student responses 16 263 facets2

SciEntsBank(Djikovska et al, 2012)15 Domains5,106 Student Responses16,263 Facets

Dataset


Scientsbank djikovska et al 2012 15 domains 13 145 train facets 16 263 test facets

SciEntsBank(Djikovska et al, 2012)15 Domains13,145 Train Facets16,263 Test Facets

Dataset


Scientsbank djikovska et al 2012 15 domains 13 145 train facets 16 263 test facets1

SciEntsBank(Djikovska et al, 2012)15 Domains13,145 Train Facets16,263 Test Facets

Dataset


Unseen answers 9 unseen questions 12 unseen domains 79

Unseen Answers9%Unseen Questions12%Unseen Domains79%

Scenarios


Unseen answers 9 unseen questions 12 unseen domains 791

Unseen Answers9%Unseen Questions12%Unseen Domains79%

Scenarios


Unseen answers 9 unseen questions 12 unseen domains 792

Unseen Answers9%Unseen Questions12%Unseen Domains79%

Scenarios


Unseen answers 9 unseen questions 12 unseen domains 793

Unseen Answers9%Unseen Questions12%Unseen Domains79%

Scenarios


Results

Results


Performance weighted f1

Performance (Weighted F1)


Summary

Summary


Recognizing partial textual entailment

Problem: Need fine-grained entailment Solution: Partial Textual Entailment Approach: Leverage existing methods Results: Significant improvement


Recognizing partial textual entailment

Problem: Need fine-grained entailment Solution: Partial Textual Entailment Approach: Leverage existing methods Results: Significant improvement


Recognizing partial textual entailment

Problem: Need fine-grained entailment Solution: Partial Textual Entailment Approach:Leverage existing methods Results: Significant improvement


Recognizing partial textual entailment

Problem: Need fine-grained entailment Solution: Partial Textual Entailment Approach:Leverage existing methodsResults: Significant improvement


Partial textual entailment is interesting

Partial Textual Entailment is interesting


Partial textual entailment is useful

Partial Textual Entailment is useful


Partial textual entailment is practical

Partial Textual Entailment is practical


Partial textual entailment is interesting1

Partial Textual Entailment is interesting


Questions

Questions?


  • Login