determining the hierarchical structure of perspective and speech expressions l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Determining the Hierarchical Structure of Perspective and Speech Expressions PowerPoint Presentation
Download Presentation
Determining the Hierarchical Structure of Perspective and Speech Expressions

Loading in 2 Seconds...

play fullscreen
1 / 38

Determining the Hierarchical Structure of Perspective and Speech Expressions - PowerPoint PPT Presentation


  • 290 Views
  • Uploaded on

Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science Events in the News Reporting events Reporting in text

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Determining the Hierarchical Structure of Perspective and Speech Expressions' - Audrey


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
determining the hierarchical structure of perspective and speech expressions

Determining the Hierarchical Structure of Perspective and Speech Expressions

Eric Breck and Claire Cardie

Cornell University

Department of Computer Science

events in the news
Events in the News

Cornell University Computer Science COLING 2004

reporting events
Reporting events

Cornell University Computer Science COLING 2004

reporting in text
Reporting in text
  • Clappsums upthe environmental movement’s reaction: “The polluters are unreasonable’’
  • Charlie was angry at Alice’s claim that Bob was unhappy

Cornell University Computer Science COLING 2004

perspective and speech expressions pse s
Perspective and Speech Expressions (pse’s)
  • A perspective expression is text denoting an explicit opinion, belief, sentiment, etc.
    • The actor was elated that …
    • John’sfirm belief in …
  • A speech expression is text denoting spoken or written communication
    • … arguedthe attorney ...
    • … the 9/11 Commission’s finalreport …

Cornell University Computer Science COLING 2004

grand vision

writer

(implicit)

Charlie

angry

Alice

claim

Bob

unhappy

Grand Vision

Charlie was angry at Alice’s claim that Bob was unhappy

that Bob was unhappy

Cornell University Computer Science COLING 2004

this work

(implicit)

angry

claim

unhappy

This Work

Cornell University Computer Science COLING 2004

system output pse hierarchy

(implicit)

angry

claim

unhappy

System Output: Pse Hierarchy

78% accurate!

Charlie was angry at Alice’s claim that Bob was unhappy

Cornell University Computer Science COLING 2004

related work abstract
Related Work: Abstract
  • Bergler, 1993
    • Lexical semantics of reporting verbs
  • Gerard, 2000
    • Abstract model of news reader

Cornell University Computer Science COLING 2004

related work concrete
Related Work: Concrete
  • Bethard et al., 2004
    • Extract propositional opinions & holders
  • Wiebe, 1994
    • Tracks “point of view” in narrative text
  • Wiebe et al., 2003
    • Preliminary results on pse identification
  • Gildea and Jurafsky, 2002
    • Semantic Role ID - use for finding sources?

Cornell University Computer Science COLING 2004

baseline 1 only filter through writer

Only 66% correct

unhappy

unhappy

unhappy

Baseline 1: Only filter through writer

(implicit)

angry

claim

Cornell University Computer Science COLING 2004

baseline 2 dependency tree

claim

claim

unhappy

unhappy

Baseline 2: Dependency Tree

(implicit)

angry

72% correct

claim

unhappy

Cornell University Computer Science COLING 2004

a learning approach
A Learning Approach
  • How do we cast the recovery of hierarchical structure as a learning problem?
  • Simplest solution
    • Learn pairwise attachment decisions
      • Is pseparent the parent of psetarget?
    • Combine decisions to form tree
  • Other solutions are possible (n-ary decisions, tree-modeling, etc.)

Cornell University Computer Science COLING 2004

training instances
Training instances

(implicit)

angry

claim

unhappy

Cornell University Computer Science COLING 2004

training instances15
Training instances

(implicit)

<unhappy, (implicit)>

angry

claim

unhappy

Cornell University Computer Science COLING 2004

training instances16
Training instances

(implicit)

<unhappy, (implicit)>

<claim, (implicit)>

angry

claim

unhappy

Cornell University Computer Science COLING 2004

training instances17
Training instances

(implicit)

<unhappy, (implicit)>

<claim, (implicit)>

<angry, (implicit)>

angry

claim

unhappy

Cornell University Computer Science COLING 2004

training instances18
Training instances

(implicit)

<unhappy, (implicit)>

<claim, (implicit)>

<angry, (implicit)>

<unhappy, claim>

<claim, unhappy>

angry

claim

unhappy

Cornell University Computer Science COLING 2004

training instances19
Training instances

(implicit)

<unhappy, (implicit)>

<claim, (implicit)>

<angry, (implicit)>

<unhappy, claim>

<claim, unhappy>

angry

claim

unhappy

<unhappy, angry>

<angry, unhappy>

Cornell University Computer Science COLING 2004

training instances20
Training instances

(implicit)

<unhappy, (implicit)>

<claim, (implicit)>

<angry, (implicit)>

<unhappy, claim>

<claim, unhappy>

angry

claim

unhappy

<unhappy, angry>

<angry, unhappy>

<angry, claim>

<claim, angry>

Cornell University Computer Science COLING 2004

decision combination
Decision Combination

(implicit)

angry

claim

unhappy

Cornell University Computer Science COLING 2004

decision combination22
Decision Combination

(implicit)

angry

0.9 <angry, (implicit)>

0.1 <angry, claim>

0.1 <angry, unhappy>

angry

claim

unhappy

Cornell University Computer Science COLING 2004

decision combination23
Decision Combination

(implicit)

angry

claim

unhappy

Cornell University Computer Science COLING 2004

decision combination24
Decision Combination

(implicit)

claim

0.5 <claim, (implicit)>

0.4 <claim, angry>

0.3 <claim, unhappy>

angry

claim

unhappy

Cornell University Computer Science COLING 2004

decision combination25
Decision Combination

(implicit)

angry

claim

unhappy

Cornell University Computer Science COLING 2004

decision combination26
Decision Combination

(implicit)

unhappy

0.7 <unhappy, claim>

0.5 <unhappy, (implicit)>

0.2 <unhappy, angry>

angry

claim

unhappy

Cornell University Computer Science COLING 2004

decision combination27
Decision Combination

(implicit)

angry

claim

unhappy

Cornell University Computer Science COLING 2004

features 1
Features(1)
  • All features based on error analysis
  • Parse-based features
    • Domination+ variants
  • Positional features
    • Relative position of pseparent and psetarget

Cornell University Computer Science COLING 2004

features 2
Features(2)
  • Lexical features
    • writer’s implicit pse
    • “said”
    • “according to”
    • part of speech
  • Genre-specific features
    • Charlie, shenoted, dislikes Chinese food.
    • “Alicedisagrees with me,” Bobsaid.

Cornell University Computer Science COLING 2004

resources
Resources
  • GATE toolkit (Cunningham et al, 2002) - part-of-speech, tokenization, sentence boundaries
  • Collins parser (1999) - extracted dependency parses
  • CASS partial parser (Abney, 1997)
  • IND decision trees (Buntine, 1993)

Cornell University Computer Science COLING 2004

slide31
Data
  • From the NRRC Multi-Perspective Question Answering workshop (Wiebe, 2002)
  • 535 newswire documents (66 for development, 469 for evaluation)
  • All pse’s annotated, along with sources and other information
    • Hierarchical pse structure annotated for each sentence*

Cornell University Computer Science COLING 2004

example truncated model
Example (truncated) model
  • One learned tree, truncated to depth 3:
    • pse0 is parent of pse1 iff
      • pse0 is (implicit)
        • And pse1 is not in quotes
      • OR pse0 is said
  • Typical trees on development data:
    • Depth ~20, ~700 leaves

Cornell University Computer Science COLING 2004

evaluation
Evaluation
  • Dependency-based metric (Lin, 1995)
    • Percentage of pse’s whose parents are identified correctly
  • Percentage of sentences with perfectly identified structure
  • Performance of binary classifier

Cornell University Computer Science COLING 2004

results
Results

Cornell University Computer Science COLING 2004

error analysis
Error Analysis
  • Pairwise decisions prevent the model from learning larger structure
  • Speech events and perspective expressions behave differently
  • Treebank-style parses don’t always have the structure we need

Cornell University Computer Science COLING 2004

future work
Future Work
  • Identify pse’s
  • Identify sources
  • Evaluate alternative structure-learning methods
  • Use the structure to generate perspective-oriented summaries

Cornell University Computer Science COLING 2004

conclusions
Conclusions
  • Understanding pse structure is important for understanding text
  • Automated analysis of pse structure is possible

Cornell University Computer Science COLING 2004

thank you
Thank you!

Cornell University Computer Science COLING 2004