Subjectivity and Sentiment Analysis: from Words to Discourse

1. Subjectivity and Sentiment Analysis: from Words to Discourse Jan Wiebe Computer Science Department Intelligent Systems Program University of Pittsburgh I2R Singapore 2009

2. Burgeoning Field • Quite a large problem space • Several terms reflecting varying goals and models • Sentiment Analysis • Opinion Mining • Opinion Extraction • Subjectivity Analysis • Appraisal Analysis • Affect Sensing • Emotion Detection • Identifying Perspective • Etc.

3. What is Subjectivity? • The linguisticexpression of somebody’s opinions, sentiments, emotions, evaluations, beliefs, speculations (private states) Private state: state that is not open to objective observation or verification Quirk, Greenbaum, Leech,Svartvik (1985). Note that this particular use of subjectivity is adapted from literary theory E.G. Banfield 1982, Fludernik 1993; Wiebe PhD Dissertation 1990.

4. Examples of Subjective Expressions • References to private states • She was enthusiastic about the plan • He was boiling with anger • References to speech or writing events expressing private states • Leaders rounding condemned his verbal assault on Israel • Expressive subjective elements • That would lead to disastrous consequences • What a freak show

5. MPQA corpus: http://www.cs.pitt.edu/mpqa direct subjective span: think source: <writer, I> attitude: direct subjective span: are happy source: <writer, I, People> attitude: attitude span: think type: positive arguing intensity: medium target: inferred attitude span: are happy because Chavez has fallen type: neg sentiment intensity: medium target: attitude span: are happy type: pos sentiment intensity: medium target: target span: people are happy because Chavez has fallen target span: Chavez has fallen target span: Chavez Manually (human) Annotated News Data Wilson PhD Dissertation 2008 I think people are happy because Chavez has fallen

6. Subjectivity and Sentiment Analysis • Automatic extraction of subjectivity (opinions) expressed in text or dialog (newspapers, blogs, conversations, etc) • Sentiment analysis: specifically looking for postiive and negative sentiments

7. Why? • Subjectivity analysis systems can provide useful input to several kinds of end applications

8. Why?Opinion Question Answering • Answer Questions about Opinions Q: What is the international reaction to the reelection of Robert Mugabe as President of Zimbabwe? Stoyanov, Cardie, Wiebe EMNLP05 Somasundaran, Wilson, Wiebe, Stoyanov ICWSM07

9. Why? Information Extraction (AAAI • Filter out false hits for Information Extraction systems “The Parliamentexplodedinto fury against the government when word leaked out…” Riloff,Wiebe, Phillips AAAI05

10. Why? Recognizing Stances in Debates Firefox is more respectful of W3C internet standards while µsoft sucks by trying to force us to use their own standards to keep their monopoly. IE is much easier to use. It also is more visually pleasing. It is much more secure as well. Pro-Firefox Pro-IE

11. Why?Product Review Mining Determine if the given product/movie review is positive or negative “… was billed as a suspense thriller along the lines of Hitchcock ..... the problem here is that writing has failed some very capable actors ....” “The last half of the film is very well done . Another thing that carries this film are the superb performances ... is a very entertaining and suspenseful film...” Negative review Positive review

12. And Several Others… • Tracking sentiments toward topics over time:Is anger ratcheting up or cooling down? • Prediction (election outcomes, market trends): Will Clinton or Obama win? • Meeting summarization: What were the main opinions expressed? • Etcetera!

13. Focus • Our focus is linguistic disambiguation; how should language be interpreted? • Is it subjective in the first place? If so, is it positive or negative? What is it about? Etc. • Subjective language is highly ambiguous

14. continuum NLP methods/resources building toward full interpretations Today: several tasks along the continuum Interpretation Lexicon of keywords out of context Full contextual Interpretation of words in text or dialogue “The dream”

15. continuum Interpretation Lexicon of keywords out of context Full contextual Interpretation of words in text or dialogue Brilliant Difference Hate Interest Love …

16. Subjectivity Lexicons • Most approaches to subjectivity and sentiment analysis exploit subjectivity lexicons. • Lists of keywords that have been gathered together because they have subjective uses

17. Automatically Identifying Subjective Words • Much work in this area E.g. Hatzivassiloglou & McKeown ACL97; Wiebe AAAI00; Turney ACL02; Kamps & Marx 2002; Wiebe, Riloff, Wilson CoNLL03; Kim & Hovy 2005; Esuli & Sebastiani 2005; Subjectivity Lexicon: http://www.cs.pitt.edu/mpqa Entries from several sources (our work and others’)

18. However… • Consider the keyword “Interest”. • It is in the subjectivity lexicon. • But, what about “interest rate”, for example?

19. Dictionary Definitions senses Interest, involvement -- (a sense of concern with and curiosity about someone or something; "an interest in music") Interest -- (a fixed charge for borrowing money; usually a percentage of the amount borrowed; "how much interest do you pay on your mortgage?")

20. S O Dictionary Definitions senses Interest, involvement -- (a sense of concern with and curiosity about someone or something; "an interest in music") Interest -- (a fixed charge for borrowing money; usually a percentage of the amount borrowed; "how much interest do you pay on your mortgage?")

21. Senses • Even in subjectivity lexicons, many senses of the keywords are objective ~50% in our study! • Thus, many appearances of keywords in texts arefalse hits

22. Senses • Hisalarm grew as the election returns came in. • He set hisalarm for 7am. • His trustgrew as the candidate spoke. • His trust grew as interest rates increased.

23. WordNet Miller 1995; Fellbaum 1998

24. Examples • “There are many differences between African and Asian elephants.” • “… dividing by the absolute value of the difference from the mean…” • “Their differences only grew as they spent more time together …” • “Her support really made a difference in my life” • “The difference after subtracting X from Y…”

25. Subjectivity Sense Labeling • Automatically classifying senses as subjective or objective Wiebe & Mihalcea ACL06 Gyamfi, Wiebe, Mihalcea, Akkaya NAACL09 See also: Esuli & Sebastiani EACL06, ACL07 Andreevskaia & Bergler EACL06, LREC06 Su & Markert Coling08, NAACL09

26. Senses WordNet

27. WordNet

28. WordNet If this sense is subjective, then maybe these senses of brainy and smart-as-a-whip are as well

29. WordNet glosses

30. WordNet Examples Glosses and examples contain clues as to the subjectivity of a sense

31. WordNet Relations

32. WordNet Relations

33. Hierarchical Structure

34. Using Hierarchical Structure The higher the IC of the LCS, the more specific it is, and the more similar the seed and target sense are Information content of the lowest common subsumer Sim(t,s) = -log(p(c)) (Resnik 1995) LCS c • Being similar to a • subjective seed • More likely the target is subjective Target Sense t Seed Sense s

35. Using Hierarchical Structure LCS Target sense Seed sense

36. Using Hierarchical Structure LCS voice#1 (objective)

37. Sense Subjectivity LCS Feature LCS c4 LCS c3 LCS c2 LCS c1 Seed Sense s4 Seed Sense s3 Seed Sense s2 Target Sense t Seed Sense s1

38. Domains • Several researchers have noted that subjectivity may be domain specific • WordNet Domains (Gliozzo et al. 2005) assigns a domain label to each synset

39. Domains • Over 80% of the subjective seed senses are in 6 domains (rest are in 35) • Factotum “other” [201] garishness#2, racism#1 • Psychological features [98] horror#1, satisfaction#1 • Person [68] meanie#1,Francophobe#1 • Law [61] swindler#1, two-timer#1 • Psychology [20] ecstasy#1,indignity#1 • Sociology [20] vandalism#1,odium#1

40. Sense Subjectivity LCS Feature LCS c4 Saves computation LCS c3 The score is the feature value for t LCS c2 LCS c1 Seed Sense s4 Seed Sense s3 Domain D Seed Sense s2 Target Sense t Seed Sense s1

41. Using Hierarchical Structure Gyamfi, Wiebe, Mihalcea, Akkaya NAACL09 • Hierarchical information is combined with other WordNet-Based knowledge to classify senses as Subjective or Objective

42. Interpretation Lexicon of keywords out of context Full contextual Interpretation of words in text or dialog continuum Brilliant sense#1 S sense#2 S … Difference sense#1 O sense#2 O sense#3 S sense#4 S sense#5 O … Now we will leave the lexicon and look at disambiguation in the context of a text or conversation

43. Contextual Subjectivity Analysis Subjectivity Sentence Classifier “He spins a riveting plot which grabs and holds the reader’s interest…” S O? Do the sentences contain subjectivity? S O? “The notes do not pay interest.” E.g. Riloff & Wiebe EMNLP03 Yu & Hatzivassiloglou EMNLP03

44. Contextual Subjectivity Analysis Subjectivity Phrase Classifier “He spins a riveting plot which grabs and holds the reader’s interest…” S O? Is a phrase containing a keyword subjective? S O? “The notes do not pay interest.” Wilson, Wiebe, Hoffmann EMNLP05

45. Contextual Subjectivity Analysis “There are many differences between African and Asian elephants.” Sentiment Phrase Classifier S O? Pos, Neg, Neutral? Is a phrase containing a keyword positive, Negative, or neutral? We’ll return to this, topic after next. But first… “Their differences only grew as they spent more time together …” S O? Pos, Neg, Neutral? Wilson, Wiebe, Hoffmann EMNLP05

46. Interpretation Lexicon of keywords out of context Full contextual Interpretation of words in text or dialog continuum Brilliant sense#1 S sense#2 S … Difference sense#1 O sense#2 O sense#3 S sense#4 S sense#5 O … Contextual Subjectivity analysis Exploiting sense labels to improve the contextual classifiers

47. Sense 4 S Sense 4 “a sense of concern with and curiosity about someone or something” OSense 1“a fixed charge for borrowing money” WSD System Sense 1 Subjectivity Tagging using WSD Subjectivity Classifier “He spins a riveting plot which grabs and holds the reader’s interest…” S O? S O? “The notes do not pay interest.”

48. S Sense 4 “a sense of concern with and curiosity about someone or something” OSense 1“a fixed charge for borrowing money” Subjectivity Tagging using WSD Subjectivity Classifier “He spins a riveting plot which grabs and holds the reader’s interest…” S O Sense 4 WSD System S O Sense 1 “The notes do not pay interest.”

49. Is it one of these? Examples • “There are many differences between African and Asian elephants.” Sense#1O • “… dividing by the absolute value of the difference from the mean…” Sense#2 O • “Their differences only grew as they spent more time together …” Sense#3 S • “Her support really made a difference in my life” Sense#4 S • “The difference after subtracting X from Y…” Sense#5 O

50. Examples • “There are many differences between African and Asian elephants.” Sense#1 O • “… dividing by the absolute value of the difference from the mean…” Sense#2 O • “Their differences only grew as they spent more time together …” Sense#3 S • “Her support really made a difference in my life” Sense#4 S • “The difference after subtracting X from Y…” Sense#5 O Or one of these?