160 likes | 163 Views
S tance Classification of Context-Dependent Claims. Sen Han leonihsam@gmail.com. 4 . June 2019. 1. Outline. Introduction Motivation Model Dataset Semantic model Three sub-tasks Target identification Sentiment identification Target : Contrastive or consistent Evaluation Result
E N D
Stance Classification of Context-Dependent Claims Sen Han leonihsam@gmail.com 4. June 2019 1
Outline • Introduction • Motivation • Model • Dataset • Semantic model • Three sub-tasks • Target identification • Sentiment identification • Target : Contrastive or consistent • Evaluation • Result • Challenge • Discussion 16 Sen Han 2
Introduction • Ondemand generation of pro and con arguments for a given controversial topic would be of great value in many domains. • Debating support and decision support • E.g • “The sale of violent video games to minors should be banned.”(controversial topic) • (pro)Violence is damaging the kindness of children. (claim) • (pro)Love makes world peaceful and harmony.(claim) 16 Sen Han 3
Motivation • Sorting extracted claims into pro and con improves the usability of both debating and decision support systems. • Topic t and extracted claims c from t • Arbitrary domains • Short claim sentences • Confidence ranking • Semantic modelfor predicting claim stance(Pro/Con) • Three sub- problems • Identify targets • Identify sentiment towards target • Consistency or contrast between targets 16 Sen Han 4
Dataset • Randomly selected 55 topics worded as “This house+…” • 2394 Claims assessed polarity by Five annotators 16 Sen Han 5
Semantic Model • (Continous)Claim stance classification model • Claim c, Topic t • Claim-target xc, topic-target xt • Claim-Sentiment sc, topic-Sentiment st • Contrast relation R(xc,xt) • e.g • Real-valued prediction • Top k predictions • threshold • Class-sign, Confidence-absolute value 16 Sen Han 6
Target Extraction and Targeted Sentiment • Open-domain, generic target identification • xt an st are given, focusing on finding xc and scwith L2-regularized logistic regression classifier • Candidate with highest classifier confidence is predicted to be target 16 Sen Han 7
Targeted sentiment analysis • sentiment score=(p-n)/(p+n+1) • p and n are the weighted sums of positive and negative sentiments detected in the claim • A weight of d^-0.5, d is distance between target and sentiment term • E.g • “ Children with many siblings receive fewer resources ” • P=1^-0.5+3^-0.5 n= 2^0.5 sc=(p-n)/(p+n+1)~=0.256 16 Sen Han 8
Contrast score • Anchor pair • (max)w(u) x |r(u,v)| x w(v) • w(u)=tf(u)/df(u) • Relatedness measure • P(Lex+|u, v) =Freq(u,Lex+,v)/ Freq(u,v) • P(Lex+|u, v) if P(Lex+|u, v) >P(Lex-|u, v), and -P(Lex-|u, v),otherwise. • Contrast score • p(xc,ac) x r(ac,at) x p(xt,at) • E.g 16 Sen Han 9
Evaluation • a training set, comprising 25 topics (1,039 claims) • a test set, comprising 30 topics (1,355 claims). • The training set was used to train the target identification classifier and the contrast classifier in our system • Predicting the test data • Accuracy and coverage • Two baseline • Our model • Combine our model with baseline 16 Sen Han 10
Result 16 Sen Han 11
Challenge • The contrast classifier does not outperform majority baseline over the whole dataset. • Higher accuracy in high coverage. • Some types of claims do not fit: • The sale of violent video games to minors(topic) • Parents, not government bureaucrats, have the right to decide what is appropriate for their children(claim) • Sentiment analyzer works not well on it 16 Sen Han 12
Discussion 16 Sen Han 13
Thanks! 16 Sen Han 14
Semantic model • Topic: This house supports the freedom of speech • (Pro)“ in favor of free discussion ” or “ criticizing censorship ” • (consistent) “ freedom of speech ” and “ free discussion ” • (contrastive) “ freedom of speech ” and “censorship” 16 Sen Han 15
Contrast score • Two corpus: Query log( advertising and marketing), Wikipedia(Military vs diplomatic) • Targets “ atheism” and “ denying the existence of God ” • Anchor pair candidates (atheism, God), (atheism, existence)… • R(atheism,God), w(atheism) and w(God) • w(u)=tf(u)/df(u) • Relatedness measure: P(Lex+|atheism, God) • Contrast score [0,1], consistency • p(“ atheism”, “ atheism ”) x p(“atheism”, “God ”) x p(“denying the existence of God”, “God”) 16 Sen Han 16