1 / 16

Semantic Model for Context-Dependent Claim Classification

This study focuses on the semantic model for classifying context-dependent claims into pro and con positions, targeting identification, sentiment analysis, and consistency evaluation. Using a dataset of 2394 claims, a continuous claim stance classification model is proposed. The approach includes identifying targets, sentiment towards targets, and determining contrast or consistency between targets. The evaluation involves a test set of 30 topics to assess accuracy and coverage. Challenges include the performance of contrast classifier and issues with certain claim types. Insights and methods for semantic model enhancement and optimized contrast scoring are discussed.

jesusr
Download Presentation

Semantic Model for Context-Dependent Claim Classification

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Stance Classification of Context-Dependent Claims Sen Han leonihsam@gmail.com 4. June 2019 1

  2. Outline • Introduction • Motivation • Model • Dataset • Semantic model • Three sub-tasks • Target identification • Sentiment identification • Target : Contrastive or consistent • Evaluation • Result • Challenge • Discussion 16 Sen Han 2

  3. Introduction • Ondemand generation of pro and con arguments for a given controversial topic would be of great value in many domains. • Debating support and decision support • E.g • “The sale of violent video games to minors should be banned.”(controversial topic) • (pro)Violence is damaging the kindness of children. (claim) • (pro)Love makes world peaceful and harmony.(claim) 16 Sen Han 3

  4. Motivation • Sorting extracted claims into pro and con improves the usability of both debating and decision support systems. • Topic t and extracted claims c from t • Arbitrary domains • Short claim sentences • Confidence ranking • Semantic modelfor predicting claim stance(Pro/Con) • Three sub- problems • Identify targets • Identify sentiment towards target • Consistency or contrast between targets 16 Sen Han 4

  5. Dataset • Randomly selected 55 topics worded as “This house+…” • 2394 Claims assessed polarity by Five annotators 16 Sen Han 5

  6. Semantic Model • (Continous)Claim stance classification model • Claim c, Topic t • Claim-target xc, topic-target xt • Claim-Sentiment sc, topic-Sentiment st • Contrast relation R(xc,xt) • e.g • Real-valued prediction • Top k predictions • threshold • Class-sign, Confidence-absolute value 16 Sen Han 6

  7. Target Extraction and Targeted Sentiment • Open-domain, generic target identification • xt an st are given, focusing on finding xc and scwith L2-regularized logistic regression classifier • Candidate with highest classifier confidence is predicted to be target 16 Sen Han 7

  8. Targeted sentiment analysis • sentiment score=(p-n)/(p+n+1) • p and n are the weighted sums of positive and negative sentiments detected in the claim • A weight of d^-0.5, d is distance between target and sentiment term • E.g • “ Children with many siblings receive fewer resources ” • P=1^-0.5+3^-0.5 n= 2^0.5 sc=(p-n)/(p+n+1)~=0.256 16 Sen Han 8

  9. Contrast score • Anchor pair • (max)w(u) x |r(u,v)| x w(v) • w(u)=tf(u)/df(u) • Relatedness measure • P(Lex+|u, v) =Freq(u,Lex+,v)/ Freq(u,v) • P(Lex+|u, v) if P(Lex+|u, v) >P(Lex-|u, v), and -P(Lex-|u, v),otherwise. • Contrast score • p(xc,ac) x r(ac,at) x p(xt,at) • E.g 16 Sen Han 9

  10. Evaluation • a training set, comprising 25 topics (1,039 claims) • a test set, comprising 30 topics (1,355 claims). • The training set was used to train the target identification classifier and the contrast classifier in our system • Predicting the test data • Accuracy and coverage • Two baseline • Our model • Combine our model with baseline 16 Sen Han 10

  11. Result 16 Sen Han 11

  12. Challenge • The contrast classifier does not outperform majority baseline over the whole dataset. • Higher accuracy in high coverage. • Some types of claims do not fit: • The sale of violent video games to minors(topic) • Parents, not government bureaucrats, have the right to decide what is appropriate for their children(claim) • Sentiment analyzer works not well on it 16 Sen Han 12

  13. Discussion 16 Sen Han 13

  14. Thanks! 16 Sen Han 14

  15. Semantic model • Topic: This house supports the freedom of speech • (Pro)“ in favor of free discussion ” or “ criticizing censorship ” • (consistent) “ freedom of speech ” and “ free discussion ” • (contrastive) “ freedom of speech ” and “censorship” 16 Sen Han 15

  16. Contrast score • Two corpus: Query log( advertising and marketing), Wikipedia(Military vs diplomatic) • Targets “ atheism” and “ denying the existence of God ” • Anchor pair candidates (atheism, God), (atheism, existence)… • R(atheism,God), w(atheism) and w(God) • w(u)=tf(u)/df(u) • Relatedness measure: P(Lex+|atheism, God) • Contrast score [0,1], consistency • p(“ atheism”, “ atheism ”) x p(“atheism”, “God ”) x p(“denying the existence of God”, “God”) 16 Sen Han 16

More Related