KDD 2011 Doctoral Session

1. Modeling Trustworthiness of Online Content V. G. Vinod Vydiswaran Advisors: Prof.ChengXiang Zhai, Prof.Dan Roth University of Illinois at Urbana-Champaign Scalable Entailed Relation Recognizer Expanded Lexical Retrieval Entailment Recognition Lookup pieces of evidence only on relevance Traditional search Lookup pieces of evidence supporting and opposing the claim Evidence search KDD 2011 Doctoral Session 1 Hypothesis (Claim) Relation Traditional two-layer fact-finder models Credibility assessment woes Incorporating text in trust models Three directions of research [KDD 2011] • Building trust models over pieces of evidence • Content-driven trust propagation framework (KDD 2011) • Utilizes similarity and trustworthiness of evidence to measure trustworthiness of sources and claims. • Scoring claims based on community knowledge • Find treatment relations in health message boards and forums • Verify if the perception formed from reading forums correlates with validity of treatments (as approved by FDA) • Squashing rumors with evidence search • Find evidence for claims from a large text collection (ACL 2009) • Find contrasting evidence (ongoing) [Yin, et al., 2007; Pasternack & Roth, 2010] Evidence Claims Sources Claim 1 Claim 1 Claim 2 Claim 2 Advantages over traditional models . . . … • Incorporates semantics in trust computation using evidence. • Claims need not be structured tuples – they can be free-text sentences. • Framework does not assume that accurate Information Extraction is available. • A source can have different trust profile for different claims – not all claims from a source get equal weight. Claim n Claim n • Even reputed sources make mistakes • Some claims (and sources) are purposefully misleading • Not all claims made by a source is equally trustworthy • Often, contradictory claims are both supported by credible evidence • How to verify free-text claims? Web sources Evidence passages Claim sentences Credibility of news sources Trustworthiness of news stories Veracity of news reporting 3 2 Community knowledge to validate claims Conclusion and future research steps Contrastive evidence retrieval [KDD-DMH 2011] • Need to determine the truth value of a claim. • Many information types available to gauge trustworthiness • Source credibility and the power of information network • Evidence trustworthiness • Signals from community knowledge • Contrastive viewpoints for claims • Biases of users accessing the information • The goal is to recognize credible information by combining these features • Next step is to understand how human biasesinteract with credibility of information they access Claim DB Claim DB Claim DB Claim DB 3 1 2 Extract relevant claims and evidence Match up claims to evidence Rate sites based on matching claims and their support Text Corpus Indexes [Initial work at ACL 2009] Evidence & Support DB Contact details My research is supported partially by the Multimodal Information Access and Synthesis (MIAS) Center at the University of Illinois at Urbana-Champaign, part of CCICADA, a DHS Science and Technology Center of Excellence, and grants from the Army Research Laboratory. vgvinodv@illinois.edu, czhai@cs.uiuc.edu, danr@illinois.edu Acknowledgments