Modeling Trustworthiness of Online Content
1 / 1

KDD 2011 Doctoral Session - PowerPoint PPT Presentation

  • Uploaded on

Modeling Trustworthiness of Online Content V. G. Vinod Vydiswaran Advisors: Prof.ChengXiang Zhai, Prof.Dan Roth University of Illinois at Urbana-Champaign. Scalable Entailed Relation Recognizer. Expanded Lexical Retrieval. Entailment Recognition. Lookup pieces of evidence only on relevance.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' KDD 2011 Doctoral Session' - xaria

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Modeling Trustworthiness of Online Content

V. G. Vinod Vydiswaran

Advisors: Prof.ChengXiang Zhai, Prof.Dan Roth

University of Illinois at Urbana-Champaign

Scalable Entailed Relation Recognizer

Expanded Lexical Retrieval

Entailment Recognition

Lookup pieces of evidence only on relevance

Traditional search

Lookup pieces of evidence supporting and opposing the claim

Evidence search

KDD 2011 Doctoral Session



(Claim) Relation

Traditional two-layer fact-finder models

Credibility assessment woes

Incorporating text in trust models

Three directions of research

[KDD 2011]

  • Building trust models over pieces of evidence

    • Content-driven trust propagation framework (KDD 2011)

    • Utilizes similarity and trustworthiness of evidence to measure trustworthiness of sources and claims.

  • Scoring claims based on community knowledge

    • Find treatment relations in health message boards and forums

    • Verify if the perception formed from reading forums correlates with validity of treatments (as approved by FDA)

  • Squashing rumors with evidence search

    • Find evidence for claims from a large text collection (ACL 2009)

    • Find contrasting evidence (ongoing)

[Yin, et al., 2007; Pasternack & Roth, 2010]




Claim 1

Claim 1

Claim 2

Claim 2

Advantages over traditional models




  • Incorporates semantics in trust computation using evidence.

  • Claims need not be structured tuples – they can be free-text sentences.

  • Framework does not assume that accurate Information Extraction is available.

  • A source can have different trust profile for different claims – not all claims from a source get equal weight.

Claim n

Claim n

  • Even reputed sources make mistakes

  • Some claims (and sources) are purposefully misleading

  • Not all claims made by a source is equally trustworthy

  • Often, contradictory claims are both supported by credible evidence

  • How to verify free-text claims?

Web sources

Evidence passages

Claim sentences

Credibility of news sources

Trustworthiness of news stories

Veracity of

news reporting



Community knowledge to validate claims

Conclusion and future research steps

Contrastive evidence retrieval

[KDD-DMH 2011]

  • Need to determine the truth value of a claim.

  • Many information types available to gauge trustworthiness

    • Source credibility and the power of information network

    • Evidence trustworthiness

    • Signals from community knowledge

    • Contrastive viewpoints for claims

    • Biases of users accessing the information

  • The goal is to recognize credible information by combining these features

  • Next step is to understand how human biasesinteract with credibility of information they access

Claim DB

Claim DB

Claim DB

Claim DB




Extract relevant claims and evidence

Match up claims to evidence

Rate sites based on matching claims and their support

Text Corpus


[Initial work at ACL 2009]

Evidence & Support DB

Contact details

My research is supported partially by the Multimodal Information Access and Synthesis (MIAS) Center at the University of Illinois at Urbana-Champaign, part of CCICADA, a DHS Science and Technology Center of Excellence, and grants from the Army Research Laboratory.

[email protected], [email protected], [email protected]