Understanding Credibility Across Disciplinary Boundaries

1. Understanding Credibility Across Disciplinary Boundaries Miriam J. Metzger Department of Communication University of California, Santa Barbara www.credibility@ucsb.edu

2. Social Scientific Views of Credibility • Research began with studies of persuasion during WWII • Credibility = expertise + trustworthiness • IQ includes credibility (e.g., especially expertise and accuracy) but also includes relevance, data security, and usability • Credibility focuses more on believability as a subjective perception of end user • Judgments of credibility are not necessarily rational • Heuristics, or mental shortcuts and rules-of-thumb, can be useful

3. Old View of Credibility Evaluation • Old view was analytic and effortful processing of information to determine credibility in terms of: • Accuracy: information is correct and free of errors • Authority: author or source of information is highly regarded • Objectivity: information is unbiased and impartial • Currency: information is sufficiently up-to-date for user’s task • Coverage: information is of sufficient depth and breadth for user’s task • Evaluations may take place at many levels (e.g., site, source, message)

4. site features information features • Professional, attractive page design • Easy navigation, well organized site • Absence of errors and broken links • Certifications, recommendations, or seals from trusted third parties • Interactive features • Paid access to information • Ranking in search engine • Domain name suffix • Absence of advertising • Sponsorship by or links to reputable organizations • Presence of privacy and security policies • Presence of date stamp showing information is current • Citations (especially to scientific data or references), links to external authorities • Message relevance, tailoring • Professional-quality and clear writing • Message accuracy, bias, plausibility • Information breadth and depth • Description of editorial review process or board Web credibility user features author features • Past experience with source • Internet experience & reliance • Age • Prior knowledge and attitudes • Motivation/goal for search task • Author identification • Author qualifications and credentials • Author contact information • Absence of commercial motive • Reputation, name recognition

5. New View of Credibility Evaluation • Credibility decisions are rarely based on thorough evaluation of five criteria • Use of heuristics is key except in cases of high motivation to process information (dual processing models of credibility, Metzger 2007) • Evaluations dictated by bounded rationality (Simon, 1957) rather than complete rationality

6. What heuristics do people use to evaluate the credibility of Web information? • Liking/agreement heuristic: People believe information if trusted others do • Consensus/bandwagon heuristic: People believe information if a lot of others do • Familiarity/recognition heuristic: People believe information from recognized sources • Similarity heuristic: People believe information if it agrees with information from independent sources • Self-confirming heuristic: People believe information that agrees with their preexisting opinions • Expectancy violation heuristic: People believe information that conforms to their expectations

7. Can heuristics inform credibility system design in useful ways? • How can we incorporate perception and emotion in algorithmic models of credibility evaluation? • How can we design systems to both capitalize on and improve upon heuristic processes of credibility evaluation? Can we help people use or apply heuristics more wisely in their credibility evaluations by augmenting with more considered information? • How can we design systems that are themselves heuristic in the sense of being easy to use and produce results that enable quick but meaningful credibility evaluations?

8. Some ideas for future research • Mine social graphs for similarity and trust information and linkages, but need to develop more meaningful trust indicators

9. Some ideas for future research • Mine social graphs for similarity and trust information and linkages, but need to develop more meaningful trust indicators • Mine and aggregate ratings, reviews, testimonials; display opinion distributions; develop tools to help users interpret ratings information appropriately • Need new ways of identifying and verifying expertise and authority; further develop and use reputation systems in credibility research • Automate cross-validation; create authoritative databases; consider human- computational systems and games • Use natural language processing and machine learning techniques to automatically discover expectancy violations for a variety of online information

10. Thank you! For more information about our work, please go towww.credibility@ucsb.edu