How Opinions are Received by Online Communities - Case study on Amazon helpfulness votes

1. How Opinions are Received by Online Communities- Casestudy on Amazon helpfulnessvotes Cristian Danescu-Niculescu-Mizil1, Gueorgi Kossinets2, Jon Kleinberg1, Lillian Lee1 1Dept. of Computer Science, Cornell University, 2Google Inc. WWW 2009 Emin Sadiyev Cmpe 493

2. Amazon.com layout Average star rating Helpfulness ratio

3. Outline • Users’ evaluation on online reviews: Helpfulness votes • Makehypothesis • Proving their validity • Coming up with a mathematical model thatexplains these behaviors

4. Introduction Opinion What did Y think of X?

5. Introduction Meta-Opinion What did Z think of Y’s opinion of X?

6. The Helpfulness of Reviews • Widely-used web sites include not just reviews, but also evaluations of the helpfulness of the reviews • The helpfulness vote • “Was this review helpful to you?” • Helpfulness ratio: • “a out of b people found the review itself helpful”

7. Flow of Presentation

8. Flow of Presentation

9. Hypotheses: Social Mechanisms • Well-studied hypotheses for how social effects influence group’s reaction to an opinion • The conformity hypothesis • The individual-bias hypothesis • The brilliant-but-cruel hypothesis • The quality-only straw-man hypothesis

10. Hypotheses • The conformity hypothesis • Review is evaluated as more helpful when its star rating is closer to the consensus star rating • Helpfulness ratio will be the highest of which reviews have star rating equal to overall average • The individual-bias hypothesis • When a user considers a review, he or she will rate it more highly if it expresses an opinion that he or she agrees with

11. Hypotheses (contd.) • The brilliant-but-cruel hypothesis • Negative reviewers are perceived as more intelligent, competent, and expert than positive reviewers • The Quality-only straw-man hypothesis • Helpfulness is being evaluated purely based on the textual content of reviews • Non-textual factors are simply correlates of textual quality

12. Flow of Presentation

13. Hypotheses

14. Absolute Deviation from Average • Consistent with conformity hypothesis • Strong inverse correlation between the median helpfulness ratio and the absolute deviation • Reviews with star rating close to the average gets higher helpfulness ratio

15. Hypotheses

16. Signed Deviation from Average • Not consistent with brilliant-but-cruel hypothesis • There is tendency towards positivity • Black lines should not be sloped that way if it is valid hypothesis

17. Hypotheses

18. Addressing Individual-bias Effects • It is hard to distinguish between the conformity and the individual-bias hypothesis • We need to examine cases in which individual people’s opinions do not come from exactly the same distribution • Cases in which there is high variance in star ratings • Otherwise conformity and individual-bias are indistinguishable • Everyone has same opinion

19. Variance of Star Rating and Helpfulness Ratio Helpfulness ratio is the highest when star ratings of reviews have average value Helpfulness ratio is the highest with reviews of which rating is slightly-above the average Two-humped camel plots: local minimum around average

20. Hypotheses

21. Quality-only hypothesis • Possible other methods • Human annotation • Could be subjective • Classification using machine learning methods • We cannot guarantee the accuracies of algorithms • Plagiarized reviews • Almost(not exact) same text • same text could be considered as spam reviews • Different non-textual information • If the quality-only straw man hypothesis holds, helpfulness ratios of documents in each pair should be the same

22. Plagiarism • Making use of plagiarism is effective way to control for the effect of review text • Definition of plagiarized pair(s) of reviews • Two or more reviews of different products • With near-complete textual overlap • Author takes %70 textual overlap as plagiarism

23. An Example

24. Experiments with Plagiarism • Text quality is not the only explanatory factor • Statistically significant difference between the helpfulness ratios of plagiarized pairs The plagiarized reviews with deviation 1 is significantly more helpful than those with deviation 1.5

25. Hypotheses

26. Flow of Presentation

27. Authors’ Model • Based on individual bias and mixtures of distributions • Two distributions: one for positive, one for negative evaluators • Balance between positive and negative evaluators: • Controversy level: • Density function of helpfulness ratios of positive evaluators • Gaussian distribution of which average is -centered • Density function of helpfulness ratios of negative evaluators • Gaussian distribution of which average is -centered

28. Validity of the Model • Empirical observation and model generated

29. Conclusion • A review’s perceived helpfulness depends not just on its content, but also the relation of its score to other scores • The dependence of the score is consistent with a simple and natural model of individual-bias in the presence of a mixture of opinion distributions • Directions for further research • Variations in the effect can be used to form hypotheses about differences in the collective behaviors of the underlying populations • It would be interesting to consider social feedback mechanisms that might be capable of modifying the effects authors observed here • Considering possible outcomes of design problem for systems enabling the expression and dissemination of opinions

30. Discussions • So, how can we use this? • In which cases would this information be helpful? • Available information is very limited • Star ratings • Helpfulness ratios • Conclusion is rather trivial • Does not present new discoveries