ARSA: A Sentiment-Aware Model for Predicting Sales Performance Using Blogs

1. ARSA: A Sentiment-Aware Model for Predicting SalesPerformance Using Blogs Yang Liu, Xiangji Huang, Aijun An and Xiaohui Yu Department of Computer Science and Engineering York University, Toronto, Canada School of Information Technology York University, Toronto, Canada SIGIR 2007

2. Introduction • What the general public thinks of a product can no doubt influence how good it sells • Blogs can be commentaries or discussions on a particular subject • Ranging from mainstream topics to highly personal interests • This paper studies the predictive power of opinions and sentiments expressed in blogs • Focus on the blogs that contain reviews on products • Blogs serve as a very good indicator of the product’s future sales performance

3. Introduction(Con.) • Developing models and algorithms that can • mine opinions and sentiments from blogs • use them for predicting product sales • Investigate how to predict box office revenues of movies using the sentiment information obtained from blog mentions

4. Why Using Movies? • data availability • daily box office revenue data are all published on the Web (IMDB) and readily available • expect the models and algorithms to be easily adapted to handle other types of products that are subject to online discussions • books, music CDs and electronics

5. Introduction(Con.) • S-PLSA Model • sentiment mining based on Probabilistic Latent Semantic Analysis • Appraisal words are exploited to compose the feature vectors for blogs • ARSA Model (Autoregressive Sentiment Aware ) • AR Model • Count in past sale performance • Combining with sentiment information mined from the blogs

6. Related Work • Sentiment mining • focuses on determining the semantic orientations of documents • machine learning approaches • evaluate the semantic distance from a word to good/bad with WordNet • Blog mining • make use of links or URLs in Blogspace • analyzing the contents of blogs

7. Characteristics of Online Discussion • based on the number of blog mentions • very difficult to make a successful prediction of sales ranks • two movies both released on May 19, 2006 • The Da Vinci Code • Over the Hedge • use the name of each movie as a query to blog search engine • fixed time stamp • starting from one week before the movie release till three weeks after the release • use the number of returned results for a particular date as a rough estimate of the number of blog mentions published on that day

8. Example 1has higher # of blog mentions 票房差不多, 甚至2有時超越1

9. Box Office Data and User Rating • collect the average user ratings of the two movies from the IMDB website • The Da Vinci Code – 6.5 • Over the Hedge - 7.1 • the number of blog mentions may not be an accurate indicator of a product’s sales performance • people’s opinions (as reflected by the user ratings) seem to be a good indicator of how the box office performance evolves

10. S-PLSA：aProbabilisticApproach to Sentiment Mining • Feature Selection • Traditional way • Compute the (relative) frequencies of various words in a given blog • Use the resulting multidimensional feature vector as the representation of the blog • focus on the set containing 2030 appraisal words extracted from the lexicon constructed by Whitelaw et al. • use their frequencies in a blog as a feature vector

11. Sentiment PLSA • sentiments are often multifaceted • differ from one another in a variety of ways • just classify the sentiments expressed in a blog as either positive or negative • too simplistic • a blog can be considered as being generated under the influence of a number of hidden sentiment factors • each hidden factor focusing on one specific aspect of the sentiments • accommodate the intricate nature of sentiments • model sentiments and opinions as a mixture of hidden factors and use PLSA for sentiment mining

12. S-PLSA : Formerly Presentation • a set of blog entries B = {b1, . . . , bN} • a set of words (appraisal words) from a vocabulary W = {w1, . . . ,wM} • blog data can be described as a N × M matrix D =(c(bi,wj))ij • c(bi,wj) is the number of times wi appears in blog entry bj • each row in D is then a frequency vector that corresponds to a blog entry

13. S-PLSA (Con.) • consider the blog entries as being generated from a number of hidden sentiment factors Z = {z1, . . . , zK} • correspond to blogger’s complex sentiments expressed in the blog review • Generative model

14. S-PLSA (Con.) • result • Assuming blog entry b and the word w are conditionally independent given the hidden sentiment factor z • Estimate model parameters • P(z), P(b|z), P(w|z)

15. S-PLSA (Con.) • maximize the following likelihood function: • EM algorithm

16. S-PLSA (Con.) • P(z|b) represents how much a hidden sentiment factor z “contributes” to the blog document b • the set of probabilities {P(z|b)|z Z} can be considered as a succinct summarization of b in terms of sentiments

17. ARSA : a Sentiment-Aware Model • Capture two different factors that can affect the box office revenue of the current day • box office revenue of the preceding days • autoregressive model (AR) • people’s sentiments about the movie

18. The autoregressive model • denote the box office revenue of the movie of interest at day t by xt • t = 1, . . . ,N • basic AR process of order p • φ1, φ2, . . . , φp : parameters of the model • : error term • Once this model is learned from training data • at day t, the box office revenue xt can be predicted by xt−1, xt−2,. . ., xt−p

19. New AR Model • AR models are only appropriate for time series that are stationary • 1st step • 2nd step (remove seasonality) • New AR model

20. Incorporating Sentiments • Bt denote the set of blogs on the movie of interest that were posted on day t • average probability of sentiment factor z = j conditional on blogs in Bt • ωt,j represents the average fraction of the sentiment “mass” thatcan be attributed to the hidden sentiment factor j

21. ARSA Model • Autoregressive Sentiment-Aware model • p, q, and K : user-chosen parameters • q : the sentiment information from how many preceding days are considered • k : the number of hidden sentiment factors used by S-PLSA to represent the sentiment information • and : parameters whose values are to be estimated using the training data

22. Training the ARSA Model • learning the set of parameters φi(i = 1, . . . , p), and ρi,j(i = 1, . . . , q; j = 1, . . . ,K), from the training data that consist of the true box office revenues • For a particular movie m(m = 1, . . . ,M) • M : total number of movies in the training data • ， • minimize

23. Empirical Study • Experiment settings • a set of blog documents on movies of interest collected from the Web • from May 1, 2006 to August 8, 2006. • timestamp ranging from one week before the release to four weeks after • the amount of blog entries collected for each movie ranges from 663 (for Waist Deep) to 2069 ( for Little Man) • 45046 blog entries comment on 30 different movies • corresponding daily box office revenue data for these movies • manually collected from IMDB

24. Experiment • choose half of the movies for training, and the other half for testing • train an S-PLSA model • For each blog entry b, the sentiments towards a movie are summarized using a vector of the posterior probabilities of the hidden sentiment factors, P(z|b) • apply ARSA model • obtain estimates of the parameters • evaluate the prediction performance of the ARSA model by experimenting it with the testing data set.

25. MAPE • use the mean absolute percentage error (MAPE) to measure the prediction accuracy • n : total amount of predictions made on the testing data • Predi : predicted value • Truei : true value of the box office revenue

26. Parameter Selection • K、p、q Lower dim. Vector Loss sentiment info. 前一天 post blog之 sentiment info. 和 prediction最相關 Factor in all influence on preceding day’s performance Overfitting High cost Get irrelevant information 12.1%

27. Result on Particular Movie • similar to observation of Figure 2 • close to true values using the optimal parameter settings

28. Comparison with Alternative Methods Pure AR model AR model utilizes the volume of blog mentions vt −i: number of blog mentions on day t−i φi and ρi: parameters to be learned

29. Comparison with other Feature Selection Method • feature vectors are computed using the (relative) frequencies of all the words appearing in the blog entries • large set、cost high • only select words with higher frequencies (excluding stop words) • same # of words as ARSA for fairness

30. Conclusions and Future Work • predicting sales performance using sentiment information mined form blogs • using movies as a case study • proposal of S-PLSA • generative model for sentiment analysis • “summarizing” sentiment information from blogs • ARSA • model for predicting sales performance based on • sentiment information • product’s past sales performance • Future Work • Clustering and classification of blogs based on their sentiments • use S-PLSA as a tool to help track and monitor the changes and trends in sentiments expressed online