Recsys12 && KDD 12 Brief Summary

1. Recsys12 && KDD 12 Brief Summary Xiwang Yang 1

2. the 6th ACM International Conference on Recommender Systems (RecSys 2012) • RecSys 2012 • Conference Introduction • Evaluation Metrics • Learning to Rank • Social RS • Context-aware and location-based recommendations • HCI, User-centric, interfaces & explanations • System Design

3. RecSys12 Introduction • Premier global forum for discussing the state of the art in recommender System • Long paper acceptance rate: 24/119 = 20% • Single Track • > 270 attendence • 1/3 from industry

4. Evaluation Metrics • Workshop:Recommendation Utility Evaluation: Beyond RMSE • Organizer: Xavier Amatriain (Netflix),Harald Steck (Netflix), Pablo Castells (UAM), Arjen de Vries, and Christian Posse (LikedIn) • Specific questions that the workshop aims to address include the following: • What are the unmet needs and challenges for evaluation in the RS field? What changes would we like to see? How could we speed up progress? • What relevant recommendation utility and quality dimensions should be cared for? How can they be captured and measured? • How can metrics be more clearly and/or formally related to the task, contexts and goals for which a recommender application is deployed? • How should IR metrics be applied to recommendation tasks? What aspects require adjustment or further clarification? What further methodologies should we draw from other disciplines (HCI, Machine Learning, etc.)?

5. Evaluation Metrics • Workshop:Recommendation Utility Evaluation: Beyond RMSE • Can we predict the success of a recommendation algorithm with our offline experiments? What offline metrics correlate better and under which conditions? • What are the outreach and limitations of offline evaluation? How can online and offline experiments complement each other? • What type of public datasets and benchmarks would we want to have available, and how can they be built? • How can the recommendation effect be traced on business outcomes? • How should the academic evaluation methodologies improve their relevance and usefulness for industrial settings? • How do we envision the evaluation of recommender systems in the future?

6. Evaluation Metrics • Industry Keynote: Ron Kohavi (Microsoft): Online Controlled Experiments: Introduction, Learnings, and Humbling Statistics • Ron Kohavi, General Manager Experimentation Platform, Microsoft • Controlled experiments at Microsoft Bing, very good work, 2012 kdd paper;

7. Controlled Experiments in One Slide • Concept is trivial • Randomly split traffic betweentwo (or more) versions • A (Control) • B (Treatment) • Collect metrics of interest • Analyze   • Must run statistical tests to confirm differences are not due to chance • Best scientific way to prove causality, i.e., the changes in metrics are caused by changes introduced in the treatment(s)

8. Evaluation Metrics • Session: Multi-Objective Recommendation and Human Factors • Multiple Objective Optimization in Recommendation Systems • Mario Rodriguez and others explain how they design LinkedIn recommendations by optimizing to several objectives at once (e.g. candidate that is good for the job + who is open to new opportunities). They report results from an AB Test run on LinkedIn

9. Evaluation Metrics • Session: Multi-Objective Recommendation and Human Factors • Pareto-Efficient Hybridization for Multi-Objective Recommender Systems • Marco Tulio Ribeiro-Universidade Federal de Minas Gerais & Zunnit Technologies • The problem of combining recommendation algorithms grows significantly harder when multiple objectives are considered simultaneously. • take the multi-objective a step further. In their case, they optimize the system to not only be accurate, but also present novel or diverse items.

10. Outline • Learning to Rank • Session: Top-N Recommendation • Social top-k RS: • Industry invited talk: Ralf Herbrich (Facebook): Distributed, Real-Time Bayesian Learning in Online Services

11. Learning to Rank • Focus more on ranking than rating prediction NOW! • Session: Top-N Recommendation • CLiMF: Learning to Maximize Reciprocal Rank with Collaborative Less-is-More Filtering • Best paper, Yue Shi, Delft, intern at Telefonica • Optimize Mean Reciprocal Rank (MRR)directly. • MRR is a well-known information retrieval metric for measuring the performance of top-k recommendations • Similar work: "TFMAP: Optimizing MAP for top-n context-aware recommendation • SIGIR 2012. • Optimize Top-N, Mean Average Precision • Uses tensor factorization to model implicit feedback data (e.g.,purchases, clicks) with contextual information • fast learning algorithm

12. Learning to Rank • Focus more on ranking than rating prediction NOW! • Session: Top-N Recommendation • "Ranking with Non-Random Missing Ratings: Influence of Popularity and Positivity on Evaluation Metrics • An interesting study on the very important issue of negative sampling, and popularity bias in learning to rank. The paper discusses these effects on the AUC (Area Under the Curve) measure.

13. Learning to Rank • Session: Top-N Recommendation • "Sparse Linear Methods with Side Information for Top-N Recommendations“ • University of Minnesota in the Twin Cities • multidimensional context-aware learning to rank • Alternating Least Squares for Personalized Ranking • Gravity R&D • Densemath • invited anyone not interested in Mathematics to leave the room • proposed a computationally efficient ranking based method RankALS that optimizes the original objective function, without sampling. • "On Top-k Recommendation Using Social Networks”

14. Learning to Rank • "Real-Time Top-N Recommendation in Social Streams • University of Hannover • Focus on analyzing social streams(twitter) in real-time for personalized topic recommendation and discovery. • Industry invited talk: Ralf Herbrich (Facebook): Distributed, Real-Time Bayesian Learning in Online Services • Bayesian Factor Models for large-scale distributed ranking • The same author and others from MSR named it as "Matchbox“, is now used in different settings • Poster "The Xbox Recommendation System“ • Apply matchbox to recommending movies and games for the Xbox • Poster “Collaborative Learning of Preference Rankings” • Erasmus School of Economics & MSR • apply it to sushi recommendation

15. Social RS • Session: Social Recommendation • "Spotting Trends: The Wisdom of the Few” • Wisdom of the Few, using a reduced set of experts for recommendations • Popular != Trending • iCoolhunt users are encouraged to take pictures of objects that they think ‘cool’, upload them and share them with friends online.

16. Social RS • Workshop on Recommender Systems and the Social Web • Extending FolkRank with Content Data • Leveraging Publication Metadata and Social Data into FolkRank for Scientific Publication Recommendation • Context Determines Content - An Approach to Resource Recommendation in Folksonomies • FReSET - An Evaluation Framework for Folksonomy-based Recommender Systems • Aggregating Content and Network Information to Curate Twitter User Lists • Online Dating Recommender Systems: The Split-Complex Number Approach • Social Media-Driven News Personalization • Trust-Based Local and Social Recommendation

17. Context-aware and location-based recommendations • Workshop: Personalizing the Local Mobile Experience • Workshop on Context-Aware Recommender Systems • Session: Contextual and Semantically Aware Recommendation • Context-Aware Music Recommendation Based on Latent Topic Sequential Patterns", • playlist generation • "Ads and the City: Considering Geographic Distance Goes a Long Way” • location-aware recommendations.

18. HCI, User-centric, interfaces & explanations • Tutorial: Conducting User Experiments in Recommender Systems • Bart Knijnenburg, UCI • overview of how to conduct user studies for recommender systems • Paper: TasteWeights: A Visual Interactive Hybrid Recommender System • USSB • Paper: Inspectability and Control in Social Recommenders • Bart Knijnenburg, UCI • Analyze the effect of giving more information and control to users in the context of social recommendations. • Workshop on Interfaces for Recommender System.

19. System Design • Tutorial: Building Industrial-scale Real-world Recommender Systems • Xavier Amatriain, Netflix • Mendeley Suggest: Engineering a Personalized Article Recommendation System • Kris Jack from Mendeley • he explained how they make use of AWS and Mahout in a system that can generate personalized recommendations for about $60 a month • From a toolkit of recommendation algorithms into a real business • Domonkos Tikk from Gravity R&D • evolved from being a team in the Netflix Prize to a real-world company with very interesting projects

20. RecSys in China- Booming • RecSys China • Technical Community, ~5000 members • RecSys 13 in HongKong • Baidu • RecSys team built two years ago, size: ~100 • Taobao • RecSys team: > 100 • Weibo • RecSys team: ~20 • Tencent • Huawei • Noah's Ark Research Lab in Hong Kong • Built July, 2012 • long term ~100 researchers

21. How user evaluate each other in social media • Keynote-Jure Leskovec-Stanford University

24. KDD2012 • August12-16, Beijing, China • First held in Asian • Attendance: 1232 • Long Paper Acceptance Rate: 133/755 = 17.6% • Three Research Tracks about Recommendation • Research Session : Personalization and Recommendation • Research Session: Ads and Video Recommendation • Research Session: Recommendation • Research Session : Matrices and Tensors

25. Mining Heterogeneous Information Networks- Jiawei Han - UIUC

34. Big Data Panel Discussion • debate on the following questions: • What is the nature of Big Data? What are the Big Data problems that you have encountered? Is this a long-term challenge or a short-term fad? • What opportunities and challenges does data mining face on Big Data? • What are effective Big Data solutions? What platforms, sampling solutions, and applications are most effective for handling Big Data?

35. Some of the Opinions • Christos Faloutsos – CMU • Large data size: > 100 machines, > 1 Tera Bytes • Jiawei Han • Large Complexity • Michael I. Jordan • ? Sampling everything • Some other voice • Big data rate in HFT