Download
1 / 34
340 likes | 345 Views
NewsMe:. A Case Study for Adaptive News Systems with Open User Model. Preliminary Examination Paper 2007 Chirayu Wongchokprasitti IS PhD Student School of Information Sciences. NewsMe. NewsMe Overview. Personalized News Access System Feed the news that response to the user’s interest
E N D
NewsMe: A Case Study for Adaptive News Systems with Open User Model Preliminary Examination Paper 2007 Chirayu Wongchokprasitti IS PhD Student School of Information Sciences
NewsMe Overview • Personalized News Access System • Feed the news that response to the user’s interest • 82 RSS news feeds, 21 sources • 8 News Topics • Ranking the news • Open User Model based system
NewsMe Interface • 4 News Sections • Recent News • Recommended News • My Profile • News History
User Feedback Method • Add a news item to Tracked News • Add a news item to Blacklist
User Model Manipulation • Update rating of news in user model
User Model Manipulation (Con’t) • List all history of viewed news • Update rating of news in user model
Learning User Models for News Access • The system uses a machine learning approach to build a simple model of each user’s interests. • A similarity-based method achieves the balance of learning and adapts quickly to change interests while avoiding brittleness.
Learning User Models for News Access(cont.) • The purpose of the user model • First, it should contain information about recently read events, so that stories which belong to the same thread can be identified. • To allow for identification of news that user already knows. • The k-nearest-neighbor algorithm (kNN) is used to achieve the desired functionality. • Convert news contents to tf-idf vectors (term-frequency/inverse-document-frequency). • Use the cosine similarity measure to quantify the similarity of two vectors.
Learning User Models for News Access(cont.) • Decay Function • Freshness of news content is our issue. • Freshness should decay exponentially day by day. • Freshness of news remains a half after fed 7 days. • is the initial freshness of news content. • is a decay instance, which its value is around 0.099.
Study Design • 20 Participants • Assign to be Information Analysts • 2 News Topic: US and Business • 2 Sessions, 3 stages per session • Retrieved News: Nov 28th – Dec 12th, 2006 • Google Notebook extension (http://www.google.com/notebook)
Implicit VS Explicit Feedback • Implicit feedback • Assuming every news user read is a tracked news • Explicit feedback • Users add news items to their user model • Tracked news as Positive sample • Blacklist News as Negative sample
Hypotheses • Performance hypotheses are: • H1: The open model system with user profile manipulation by users performs better than the open model system without them, • H1.1: The open model system with explicit feedback generates results with better performance, and, • H1.2: Users with explicit feedback system demonstrate higher task performance.
Hypotheses (Con’t) • User Perspective hypotheses are: • H2: Users prefer the user profile manipulation features in the open model system, • H2.1: Users appreciate better in the system with explicit feedback, and, • H2.2: Users appreciate the ability to control their profiles.
Preliminary Results • The Ground Truth • System Performance Analysis • User Performance Analysis • User Feedback Analysis
The Ground Truth • F-measure defines as follows: • Summary of news items in the study
User Feedback Analysis • A two-way ANOVA was performed on the questionnaire data to examine significant differences in user answers by system and by stage. • On the question 3, subjects indicated they trusted in system’s ability to find useful information for the US topic versus the Business topic in overall (p-value = 0.017). • On the question 7, subjects indicated My Profile helps them to understand how the system finds useful news items for the US topic versus the Business topic in overall (p-value = 0.013).
Future Work • Open Model with explicit feedback did not outperform the baseline. • The experiment indicates that without caution, user model manipulation not only benefit the performance but lower the output. • Binary rating might not be a suitable way. • Fuzzy rating is a good way to study further.
