Newsweeder learning to filter netnews
Download
1 / 36

NewsWeeder: Learning to Filter Netnews - PowerPoint PPT Presentation


  • 65 Views
  • Uploaded on

NewsWeeder: Learning to Filter Netnews. By: Ken Lang Presented by Salah Omer. Introduction Theoretical Framework Approach Analysis & Evaluation. Introduction:.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' NewsWeeder: Learning to Filter Netnews' - yannis


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Newsweeder learning to filter netnews
NewsWeeder: Learning to Filter Netnews

By: Ken Lang

Presented by Salah Omer


  • Introduction

  • Theoretical Framework

  • Approach

  • Analysis & Evaluation


Introduction
Introduction:

  • Newsweeder is a Netnews filtering system that addresses the classification problem by letting the user rates his or her interest level for each article being read.

  • The goal of an information filtering system is to sort through large volumes of information and present to the user those documents that are likely to satisfy his or her information requirement.


Introduction1
Introduction:

Several learning techniques are used in information filtering

  • Rule-based user-driven mode.

  • Explicit-learning mode.

  • Implicit-learning mode.


Introduction2
Introduction:

  • Rule-based user-driven mode:

    This mode requires direct and explicit user input in establishing and maintaining the user model. SIFT (Yan, & Garcia-Molina, 1995), Lens (Malone et al, 1987), and Infoscope (Fischer, & Steven, 1991) are some examples of systems that use this approach. Some research systems have extended by making complex rule based key word matching profile. However, average users will not be interested to follow these complex systems.


Introduction3
Introduction:

  • Explicit-learning mode: This mode creates and updates the user model by eliciting explicit user feedback. Ratings on presented items, provided by users based on one or more ordinal or qualitative scales, are often employed to directly indicate users’ preference.


Introduction4
Introduction:

  • Implicit-learning mode: This mode aims to acquire users’ preferences with minimal or no additional effort from them. Kass, & Finin (1988) defined implicit mode as “observing the behavior of the user and inferring facts about the user from the observed behavior”.



Theoretical framework td idf
Theoretical FrameworkTD-IDF

  • Use the Information Retrieval techniques of term frequency inverse document frequency tf-idf.

  • The technique as we learn in the class is based on two empirical observations.


Theoretical framework tf idf
Theoretical FrameworkTF-IDF

  • First, the more times a token appears in a document i.e. term frequency (TF), the more likely the term is relevant to the document.

  • Second, the more time the term is observe the throughout the documents in the documents’ set, the less it discriminates between documents (IDF).


Theoretical framework minimum description length
Theoretical FrameworkMinimum Description Length

  • MDL is based on the following insight:

    • Any regularity in the data can be used to compress the data, i.e. to describe it using fewer symbols than the number of symbols needed to describe the data literally.

    • The more we are able to compress the data, the more we have learned about the data.


Theoretical framework minimum description length1
Theoretical FrameworkMinimum Description Length

  • MDL provide the frame work for balancing the tradeoff between model complexity and training error.

  • In the NewsWeeder domain, the tradeoff involves the importance of each token and how to decide which one to drop and which one to keep.


Theoretical framework minimum description length2
Theoretical FrameworkMinimum Description Length

  • MDL is based on the Bayes’s Rules

    P(H\D) = P(D\H)P(H)

    P(D)

    In the Bayes rule in order to maximize H in P(H\D), we need to maximize P(D\H)P(H)

    Or


Theoretical framework minimum description length3
Theoretical FrameworkMinimum Description Length

P(H\D) = P(D\H)P(H)

P(D)

to minimize

- Log(P(D\H)P(H))

which is equal to

- log(P(D\H)) - log(P(H))


Theoretical framework minimum description length4
Theoretical FrameworkMinimum Description Length

MDL interpretation of

-log(P(D\H)) - log(P(H))

as per Shannon information theory is to find the hypothesis which minimizes the total encoding length.


Approach
Approach:

Representation:

  • First: Raw text is the parsed into tokens. For NewsWeeder tokens are kept to the word level.

  • Second: create vectors of the token counts for the document. Using the bag of word approach and keeping the word in their unstemmed form.


Approach1
Approach:

Learning:TF-IDF

  • NewsWeeder uses the weight derived from the multiplication of tf and idf as express in the formula

    w(t,d)=tft,dlog(|N|dft)

  • The logarithm is used to normalize large values.


Approach2
Approach:

Learning:TF-IDF

  • In the NewsWeeder the documents in each category are converted into tf-idf vector.


Approach3
Approach

Learning:TF-IDF

  • To classify a new document, first, the new document is compared to the prototype vector then given a predicted rating based on the cosine to each category.

  • Second, the result is converted from the categorization procedure to a continuous value using a linear regression.


Approach4
Approach

Learning:MDL

  • First, perform the categorization step then convert the categorization similarities into continuous rating prediction using.

    argmax{p(ci\Td, ld, Dtrain} =

    argmin{-log(p(Td\ci,ld,Dtrain)) – log(pci\ld, Dtrain))}

    Given

    d = document, Td = token vector, Dtrain = training data

    The most probable category ci for d is that which minimizes the bits needed to encode Td plus ci


Approach5
Approach

Learning:MDL

  • Second, apply the probabilistic model to compute the similarity for each training document.

  • The probability of the data in a document given it length and category is the product of the individual token probabilities.

    p(ci\Td, ld, Dtrain) = ∏p(td\ci,ld,Dtrain)


Approach6
Approach

Learning:MDL

  • Drive the probability estimate for ti,d

    Computed the number of document contains token

    ti = ∑ti,j

    jЄN

    The correlation ri,l between ti,d and Id

  • Combine the token probability distribution that is independent of the document length with the one that is dependent on the document length weighted by ri,l


Approach7
Approach

Learning:MDL

  • Based on the hypothesis that each token either has specialized distribution for a category or that it is unrelated. The MDL chooses the category specific hypothesis is the total bits saved in using the hypothesis

    ∑ -log(p(ti,d\ld)) – [- log(p(ti,d\ld, ck))]

    dЄNck

    is greater than the complexity of cost of including the extra parameters.


Learning algorithm summary for tf idf and mdl
Learning Algorithm Summary for TF IDF and MDL

For the two approaches:

  • Divide the articles into training and test (unseen) sets.

  • Parse the training articles, throwing out tokens occurring less than three times total.

  • Compute tiand ri,lfor each token.


Learning algorithm summary for tf idf and mdl1
Learning Algorithm Summary for TF IDF and MDL

  • For TF-IDF

  • throw out the M most frequent tokens over the entire training set.

  • compute the term weights, normalize the weight vector for each article, and find the average of the vectors for each rating category.

  • compute the similarity of each training document to each rating category prototype using the cosine similarity metric.


Learning algorithm summary for tf idf and mdl2
Learning Algorithm Summary for TF IDF and MDL

  • For MDL

  • Decide for each token t and category c whether to use p(t|l,c)=p(t|l), or to use a category dependent model for when t occurs in c. Then pre-compute the encoding lengths for no tokens occurring for documents in each category.

  • Compute the similarity of each training document to each rating category by taking the inverse of the number of bits needed to encode Td under the category’s probabilistic model.


Learning algorithm summary for tf idf and mdl3
Learning Algorithm Summary for TF IDF and MDL

  • For the two approaches:

  • Using the similarity measurements computed in steps 3-TD-IDF or 2-MDL on the training data, compute a linear regression from rating category similarities to continuous rating predictions.

  • Apply the model obtained in steps 6 and step 9 similarly measurement to test articles.


Result evaluation
Result Evaluation:

  • Two methods to evaluate the performance on the result:

  • The precision metrics (the ratio of relevant documents retrieved to all documents retrieved.

  • Confusion matrix (column of the matrix represents the instances in a predicted class, while each row represents the instances in an actual class) of error generated by text classifier.


Data

Table 1: Rating Labels

The “interesting” label indicates the articles is rated 2 or better.


Data

  • Table 2 summarized the data collection over one year. NewsWeeder experiment used 40 individual over 1 year. Since our model assumes a stable distribution pool, and because it has no temporal dependence, users’ interests that lasted less than the one-year period will add some amount of error to the performance.

  • User B has rated 16% of the articles as interesting (a rating of 1 or 2), it is possible for a considerably smaller percentage of interesting articles to be in the newsgroups read by User A.



Tf idf performance analysis
TF-IDF Performance Analysis:

  • Graph 1 show the effect of removing the top N most frequent words in the top 10%,; the top predicted rating articles.

  • Five trials with training/test data split of 80/20 for each trial removed 100 to 400 words the best precision 43% reached at 300 words removed.


Mdl performance analysis
MDL Performance Analysis:

  • Graph 2 shows that MDL outperform TF-IDF for both user A and B. Performance is measured in the percentage of the interesting articles found in the top 10% with highest predicted rating.

  • MDL reach a precision of 44% for User A and 59% for user B compared to TF-IDF (37% for A and 49 % for B)



ad