Naive Bayes

1. Naive Bayes Swipe

2. Bayes Theorem Given a hypothesis h and data D which bears on the hypothesis: P (h | D) P(h) P (h | D) = P(D) P(h): independent probability of h: prior probability P(D): independent probability of D P(D|h): conditional probability of D given h: likelihood P(h|D): conditional probability of h given D: posterior probability

3. Bayesian Classifier The classification problem may be formalized using a-posterior probabilities. P(C|X) = prob. that the sample tuple X= is of class C. E.g. P(class=N | outlook= sunny, windy=true,…) Idea: assign to sample X the class label C such that P(C|X) is maximal

4. Naive Bayes Naïve Bayes is a basic learning method that uses Bayes rule with an characteristics are conditionally independent, given the class. assumption that the It assumptions are frequently broken in practice. should be noted that independence yet nonetheless. naive Bayes gives good classifications

5. Uses of Naive Bayes classification Text Classification Spam Filtering Hybrid Recommender System Recommender learning and data mining techniques for filtering unseen information and can predict whether a user would like a given resource. Systems apply machine Online Application Simple Emotion Modeling

6. Why text classification? Learning which articles are of interest Classify web pages by topic Information extraction Internet filters

7. Examples of Text Classification CLASSES=BINARY “spam” / “not spam” CLASSES =TOPICS “finance” / “sports” / “politics” CLASSES =OPINION “like” / “hate” / “neutral” CLASSES =TOPICS “AI” / “Theory” / “Graphics” CLASSES =AUTHOR “Shakespeare” / “Marlowe” / “Ben Jonson”

8. Naive Bayes Approach Build the Vocabulary as the list of all distinct words that appear in all the documents of the training set. Remove stop words and markings The words in the vocabulary become the attributes, assuming independent of the positions of the word Each document in the training set becomes a record with frequencies for each word in the Vocabulary. Train the classifier based on the training data set, by computing the prior probabilities for each class and attributes. Evaluate the results on Test data that classification is

9. Text Classification Algorithm Tct + 1 Tct + 1 = P(t|c)= (t 1EV Tct1) + B1 t EV(Tct1 + 1) Tct – Number of particular word in particular class Tct’ – Number of total words in particular class B´ – Number of distinct words in all class

10. Topics for next Post Linear Discriminant Analysis Decision tree k-nearest neighbor algorithm Stay Tuned with