1 / 47

Comment Spam Identification

Comment Spam Identification. Eric Cheng & Eric Steinlauf. What is comment spam?. Total spam: 1,226,026,178 Total ham: 62,723,306. 95% are spam!. Source: http://akismet.com/stats/ Retrieved 4/22/2007. Countermeasures. 5yx.org 9kx.com aakl.com aaql.com aazl.com abcwaynet.com

walt
Download Presentation

Comment Spam Identification

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Comment Spam Identification Eric Cheng & Eric Steinlauf

  2. What is comment spam?

  3. Total spam: 1,226,026,178 Total ham: 62,723,306 95% are spam! Source: http://akismet.com/stats/ Retrieved 4/22/2007

  4. Countermeasures

  5. 5yx.org 9kx.com aakl.com aaql.com aazl.com abcwaynet.com abgv.com abjg.com ablazeglass.com abseilextreme.net actionbenevole.com acvt.com adbx.com adhouseaz.com advantechmicro.com aeur.com aeza.com agentcom.com ailh.org akbu.com alaskafibre.com alkm.com alqx.com alumcasting-eng-inc.co! americanasb.com amwayau.com amwaynz.com amwaysa.com amysudderjoy.com anfb.com anlusa.net aobr.com aoeb.com apoctech.com apqf.com areagent.com artstonehalloweencostumes.com globalplasticscrap.com gowest-veritas.com greenlightgo.org hadjimitsis.com healthcarefx.com herctrade.com hobbyhighway.com hominginc.com hongkongdivas.com hpspyacademy.com hzlr.com idlemindsonline.com internetmarketingserve.com jesh.org jfcp.com jfss.com jittersjapan.com jkjf.com jkmrw.com jknr.com jksp.com jkys.com jtjk.com justfareed.com justyourbag.com kimsanghee.org kiosksusa.com knivesnstuff.com knoxvillevideo.com ksj! kwscuolashop.com lancashiremcs.com lnjk.com localmediaaccess.com lrgww.com marketing-in-china.com Blacklisting rockymountainair.org rstechresources.com samsung-integer.com sandiegonhs.org screwpile.org scvend.org sell-in-china.com sensationalwraps.com sevierdesign.com starbikeshop.com struthersinc.com swarangeet.com thecorporategroup.net thehawleyco.com thehumancrystal.com thinkaids.org thisandthatgiftshop.net thomsungroup.com ti0.org timeby.net tradewindswf.com tradingb2c.com turkeycogroup.net vassagospalace.com vyoung.net web-toggery.com webedgewars.com webshoponsalead.com webtoggery.com willman-paris.com worldwidegoans.com

  6. Captchas • "Completely Automated Public Turing test to tell Computers and Humans Apart"

  7. Other ad-hoc/weak methods • Authentication / registration • Comment throttling • Disallowing links in comments • Moderation

  8. Our Approach – Naïve Bayes • Statistical • Adaptive • Automatic • Scalable and extensible • Works well for spam e-mail

  9. Naïve Bayes

  10. P(A|B) ∙ P(B) = P(AB) = P(B|A) ∙ P(A)

  11. P(A|B) ∙ P(B) = P(B|A) ∙ P(A)

  12. P(A|B) = P(B|A) ∙ P(A) / P(B)

  13. P(spam|comment) = P(comment|spam) ∙ P(spam) / P(comment)

  14. P(spam|comment) = P(comment|spam) ∙ P(spam) / P(comment)

  15. P(spam|comment) = P(w1|spam) ∙ P(w2|spam) ∙ … P(wn|spam) ∙ P(spam) / P(comment) Probability of w1 occurring given a spam comment (naïve assumption)

  16. Corpus Incoming Comment Texas casino Online Texas hold’em Texas gambling site P(Texas|spam) = 1 – (1 – 2/5)3 = 0.784 P(w1|spam) = 1 – (1 – x/y)n Probability of w1 occurring given a spam comment where x is the number of times w1 appears in all spam messages,y is the total number of words in all spam messages, andn is the length of the given comment

  17. P(spam|comment) = P(w1|spam) ∙ P(w2|spam) ∙ … P(wn|spam) ∙ P(spam) / P(comment) Probability of w1 occurring given a spam comment

  18. P(spam|comment) = P(w1|spam) ∙ P(w2|spam) ∙ … P(wn|spam) ∙ P(spam) / P(comment) Probability of w1 occurring given a spam comment Probability of something being spam

  19. P(spam|comment) = P(w1|spam) ∙ P(w2|spam) ∙ … P(wn|spam) ∙ P(spam) / P(comment) Probability of w1 occurring given a spam comment Probability of something being spam ??????

  20. P(ham|comment) = P(w1|ham) ∙ P(w2|ham) ∙ … P(wn|ham) ∙ P(ham) / P(comment) P(spam|comment) = P(w1|spam) ∙ P(w2|spam) ∙ … P(wn|spam) ∙ P(spam) / P(comment) Probability of w1 occurring given a spam comment Probability of something being spam ??????

  21. P(ham|comment)  P(w1|ham) ∙ P(w2|ham) ∙ … P(wn|ham) ∙ P(ham) P(spam|comment)  P(w1|spam) ∙ P(w2|spam) ∙ … P(wn|spam) ∙ P(spam) Probability of w1 occurring given a spam comment Probability of something being spam

  22. P(ham|comment)  P(w1|ham) ∙ P(w2|ham) ∙ … P(wn|ham) ∙ P(ham)) log( ) log( P(spam|comment)  P(w1|spam) ∙ P(w2|spam) ∙ … P(wn|spam) ∙ P(spam)) log( ) log(

  23. log(P(ham|comment)) log(P(w1|ham))+ log(P(w2|ham))+ … log(P(wn|ham))+log(P(ham)) log(P(spam|comment)) log(P(w1|spam))+ log(P(w2|spam))+ … log(P(wn|spam))+log(P(spam))

  24. Fact: P(spam|comment) = 1 – P(ham|comment) Abuse of notation: P(s) = P(spam|comment) P(h) = P(ham|comment)

  25. P(s) = 1 – P(h) m = log(P(s)) – log(P(h)) = log(P(s)/P(h)) em = elog(P(s)/P(h)) = P(s)/P(h) em ∙ P(h) = P(s)

  26. P(s) = 1 – P(h) em ∙ P(h) = P(s) em ∙P(h) = 1 – P(h) (em + 1) ∙ P(h) = 1 P(h) = 1/(em+1) P(s) = 1 – P(h) m = log(P(s)) – log(P(h))

  27. P(h) = 1/(em+1) P(s) = 1 – P(h) m = log(P(s)) – log(P(h))

  28. P(ham|comment) = 1/(em+1) P(spam|comment) = 1 – P(ham|comment) m = log(P(spam|comment)) – log(P(ham|comment))

  29. In practice, just compare log(P(ham|comment)) log(P(spam|comment))

  30. Implementation

  31. Corpus • A collection of 50 blog pages with 1024 comments • Manually tagged as spam/non-spam • 67% are spam • Provided by the Informatics Institute at University of Amsterdam Blocking Blog Spam with Language Model Disagreement, G. Mishne, D. Carmel, and R. Lempel. In: AIRWeb '05 - First International Workshop on Adversarial Information Retrieval on the Web, at the 14th International World Wide Web Conference (WWW2005), 2005.

  32. Most popular spam words

  33. “Clean” words

  34. Implementation • Corpus parsing and processing • Naïve Bayes algorithm • Randomly select 70% for training, 30% for testing • Stand-alone web service • Written entirely in Python

  35. It’s showtime!

  36. Configurations • Separator used to tokenize comment • Inclusion of words from header • Classify based only on most significant words • Double count non-spam comments • Include article body as non-spam example • Boosting

  37. Minimum Error Configuration • Separator: [^a-z<>]+ • Header: Both • Significant words: All • Double count: No • Include body: No • Boosting: No

  38. Varying Configuration Parameters

  39. Varying Configuration Parameters

  40. Boosting • Naïve Bayes is applied repeatedly to the data. • Produces Weighted Majority Model bayesModels = empty list weights = vector(1) for i in 1 to M: model = naiveBayes(examples, weights) error = computeError(model, examples) weights = adjustWeights(examples, weights, error) bayesModels[i] = [model, error] if error==0: break

  41. Boosting

  42. Future work(or what we did not do)

  43. Data Processing • Follow links in comment and include words in target web page • More sophisticated tokenization and URL handling (handling $100,000...) • Word stemming

  44. Features • Ability to incorporate incoming comments into corpus • Ability to mark comment as spam/non-spam • Assign more weight on page content • Adjust probability table based on page content, providing content-sensitive filtering

  45. Comments? No spam, please.

More Related