how a major isp built a new anti abuse platform l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
How a major ISP built a new anti-abuse platform PowerPoint Presentation
Download Presentation
How a major ISP built a new anti-abuse platform

Loading in 2 Seconds...

play fullscreen
1 / 11

How a major ISP built a new anti-abuse platform - PowerPoint PPT Presentation


  • 265 Views
  • Uploaded on

How a major ISP built a new anti-abuse platform Mike O’Reirdan Comcast Distinguished Engineer Internet Systems Engineering Comcast National Engineering & Technical Operations Outline Comcast facts and figures Why build a new platform Fundamentals of anti spam Size of the problem

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'How a major ISP built a new anti-abuse platform' - daniel_millan


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
how a major isp built a new anti abuse platform

How a major ISP built a new anti-abuse platform

Mike O’Reirdan

Comcast Distinguished Engineer

Internet Systems Engineering

Comcast National Engineering & Technical Operations

outline
Outline
  • Comcast facts and figures
  • Why build a new platform
  • Fundamentals of anti spam
  • Size of the problem
  • Previous approach
  • Current solution
  • Migration methods
  • Current status
why a new platform
Why a new platform?
  • Moved from a hosted to an in-house platform
  • Need to improve customer experience by further reducing volumes of spam to the mailbox
  • Deploy a platform which can economically and easily scale
  • Emerging threats in abuse landscape
    • Image spam
    • Botnets
    • VoIP spam (SPIT)
  • Need to have a plug-and-play architecture
    • Firmly believe that no one vendor will be the best forever
    • We need a mix of vendors and approaches to hedge our bets and reduce risk
    • Somebody in this room may be our next vendor when you have gone from the lab to the VC and into beta 
size of the problem
Size of the problem
  • Volumes of spam are astronomical
    • 596 million connection attempts (Jan25th 2008)
    • 539 million connection attempts rejected
    • 93% spam
    • 76 million messages delivered
  • Connection attempts increases massively above this around holidays such as Thanksgiving.
  • The problems is criminality at massive scale
fundamentals of anti spam
Fundamentals of anti-spam
  • Not much differentiation between major mail box hosters and other ISPs with regard to spam percentages and volumes
  • Three stages
    • Blocking based on IP (reputation and DUL space)
      • 5% of CPU cycles
      • Removes ~70% of the spam
    • Blocking based on message protocol and heuristics
      • 10% of CPU cycles
      • Removes ~15% of the spam
    • Blocking based on content
      • 85% of CPU cycles
      • Remove ~10% of the spam
  • Idea is to use the least cycles to remove the most messages
previous approach
Previous approach
  • 100s of Linux blade servers
    • No site fail over
  • Multiple RBLs using BIND for DNS
  • Heuristics and protocol filtering
  • Spam content filtering using industry standard software
  • Virus filtering using industry standard software
new approach
New Approach
  • Fewer Linux Blade servers distributed over two sites
    • Full dual site redundancy with each site fully capable of carrying 100% of traffic
  • RBLs hosted on a specialised DNS based platform
    • Trend
    • Spamhaus
    • Return Path
  • Protocol and heuristics filtering performed on the Bizanga IMP MTAs which run on Linux
  • Spam content filtering technology
  • Anti-virus technology
heuristics employed
Heuristics employed
  • Directory Harvest attack
  • Dictionary attack
  • rDNS check
  • Throttling
  • Dynamic space blocking
  • Non-existent user block
content filtering detecting spammy content
Content filtering-detecting spammy content
  • Cloudmark
    • Relies on multiple sources of data
      • Spam / no Spam reports from end users
      • Honeypots
    • Initially based on Vipul’s Razor
    • Applies algorithmically derived signatures to incoming email (Proprietary)
    • Zero hour anti virus
  • Trend Anti-virus
    • Signature analysis
    • Heuristics
migration
Migration
  • Relatively simple process to migrate from old platform
  • Moved traffic across by re-pointing comcast.net MX records to new platform and making lots of involved highly planned DNS configuration changes
  • Performed a series of increasing short duration burst test scale
  • Then moved 5% of the traffic. After platform rules proved stable, traffic was moved across in slightly larger increments over several days to the new platform.
  • This method allowed us to quickly revert back (under 30 minutes) to old platform in the event of any issues without customer impact
lessons learned
Lessons learned
  • It always helps to be able to test the new platform against an existing live e-mail flow but this is difficult at our scale with a multi-Gbps mail flow
  • Failing that, heavy reliance has to be placed on cooperation with vendors and existing platform technology users
  • Rules used on an old platform do not always map across neatly to a new one