Can spam
1 / 40

Can Spam - PowerPoint PPT Presentation

  • Uploaded on

A presentation by… Bodek Frak & Craig Brown. Can Spam. - OUCC 2004 -. What is Spam?. Unsolicited: You did not ask for it Commercial: Trying to sell you something (legal/illegal) Bulk: Sent in large quantities UCE: Unsolicited commercial email

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Can Spam' - josef

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Can spam

A presentation by…

Bodek Frak & Craig Brown

Can Spam

- OUCC 2004 -

What is spam

What is Spam?

  • Unsolicited:

  • You did not ask for it

  • Commercial:

  • Trying to sell you something (legal/illegal)

  • Bulk:

  • Sent in large quantities

  • UCE: Unsolicited commercial email

  • UBE: Unsolicited bulk email

What is not spam

What is NOT Spam?

  • E-mails from mailing lists that a person subscribed to and does not know how to unsubscribe.

  • E-mails from on-line shopping companies where person was shopping.

  • E-mails generated by viruses mailing themselves as attachments

  • “Chain letters” sent to you by your friends or people you barely know….

False positive negative

False Positive / Negative

  • False Positives: legitimate e-mails that get mistakenly identified as spam.

  • False Negatives: spam e-mails that slip undetected past the filters.

  • “For most users, missing legitimate e-mail is an order of magnitude worse than receiving spam, so a filter that yields false positives is like an acne cure that carries a risk of death to the patient.”


  • Not regulated as much as telemarketing or paper mail.

  • Anonymous nature of e-mail (SMTP limitations) and Internet.

  • Cheap: The sheer number of spam mail sent means that even tiny response rates, reportedly 0.0001%, means junk mailers turn a profit.

  • Number of people using e-mail is rising.


  • Bots searching web sites and newsgroup postings for anything with [email protected] sign.

  • Directory harvest attacks.

  • Some dishonest companies sell their customer data.

  • “Unsubscribe link”

  • Guessing: [email protected], [email protected], [email protected], etc.

  • Viruses and worms stealing addresses from PC’s address book.



  • Storage and bandwidth costs

  • Administrative overhead

  • E-mail delivery delays

  • E-mail server performance decreased

  • Lost employee productivity

  • Frustration (keyboards/monitors) 

The battle

The Battle

  • Legal initiatives (new laws, lawsuits)Problem: most spam comes from outside N. Am

  • More effective and widespread spam filters (spam business model no longer profitable)Problem: high cost, complexity

  • Next generation (“not-so-simple”) SMTP Problem: will take time before widely adopted

  • Better user awareness



  • Outsource: Redirect all your incoming mail through a company that filters out the spam and delivers good mail to your users

  • In-house: Hardware/software spam control solution deployed at the network level (server-side) or client level

  • Plan B: Switch to paper and pen …. 

Two extremes

Two Extremes

  • Stop spam before it reaches the user

  • All mail is suspect and scrutinized by the gateway/server for point of origin and content; spam is discarded.

  • Pros:

  • - Avoids wasting user’s time

  • - Saves internal network bandwidth and server storage, processing power

  • Cons:

  • - False positives never detected (high impact)

  • - User not in control

Two extremes1

Two Extremes

  • Let everything in

  • Point of origin and content not scrutinized but users are given tools to deal with unwanted mail

  • Pros:

  • - User makes his own spam decisions

  • - Minimal impact of false positives

  • Cons:

  • - Lost bandwidth, storage, processing power

  • - User learning curve

Spam detection

Spam Detection

  • SMTP Level Rules: - DNS Lookup (connecting host, return address) - RBL: DNS real-time blackhole lists (controversial)

  • Content Checking Rules (score):

  • - SMTP headers

  • - Content (subject line & body)

  • Whitelists/Blacklists

  • Challenge-Response

Bayesian filtering probability

Bayesian Filtering (Probability)

  • Pros:

  • Widely acknowledged to be the best way to catch spam

  • Learns all the time, and takes into account your valid e-mails (known as “ham”)

  • Takes a statistical approach and does not rely solely on static updates from the vendor

  • Cons:

  • My spam may not be the same as your spam

  • Takes time to build the spam/ham database

  • Adds complexity for the user

Two strategies

Two Strategies

  • All external e-mails are legitimate except those from blacklisted (blocked) addresses and/or domains and those that do not pass the rules (more common)

  • All external e-mails are spam except those from whitelisted (protected) addresses and/or domains (more effective)

Best practices

Best Practices

  • Never make a purchase from an unsolicited e-mail

  • If you do not know the sender of an unsolicited e-mail message, delete it.

  • Never respond to any spam messages or click on any links in the message.

  • Avoid using the preview functionality of your e-mail client software.

Best practices1

Best Practices

  • When sending e-mail messages to a large number of recipients, use the blind copy (BCC) field to conceal their e-mail addresses

  • Never provide your e-mail address on websites, newsgroup lists or other online forums.

  • Never give your primary e-mail address to anyone or any site you don’t trust.

  • Have and use one or two secondary e-mail addresses.



  • Sophos “Field Guide To Spam”


  • SpamNews

  • Inventor of Bayesian Filtering

  • Bayesian Spam Filtering

The university of windsor problem

The University of Windsor Problem

  • Currently receiving over 200,000 e-mail messages per day

  • In-house filters deleting up to 100,000 e- mails per day

  • Many more spam messages getting through

  • Our end users are frustrated!

Spam solutions

Spam Solutions

  • We looked at 3 categories of software solutions

  • - E-mail server add-on

  • - SMTP gateway products

  • - Client side products

  • Hardware based solutions

  • - Complete solution that consists of both hardware and software in one package

E mail server add on

E-Mail Server Add-On

  • A solution implemented on the e-mail server

  • Pros:

  • Works in conjunction with e-mail server

  • Vendor specific

  • Cons:

  • Vendor specific

  • - The University of Windsor would need at least two solutions, one for faculty/staff, the other for students

Smtp based gateway products

SMTP-Based Gateway Products

  • An independent solution usually in front ofe-mail servers

  • Pros:

  • Vendor independent

  • Identifies spam before it hits the e-mail servers

  • Reduces load on e-mail servers

  • Cons:

  • Usually more expensive, in some cases requires purchase of new hardware

Client side products

Client Side Products

  • Installed on the user’s PC

  • Works with e-mail client to filter/tag suspected spam when messages are being downloaded from the mail server.

  • Pros:

  • Provides individual with complete control over spam filtering

  • Cons:

  • Possible deployment / support issues

Our requirements

Our Requirements

  • Minimal involvement of IT staff in maintaining the solution

  • Significantly higher rate of catching spam when compared to our current “in-house” solution - Currently identifying between 10-30% of total e-mail volume as spam

  • More advanced set of features and options “out of the box”

  • Technical support from vendor, including upgrades/updates

  • Ability for end users to control how their spam is dealt with

  • LDAP Compliant

Essential spam detection techniques

Essential Spam Detection Techniques

  • Keyword search and proximity search in subject/body

  • Keyword search using pattern matching or heuristic analysis

  • Message format analysis

  • Statistical Analysis (Including Bayesian filtering)

  • Blacklist of known bad e-mail and IP addresses

  • Whitelist

  • Open proxy lists, DNS verification

  • Ability to filter viral attachments (not essential!)

Spam engine settings

Spam Engine Settings

  • Settings can be changed system-wide

  • Settings can be changed for groups of users

  • Settings can be changed for individual users

Actions on spam

Actions on Spam

  • Delete the message

  • Quarantined in either a per-system quarantine, or a per-user quarantine

  • Tagged within the message header

  • Tagged by adding something to the subject line

Updates to the software

Updates to the Software

  • User or administrator updates by editing rules

  • User or administrator trains spam engine through human identification of spam

  • Vendor keeps engine updated through periodic updates

Product selection

Product Selection

  • Using the criteria developed by the committee, a list of products that fit the criteria was established.

  • These products were identified through magazine articles, newsletters, other campus solutions, and previous vendor contacts

Product selection1

Product Selection

  • A number of products were eliminated from the list

  • Reasons for elimination included:

  • - Operating System (Our in-house expertise is with UNIX/Sendmail)

  • - Scalability

  • - Lack of essential features

Finally a shortlist

Finally – A Shortlist!

  • Spam Assassin

  • BrightMail

  • Can-It Pro

  • PureMessage

Decisions decisions

Decisions, Decisions …

  • The committee reviewed the short-listed products, viewed presentations, and tested two solutions

  • The solutions tested were:

  • - Can-It Pro - PureMessage

Live evaluation

Live Evaluation

  • Each product installed on a test server

  • Members of committee had e-mail directed to test server

  • Can-It Pro

  • - Very low false-positive rate

  • - Acceptable spam capture rate out of the box

  • PureMessage

  • - High spam capture rate out of the box

  • - Very low false-positive rate

The final decision

The Final Decision

  • PureMessage was chosen to be our campus wide spam solution

  • High spam capture rate

  • Low positive rate

  • Low maintenance

  • Offers all the features we required

  • Fit our budget



  • Solution will run on four Sun Sunfire V440 Servers

  • 2 servers for incoming mail, 2 servers for outgoing mail, 1 server for spooling

  • Currently waiting for server hardware to arrive

  • Current implementation target: Summer 2004

  • Support staff have been added to the current pilot to test

Default configuration

Default Configuration

  • Default configuration will be “Quarantine”

  • Other options include “Tag & Deliver” and “Opt-Out”

Default configuration1

Default Configuration

  • True spam (messages > 90%) will be deleted

  • Possible spam (scoring 50-90%) will be quarantined

  • Legitimate (messages < 50%) will be delivered

  • A large amount of messages will never reach the messaging servers

  • Daily digest will be mailed each morning, detailing messages in quarantine from the previous day

  • End users will have on-demand access to their quarantine through a web-based interface

Default configuration2

Default Configuration

  • In addition to the PureMessage AntiSpam solution, we also purchased the PureMessage Policy Manager

  • This will allow us to filter out unwanted attachments that may be viral – EXE, COM, BAT, PIF, SCR, etc.

Changes to e mail handling

Changes to E-Mail Handling

  • In addition to spam filtering, other initiatives have been proposed to curb the amount of spam on campus. These include:

  • - Only accepting mail addressed to valid addresses

  • - Only accepting mail with valid return addresses (domain must resolve in DNS)

Implementation issues

Implementation Issues

  • Some users on other non-ITS controlled servers may not be in our LDAP database

  • Possible issues with the handling of ListServs

  • Possible issues with Shared Mailboxes