User Whitelist Spam Filter Overview

Sambar Server Documentation

User Whitelist Spam Filter
Pro Server Only

User Whitelist Spam Filter
Important! This functionalit is currently under development and will be introduced in the 5.2 release cycle.

There are two types of filtering: positive and negative. Negative filtering can provide an effective "first pass" at eliminating SPAM, but is not as powerful positive filtering. Below is an outline of each technique and where/when each technique is most appropriate.

Negative Filters

Negative filters are used to detect spam and then to delete it or move it to a folder for later deletion. Common negative filters include rules for blocking mail with "free" or "cash" in the subject line and rules to block mail from known spammers. Maintaining a "blacklist" containing e-mail addresses, domains, and profiles of message headers and message body text of percieved spam messages requires a huge number of filters and a great deal of CPU time processing for incoming message. The other problem with this approach is that spammer's intrusion techniques are evolving as fast as the prevention techniques are, so the battle is never ending. The chance of "false positives" is also significantly higher with this approach.

When negative filtering is implemented, a third-party solution such as BrightMail or the MAPS Realtime Blackhole List is typically used. These organizations typically have a large number of individuals dedicated to maintaining their lists.

For most systems, negative filters are not a practical solution for fighting spam. On the other hand, negative filters are effective at blocking known virus attacks. It is very common for mail system administrators to block all email with attachments that are typically used to carry viruses: .vbs, .exe .bat, .com, .pif, .shs, .scr, .vbe, .ocx. See the SPAM filtering overview for how to enable the SMTP "negative" filters.

Positive Filters

Positive filtering is simple and powerful -- you filter out the email addresses you want to keep, and send everything else into a folder for mass deletion. Basically, you construct and maintain a whitelist and then apply the filter to email when the sender address matches anything in the list. The whitelist can be "seeded" with the email addresses found in the user's existing mailbox and address book, and can be automatically supplemented with recipients of outgoing mail messages sent by the user.

Mail that does not match the whitelist goes into a 'spam' folder. Users can periodically scan this folder and if any worthwhile messages appear in there they can add the address to their positive filter list. The rest gets deleted. The positive filter method is occasionally defeated by spammers who forge addresses that belong to regular correspondents (see limits to filtering. below).
The whitelist-centric strategy can be taken one step further with a policy of "deny everything that is not explicitly allowed". Under this model, messages from unknown senders are held in a pending queue until the sender responds with a confirmation to their original message and are deemed legimate. This method has the advantage of being very selective about what it allows in, while at the same time permitting legitimate, but previously unknown senders to reach you. The downsides of this strategy is that it places an additional burden on senders, many senders find such messages offensive, and finally, this method is susceptible to more sophisticated address forgery.

Performance Requirements

The performance requirements of a positive filtering implementation can be fairly significant. A typical user might have a whitelist that is comprised of several hundred correspondents (after using the system for a prolonged period). The CPU cycles needed to filter each incoming mail message against a user's whitelist can be considerable resulting in additional mail server hardware. Yahoo currently limits users to 100 entries in their list of addresses to block; this facility, though a form of negative filtering, operates similarly to the whitelist filtering proposed here and seems adequate for "casual" e-mail users. Any system planning to introduce a whitelist solution would need to accomodate at least 100 entries to be effective; 300 or more entries would be desirable to meet the needs of more active users.

The Limits of Filtering

Regretably, the release of the Klez virus in early 2002 has harkened the advent of more sophisticated address forgery. Spammers routinely send mail where the sender's address is from someone in the recipient's domain. Custom address forgery is the next step in spam evolution. In this way spam and internet worms/viruses are showing convergent evolution, both using the same technical and social engineering techniques to bypass filters and security systems and to then be activated by a human being (spam seeks to be read, worms to be activated). The whitelist solution outlined in this document do not block these more sophisticated techniques.