Over the span of the last few years, we have seen email, a once useful means of communication, transform into a hideous monster that appears to have outlived it’s usefulness. We humans have a tendency to quickly get over something good that has happened to us, while even a small affliction can continue to torment us for a long period of time. As an example, take Spam; email’s worst enemy and one of the biggest scourges of the Internet age.
At one point, I used to be so overwhelmed by spam, that I must have spent hours every day just dealing with it and tweaking the spam filters. All I ever wanted was to find a way to rid the world of the evildoers that caused so much pain and suffering to humanity.
There was an article on Wired a couple of months back on how Google thinks that Spammers are giving up. Though that may not really be the case, I’m tempted to believe this theory. Spam had stopped being an issue for me a couple of months ago and I hadn’t even realized it. However, just like in Gmail’s case, this is probably due to improved spam filtering than spammers mending their ways. In any case, after my last Linux mail system upgrade, the amount of spam appearing in my mailboxes has virtually dropped to zero without causing any false positives. It is as though it has become a firewall for spam.
Now on to the technical stuff. I’ve stopped using bogofilter since I haven’t kept up with the project and my configuration was getting quite inefficient at filtering spam. Exim is a wonderful mail transfer agent and has recently added native support for a number of techniques and filters so it was about time I started fresh. I won’t list down the whole configuration here, but these are the simple techniques I’m using that have seemingly worked wonders:
1. HELO header check
This is a simple Exim ACL that drops incoming mails that include my server’s IP address in the mail header or any value that begins with an IP address.
Number of mails rejected using this method: about 30 per day
2. Sender verification
A large amount of spam is generated with the “From” address set to some random email address which may or may not exist. This technique verifies if this address actually exists and filters emails when it doesn’t. I setup a small utility for email address verification on my network utilities site a few months back.
Number of mails rejected using this method: about 40 per day
3. DSBLs check
This must be one of the most effective checks. I’m using the zen.spamhaus.org blacklist to see if the sending party’s IP has been blacklisted due to spamming. If it is, the mail gets dropped.
Number of mails rejected using this method: about 500 per day
4. ClamAV » Open Source Anti-Virus
In addition to its ability to find and filter email viruses, ClamAV has support for detecting spam using known checksums (via spam signatures from Sane Security). However, viruses aren’t as common these days as they were some years back and ClamAV usually just sits there resting.
Spamassassin runs as a separate daemon that gives a score to each email, based on various tests. This score indicates the likeliness of the email being spam or ham and can then be used to decide if it should be dropped, delivered or stored in the junk folder. It is a bit tough to configure, but once up and running, it works like a charm.
6. Block risky attachments
Infected attachments are also not very common these days, but I still block all exe, pif, bat, scr, lnk and com attachments.
7. Block character sets
Ever get that useless foreign language spam that you can’t read a word of? Though not recommended, you can filter mails based on the language character set of an email. This is quite helpful if you suddenly start getting lots of spam from a specific region.
So, the conclusion is that there is still hope for mankind. We are no longer at the mercy of the pill-wielding, loan-peddling, scum of the earth and it isn’t impossible to stand against them. The tide appears to be turning and with so many advances in mail filtering, I doubt the spammers can afford to keep up the fight for long. Not in the email arena, at least.
It seems spam filters and technologies have gotten a bit smart.
There was a time when I processing around 2.5 million messages a day on 5 mail servers with spam rate of around 40%. Then came DSBL with there own problems. It was effective but tier one isp like AOL , Att and bell were never interested in getting themselves off the DSBL list. We started losing customers hence DSBL was a delicate issue but it worked wonders!
Lastly came Postini (which is now gmail) excellent solution. Was a bit expensive but had excellent engine for spam. You would change MX to their servers and then the will do a smart routing to your mail server. They would even do load balancing or fail over to multiple SMTP servers.
Bottom line is things have improved. No mor ViaaAg** , Rolex and lotteries emails.
Sender verification is tricky. If you have a lot of incoming mail, it can bog down your server. Secondly, you run the risk of being blocked by the mail service provider due to too many SMTP probes.
For 4, my experience was wonderful. Just have the signature updated automatically and it blocks quite a lot of spam messages. It’s a lot faster than SA due to less checks.
I use a combination of DSBL, spamassassin,and FMX on my mail server. Since I implemented FMX I have had very little to no spam.