[BioPython] O|B|F mail update -- making progress on anti-spam issues with our mailing lists

[Chris Dagdigian]

Hi folks,

Apologies for the cross-posting but I just wanted to give our list 
members and admins an update on some new anti-spam measures we have 
(re)enabled. Good news to report basically...

The most annoying spams recently have been the simple plain text 
messages without any HTML, attachments or mime-encoding that just slip 
right by our filters.  Some lists have been forced to switch over to 
"only members can post" while other lists (like bioperl) have 
consistantly voted to stay as open as possible.

I'll update you on our current efforts as well as a new effort that is 
about 24 hours old but already working really well so far.

Until yesterday we had three main lines of defense against spam:

1. The mailserver itself (rejects mail from nonexistant domains, etc.)

2. The sendmail Mail::Milter extention (MIMEDefang+SpamAssassin are used 
to scan all incoming messages. Anything that scores higher than 8.0 is 
simply discarded automatically. MIMEDefang also strips dangerous 
attachments like .exe and .pif)

3. Our mailing list moderation queue (emails with attachments, odd MIME 
encodings and spamassassin scores from 0.0 - 7.9 are held in a moderator 
queue for a human to make an accept/discard decision)

Here are some stats on how this system worked over the past few days:

  o 138 attempts to relay mail through our server blocked
  o 192 emails blocked due to forged or unresolvable sender domain
  o 577 emails discarded automatically by SpamAssassin+MIMEDefang

This system worked *ok* but put a lot of work onto the shoulders of our 
list admins who constantly had to weed out the spam caught up in the 
mailing list moderator system.

Yesterday I brought online another system that seems to be already 
working really well. It catches spam before we even accept it on our 
server which makes the load easier on both our scanning software and our 
  human list moderators.

The system is the RBL+ blackhole list from http://www.mail-abuse.org and 
the way it works is that we now query (via DNS) the RBL+ database each 
time someone connects to our mail server. If the RBL check against the 
sender IP address comes back as "positive" we reject the incoming email.

RBL+ is a combination of four constantly updated databases:

  1. RBL -- IP addresses of known, documented spammers and spam machines
  2. RSS -- IP addresses of documented/tested unsecured email relays
  3. OPS -- IP addresses of documented open proxy servers w/ spam history
  3. DUL -- IP addresses belonging to ISP dialup and DHCP customers

We have already blocked 137 email attempts in the last 24 hours from 
machines that were listed in one or more of the RBL databases.

It is too soon to tell but if the RBL+ system plus our existing 
anti-spam measures work well enough we may be in a position where our 
"closed" mailing lists could revert back to being 'anyone can post'.

Feedback appreciated. Especially if you get a "reject" message from us 
saying that you are listed in the RBL+ blackhole database!


Regards,
Chris
O|B|F