Spam to this list

[John E. Malmberg]

Martin Pool wrote:
> John Van Essen wrote: 
> 
> The policy is to block as much spam as possible without blocking
> legitimate posts.  A 100% solution is impossible, even if we had human
> moderation (humans make mistakes).

I am seeing reports on news.admin.net-abuse.email from Steve Linford 
that he is getting at least 99% accuracy in removing spam with zero loss 
of real e-mail.

He is removing about 85% of the spam with DNSbls so that it does not 
even get inside of the mail server, and then using SpamAasssin 3.0 with 
it's new test on URLs inside of mail, where if the URL resolves to an IP 
address that is known to be controlled by a spammer, the e-mail is rejected.

And he is reporting that he is not using a DHCP list for doing rejections.

> 
>> The first one has been in the dul.dnsbl.sorbs.net blacklist since Oct.
>> I use these 4 DNS-based blacklists in the mail server that I manage:
>>
>>   sbl-xbl.spamhaus.org

I have not ever seen a report of an incorrect listing in the 
xbl.spamhaus.org.  I have only seen one reported error in several years 
of the sbl.spamhaus.org and it was corrected with in 1/2 hour of this 
being pointed out on news.admin.net-abuse.email.

It is a merging of 3 dnsbls for convenience.

    sbl.spamhaus.org - Hand maintained list of I.P. addresses controlled
                       by spammers.

   The sbl.spamhaus.org is probably now the most widely used dnsbl in
   the world.  An ISP has to work hard at supporting spam to get any
   of it's IP addresses listed in the sbl.spamhaus.org.

   xbl.spamhaus.org is a combination of opm.blitz.org and
   cbl.abuseat.org.

   The cbl.abuseat.org runs spamtraps that filter out auto-responders.
   In the time it has been in existence, I have seen zero reports of
   an incorrect listing.  It will delist on request once per week, and
   listings age off.

   The opm.blitz.org verifies that the I.P. address is an open proxy,
   and ages off old listings.

>>   list.dsbl.org

This is a list of known compromised I.P. addresses where no responsible 
party has demonstrated they have an RFC compliant mailbox set reading 
abuse complaints.  If a real mail server is listed, it means that it is 
either an active compromised machine, or that their is no one that is 
reading messages to their abuse or postmaster e-mail addresses.

It is extremely widely used to reject e-mail, possibly the most used 
after the spamhaus.org.

>>   dul.dnsbl.sorbs.net

In the past, the dul.dnsbl.sorbs.net used to run a higher false positive 
rate.  Now it is almost not measurable.

dul.dnsbl.sorbs.net now allows owners of mistaken static entries to use 
a webform to remove them as long as they can show a forward DNS name 
pointing to that I.P. with a long enough TTL to show it is static.

Currently a listing in dul.dnsbl.sorbs.net indicates well over a 99% 
chance of spam.

>>   web.dnsbl.sorbs.net

I have heard nothing good or bad about that one.  In the spam I sent 
through spamcop.net in the past year, I recall seeing it only flag one 
spam that was not detected by either the cbl.abuseat.org or njabl as 
being in that DNSBL.

 From what I have seen, the only zone in sorbs that is likely to cause 
real e-mail to be rejected is the spam.dnsbl.sorbs.net as it is usually 
listing multi-hop exploits of the mail servers of major ISP's and they 
have to jump through hoops to get off of it.  The other SORBS zones do 
not require such extra actions.

>> And they have helped a LOT.

>> The other 3 have no reverse DNS entries.  A machine with no reverse DNS
>> that is sending email is not very likely to be a legitimate email server.
>> It's much more likely a compromised machine on a clueless ISP's network.
>> Rejecting email from those unidentified machines also has helped a lot.
> 
> 
> Using any of those measures alone tends to block legitimate posters,

Can you find a legitimate post that was blocked by the 
sbl-xbl.spamhaus.org?  I have not heard of an error on that list yet.

 From the reports that I have seen on the various e-mail forums, reverse 
DNS is now an RFC requirement for operating any server on the public 
internet.  Networks with no rDNS are demonstrating that they do not 
understand how to be properly connected to the internet and have proven 
to be a large source of problems.  The fastest way to get that problem 
fixed is to take AOL's approach and refuse all e-mail with no rDNS on it 
at all.

> particularly those running their own mail server, which to my mind is a
> greater harm than letting ocassional spam go through.  Our purpose here
> is to run a mailing list, not punish ISPs.  So we use all the things you
> named as part of a weighted score.

Actually what is a result is that you are allowing the list recipients 
to be punished by incompetent ISP's.

At some point, it is not worth attempting to try to find a potential 
real e-mail from a network that has allowed spammers to infest it by 
either neglect or by willful act.

If you can put a [SPAM?] tag on mail trapped by a the following 
algorithm, I would be surprised if any real postings made it through to
the list with the tag.

Any open relay - reject as spam.
Any open proxy (list.dsbl.org, njabl.org) - reject as spam.
Anything in the sbl-xbl.spamhaus.org reject as spam.

if (bad or missing rDNS) || (dul.dnsbl.sorbs.net) || (bl.spamcop.net) || 
(multihop type lists) {
   [SpamAssasin 3.0] - If a URL in the message resolves to an
   I.P. address listed in the sbl-xbl.spamhaus.org

   reject as spam.
}

if (bad or missing rDNS) && (bl.spamcop.net) reject as spam.

This is a conservative algorithm using what appears to be state of the 
art in open source spam detection algorithms, and has been available 
since at least last august.

It does not detect Advance Fee / Nigerian 419 scams / college diploma 
scams, most of which are not making it through the current spam defenses.


bl.spamcop.net is a strong indicator of spam, but will on occasion list 
real mail servers when users are not watching for spam parsing errors.
It would probably affect less than 4 percent of real list posters, and 
then usually for less than 24 hours, but that is probably still too high 
for absolute rejections, but good enough to trigger additional tests.

If you tagged on spamcop detection alone, I would expect about 3 to 4 
localized bursts of false positives a year.

The dul.dnsbl.sorbs.net will have occasional errors in it from ISP's 
renumbering it, and there are still some people who are trying to run a 
mail server on a DHCP allocated address that apparently do not need to 
send e-mail to most of the mail servers.

 From all the mail servers that I get e-mail on, the SAMBA lists are now 
the only ones that are still accepting e-mail from known dhcp addresses.

If you tag on dul.dnsbl.sorbs.net, because of the world wide nature of 
the samba lists, I would expect only a few postings to get tagged per year.

If you apply the more "aggressive" tests only to e-mail addresses that 
are not subscribed to the list, I would expect no real postings to get 
tagged.

-John
wb8tyw at qsl.net
Personal Opinion Only