Sendmail::M4::Mail8 - Stop fake MX and most spammers, sendmail M4 hack file
Version 0.31 (Beta)
Now running at mail.celmorlauren.com our own mail server, and has been doing so since 0.23
SPAM consitutes the bulk of e-mail on the internet, many methods exist to fight this scurge, some better than others. However we think that this module is the simplest, quickest and most efective, relying as it does on the basic power of sendmail macros for most of its methods.
As all systems have an IP address and most have some sought of domain-name, it is possible to base the protection on wether the IP ties up with whom they claim to be at the <helo> stage. You can set sendmail to be picky about this. But many peoples IP address does not resolve to what they would like, its easy to setup domain to IP via people like network solutions, but the other way round needs a friendy ISP. So as this is a common problem, base the protection on the helo resolving to their IP.
Next check that their domain does not contain their IP encoded somehow, people who are not real MXs tend to have numeric user addresses, this has tuning to control how strict this is.
Keep a record of whom the system has sent mail to, so that we have a chance of spotting spammers using fake bounces to fill up a users email box, at the most paranoid this refuses all bounces. That causes some problems with some systems who use fake bounces to check wether you are an MX, some even come from a completly different domain to the one being talked to at the time!??! Stupid or what?
Next check that the From address is not pretending to be one of your own hosted domains, ie the IP is external and is not known to you as an outside user.
After that noraml sendmail DB files will do the rest, use the cookbook, all you need to know is there.
Sendmail::M4::Utils does most of the work for this module, all this does is format the rules and supply a default name for the hack. Various tuning methods exist, and of course you can add your own rules to this.
This module is non OO, and exports the methods descriped under EXPORTS.
Ian McNulty, celmorlauren limited (registered in England & Wales 5418604).
email <development@celmorlauren.com>
Versions
Nov 2006 1st version, pure sendmail M4 hack, using plug-in Perl programs.
25 August, this 1st CPAN M4 hack script, original script split into, this Mail8 and Utils for creation and testing.
21 Sept 2007 Released onto CPAN.
Amendments to release version
Most changes are recorded in Utils as this really was a messy hack conversion, with many revisions required to get something that worked.
Sept 2007, added this HISTORY
mail8 DataBase file refrences (apart from mail4) have been removed, and will now only added if the {Perl_Helpers} have been installed, currently these scripts are not available, their future will be decided later.
mail8_zombie now included in distro, install it to get the full benifits
22 September 2007 CPAN Release
September 2007 sendmail works fine in command line test mode. But complains of "too many long names" when this is installed and run, "sendmail" assigns on the "fly" macro names!????!!!!?.
A limit of 96 (8 Bit limits?) is really too constrictive.
1st try now using {MashTemp} instead of {MashStack}, saves some, so sendmail falls over a little later!
OPTION NOMASH used where possible, also "OPTION MASH 1" to conserve name space in other places, this and changes to Utils have removed 20 names
23 September 2007 CPAN Release
Sept 2007, on SuSE 10.1 (remastered) sendmail version 8.13.6, complained of an unbalanced >, not the fault of the sender but a macro? why did sendmail then decide to 553???, had no problem with the same code on SuSE 9.3 sendmail 8.13.3?
So a single line added and hopefully all will be well, and this system can finally protect celmorlauren, we suffer quite a lot of spam, so we should be able to spot any other weakness in the code.
Sept 2007, it appears we have caused a problem with the changes above, HELO's are failing when they should not. added failed hosts to test data, to try and ensure it does not happen again.
24 September 2007 CPAN Release
Sept 2007, Patch required, spammer whose domain matched who they said they were, was not put through the numeric zombie check as expected, a Fix broke the intended logic, test data added to mail8.
24 September 2007 CPAN Patch Release
Sept 2007, HELO checking for {daemon_addr} added, should have there already (whoops sorry), however numeric addresses will not resolve anyway, so CPAN patch release delayed untill more serious problems encountered. Develelpment on test_cgi will cause further amendments, so will wait for that before release.
Sept 2007, ref to "network associates" changed to "network solutions", just been amending my "domains", so spotted the error, as I tend to look for them using google. Other typo's will be corrected when found.
Noted rather more spam than normal has been arriving at my mail system, all fake MX's have been stopped as expected, but a few from Open Relays get through this (the 1st level filter), only to fail at the 2nd level email system. In an attempt to allow the 2nd level email system to sleep more, and only have to deal with real e-mail.
Header From: checked is the same as f or at least contains f. Failures are {Refused}, no exceptions all users are checked!
Header Recieved: check now sets {Refused} for failures. {GoodRelay} hosts are exempted from this check.
Found during testing, that mail only worked once only! corrections made to ensure essential macros are set up again, and others use sendmail macros such as f instead of supplied values, which may be the from address 1st time round, 2nd and more times it is (conneted as HOST & IP)?!
30 September 2007 CPAN Patch Release
Sept 2007, noted that the Pause upload server falls foul of the just uploaded system, so the mail4.db will now permit those listed to skip all checks, however local users will not be allowed to skip mail FROM checks.
Oct 2007, sendmail needs more free "long names" for its own use, this had been using only 21 "long names", Sendmail::M4::Utils has been modified to provide routines that can use just one "long name" {MashFound}, so this will be recoded as required as the {macros} used before can no longer be direcltly accessed. Total saving minus 5, leaving 16.
Also found 2 other names, that did not need persistance, these now MashTempC & D
02 October 2007 CPAN Patch Release
Oct 2007 noticed that header line "From:" where email address not included in < > brackets, will not match when should.
03 October 2007 CPAN Patch Release
Oct 2007, the Received: header checking does not work, and currently nothing tried makes it work, fine in test mode. But as headers are not tokenised with sendmail 8.13, can not do anthing usefull with the supplied tools.
So currently does nothing, just returns.
Fix must wait for Sendmail::M4::mail8_daemon and its worker module Sendmail::M4::Mail8_daemon.pm, which will now do more than originally designed to do.
Patch required to remove useless error messages from the logs.
Oct 2007, noted that dots and dashs did not work in From: header checker for names, fixed. However underscores will still not work
05 October 2007 CPAN Patch Release
Sendmail::M4::Utils the module created to make testing this easier
This configures this module, and is allways required first.
Expected/Allowed values allways as a (hash value pairing), see Sendmail::M4::Utils for hash=>value pairings it expects, the list bellow are either default values or additional for use by this.
Sendmail::M4::Utils
file SCALAR with default value of "mail8-stop-fake-mx.m4", build SCALAR with default value of 1 install SCALAR with default value of 1 test SCALAR with default value of 1 paranoid SCALAR see heading below for values
not paranoid at all, has local users and is content to accept bounces and "callback verify" sudo bounces.
Standard sendmail rules and databases will handle user and bounce requests. This just verifys that the sending host appears to be legimate. Assuming that the hit rate on the system is not too great, use sendmail "milters" as well to take care of "spam" and "viruses"
slightly paranoid, has local users and is content to accept "callback verify" sudo bounces, but will refuse any bounce request that really is a bounce, that is a bounce with data.
mildly paranoid, is a relay host with no local users, will say OK to all "callback verify" requests that refer to hosted domains, regardless of wether the user exits or not! Refuses all real bounces.
paranoid, is a hassled relay host, will just say OK to any "callback verify" request, regardless of wether it relays for that domain or not! Refuses all real bounces.
fairly paranoid, is a really hasseled relay host, and has no time for any type of bounce, all refused. Most e-mail and even more bounces are bogus.
copyright message to list at the start of the hack, anything supplied will replace the first two lines below.
Copyright (c) 2007 celmorlauren Limited England Author: Ian McNulty <development\@celmorlauren.com>
this should live in /usr/share/sendmail/hack/
some settings that are advised FEATURE(`access_db', `hash -T<TMPF> -o /etc/mail/access.db') FEATURE(`greet_pause', `2000') define(`confPRIVACY_FLAGS', `goaway')
This is really a reminder to use VERSIONID with your own value, or just use this to use the default
VERSIONID "ANTI SPAM & FAKE MX"
Required statement, this inserts required statements into the hack file.
This inserts required statements before and after LOCAL_CONFIG, you may add more statements that belong here.
Main items "-" added to confOPERATORS KRlookup for DNS check on HELO host name H*: $>+ScreenHeader to check received headers KMath arith to join the IP address together into a single token KCleanFrom regex to enable checking of Header line "From:"
KZombie program /etc/mail/mail8/mail8_zombie this is included in the script regardless of wether it is installed or not. Included as part of the distro, install it to get the full benifits.
This enables the use of the additional Perl scripts to identify and block bogus e-mail hosts, especialy when the site is being bombed by an abusive system.
None of the scripts are currently available on CPAN, and there is no current intention of releasing them at this time, this is mostly due to the extra system setup required, such as interfaces to the iptables firewall script bring used!
If you would like to use these, contact celmorlauren for help.
These are configured automatically by the above PerlHelpers, and are only useable with PerlHelpers, if the above has not been defined then this returns without doing anything.
However there is one exception to this, mail4 is always required as there is no other way of allowing "broken mail systems" that you want to accept mail from!
To keep sendmail error messages down (/var/log/mail) ensure you create all the required database's by hand, mail4 at the very least!
For manual creation of files, use vi ###; makemap hash ###.db <### where ### is the database source.
/etc/mail/mail8/mail9.db ip (address port 25) to TarPit in firewall rules /etc/mail/mail8/mail8.db refuse connect to SPAMMER access.db also does this and more /etc/mail/mail8/mail4.db allow, OK this host would fail tests /etc/mail/mail8/mail3.db single shot, allow, like mail4 /etc/mail/mail8/mail2.db relays hosted domains $=R, $=w, & ${VirtHost} also does this /etc/mail/mail8/mail1.db relays internal hosts by IP 192.168.#.# assummed local 172.16.#.# assummed local 10.#.#.# assummed local
NOTE This files are all optional, so this can be specified even if none of these exist.
The single useable argument if SCALAR will place the DataBases in /var/tmp/mail8, which enables you to test with alternate files to the running version.
This inserts required statements before and after LOCAL_RULESETS, you may add more statements that belong here.
Main items D{Paranoid}"%setup{paranoid}" paranoid level set above D{mail8yhabr}"YOU HAVE ALREADY BEEN REFUSED!" D{mail8ctboood}"SPAMMER CLAIMED TO BE ONE OF OUR DOMAINS!" D{mail3tt}"ONLY MAIL TO SUPPLIED Trouble Ticket ACCEPTED"
HELO DOMAIN NAME CHECKING
Most SPAMMERS use ZOMBIE PC's to send their spam, most if not all have completly numeric DNS names.
Most have their IP address in simple dotted or dashed notation, often all 4 parts of their IP address, we will not let any who have 2 or more parts of their address as their name through.
We are considering a tuning element to vary this, maybe to just one, however a lot of real senders with several servers use the last part of the IP address in their name.
Some unhelpfull ISP's string the IP's together as a single number.
Some very unhelpfull ISP's encode the numbers in HEX.
And finally totally random strings of numbers and letters, which have led us in the past to completly block the entire domain in the standard access.db file, this is often the best thing to do with hard to otherwise stop SPAMMER domains.
As much as possible of the code is INLINED to reduce the "named rulesets" total, as inlined code must be defined before use, macros are in reverse order.
[Pad,Hex,PadHex]Number convert single digits to what the name sugests.
(Pad,Hex,PadHex)IpNumber
convert four digit IP address to what the name sugests.
ScreenMash
The worker, matchs supplied patten with $s (HELLO).
However Hello must be clearly tokenised.
Splice used by ScreenIP
There must be a better way to do this, however as far as decimal numerics goes this works, nothing todate (lots of time spent trying) works for HEX.
celmorlauren will continue to use the original Perl helpers for now.
ScreenIpPatten used by above, trys patten dotted then dashed
ScreenIP
used by above, trims IP address from 4 then 3 then 2, also trys re-arranging and all 4 parts spliced together as a single token
ScreenDomainIP small often used macro, re-arranges IP to check
CONTACT
This bit arrived at on first contact, and so permissions based on IP can be set
Local_check_relay standard rule, to check incoming connection against mail8 databases and of course standard local ip addresses, further rules are based on what happens here.
uses Macro Screem_bad_relay (GLOBAL B) to do the main checking
{GoodRelay} and {BadRelay} both contain result of check, such values as (where # is checked value).
#.Local.FOUND $w {GoodRelay} #.VirtHost.FOUND ${VirtHost} {GoodRelay} #.RelayDomain.FOUND $R {GoodRelay} #.mail1.FOUND mail1.db {GoodRelay} #.Private.FOUND 192.168.#.# 172.16.#.# 10.#.#.# {GoodRelay} #.mail4.FOUND #.mail3.FOUND
mail8 amd mail9 checks result im $#error
Being found does not mean that the host is a BadRelay, just that it will need handling differently to other hosts. Hosts recorded as being GoodRelay.
HELO & FROM
After intial HELO and every FROM following
This insists that the HELO host name must either be the same as the {client_name} or resolve to an address that is the same as the {client_name}.
This also handles empty FROM's which are normally bounces of some kind, or the un-helpfull callback verify sudo bounce, which often originates from poorly configured e-mail systems that blindly bounce back to Forged FROM addresses.
{Bounce} records that a empty FROM has been recieved, these are accepted according to the value of {Paranoid}.
{Refused} and {RefusedAgain} record that the connection has been refused, only spammers will cause {RefusedAgain} to be generated, also if the Perl Helpers are installed these will attempt to ammend both sendmail databases and the firewall rules.
Refers to
this is called regardless of wether the PerlHelpers have been installed.
this checks the HELO host for being highly numeric, and having its IP encoded in the name.
RCPT
standard sendmail rules do most of the work, however depending on the value {Paranoid} responses will vary in responce to {Bounce} requests.
DATA
This bit is for checking for callback verify requests which are a fake bounce with no DATA, action depends on setting of {Paranoid} and {Bounce}
For all values of {Paranoid} other than 0, this will not accept any bounces from any unknown systems.
{Bounce} will not be defined if mail is from permitted systems.
HEADER LINES
All this does at the moment is check the following header statements
Received:
some spammers pass other tests but show themselves by pretending to send from one of our domains!
From:
ALL users must conform, they must not FAKE their "From:" header to show something other than the FROM used in the mail discussion with the mail host. No exceptions! A considerable amount of SPAM comes from mailers that allow their users to send SPAM. It is in your own intrest to stop your users sending SPAM as you are very likly to be registered as a SPAMMER and be blocked by mail servers the world over.
Other tests are possible. But on balance we think that these should be left to a 2nd level e-mail system that can take a closer look with both anti virus and something like SpamAssassin, it should be noted that these systems tend to be rather slow, so should never be run on a busy front line e-mail system under constant attack. If you wish additional rules may be supplied, these will be tacked on the end of the SScreenHeader definition.
7 POD Errors
The following errors were encountered while parsing the POD:
Expected text after =item, not a number
You forgot a '=back' before '=head2'
To install Sendmail::M4::Utils, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Sendmail::M4::Utils
CPAN shell
perl -MCPAN -e shell install Sendmail::M4::Utils
For more information on module installation, please visit the detailed CPAN module installation guide.