NAME

TipJar::MTA - outgoing SMTP with exponential random backoff.

SYNOPSIS

  use TipJar::MTA '/var/spool/MTA';     # must be a writable -d
                                        # defaults to ./MTAdir
  $TipJar::MTA::interval='100';         # the default is 17
  $TipJar::MTA::TimeStampFrequency='35';        # the default is 200
  $TipJar::MTA::AgeBeforeDeferralReport=7000;   # default is 4 hours
  $TipJar::MTA::MyDomain='peanut.af.mil';       # defaults to `hostname`
                                        # And away we go,
  TipJar::MTA::run();                   # logging to /var/spool/MTA/log/current
  

DESCRIPTION

On startup, we identify the base directory and make sure we can write to it, check for and create a few subdirectories, check if there is an MTA already running and stop if there is, so that TipJar::MTA can be restarted from cron.

We are not concerned with either listening on port 25 or with local delivery. This module implements outgoing SMTP with exponentially deferred random backoffs on temporary failure. Future delivery scheduling is determined by what directory a message appears in. File age, according to stat(), is used to determine repeated deferral.

Every $interval seconds, we fork a child process.

A new child process first goes through all new outbound messages and expands them into individual messages and tries to send them. New messages are to be formatted with the return address on the first line, then recipient addresses on subsequent lines, then a blank line (rather, a line with no @ sign), then the body of the message. The TipJar::MTA::queue module will help compose such files if needed.

Messages are rewritten into multiple messages when they are for multiple recipients, and then attempted in the order that the recipients appeared in the file.

After attempting new messages, a child process attempts all messages in the "immediate" directory.

After attempting all messages in the immediate directory, a child process moves deferred messages whose times have arrived into the immediate directory for processing by later children.

Deferred messages are stored in directories named according to when a message is to be reattempted. Reattempt times are assigned at requeueing time to be now plus between three and five quarters of the message age. Messages more than a week old are not reattempted. An undeliverable message that got the maximum deferrment after getting attempted just shy of the one-week deadline could conceivably be attempted for the final time fifteen and three quarters days after it was originally enqueued. Then it would be deleted.

The format for new messages is as follows:

return address

The first line of the message contains the return address. It can be bare or contained in angle-brackets. If there are angle brackets, the part of the line not in them is discarded.

recipient list

All recipients are listed each on their own line. Recipients must have at-signs in them.

blank line

The first line (after the first line) that does not contain a @ symbol marks the end of the recipients. We are not concerned with local delivery.

data

Follow the routing information with the data, starting with header lines.

EXPORT

None.

DEPENDENCIES

the dnsmx() function uses the dnsmx program from the djbdns tool package: it is abstracted into a function for easy replacement with your preferred MX lookup tool.

The file system holding the queue must support reading from a file handle after the file has been unlinked from its directory. If your computer can't do this, see the spot in the code near the phrase "UNLINK ISSUE" and follow the instructions.

For that matter, we also generate some long file names with lots of dots in them, which could conceivably not be portable.

HISTORY

0.03 17 April 2003

threw away some inefficient archtecture ideas, such as per-domain queues for connection reuse, in order to have a working system ASAP. Testing kill-zero functionality in test script.

0.04 20 April 2003

logging to $basedir/log/current instead of stdout, unless $LogToStdout is true. $AgeBeforeDeferralReport variable to suppress deferral bounces when a message has been queued for less than an interval.

0.05 22 April 2003

slight code and documentation cleanup

0.06 6 May 2003

Testing, testing, testing! make test on TipJar::MTA::queue before making test on this module, and you will send me two e-mails. Now using Sys::Hostname instead of `hostname` and gracefully handling absolutely any combination of carriage-returns and line-feeds as valid line termination.

0.07 1 June 2003

Wrapped all reads and writes to the SMTP socket in eval blocks, and installed a ALRM signal handler, for better handling of time-out conditions. Also added a $TipJar::MTA::TimeStampFrequency variable which is how many iterations of the main fork-and-send loop to make before logging a timestamp. The default is 200.

0.08 10 June 2003

minor cleanup.

0.09 12 June 2003

AOL and Yahoo.com and who knows how many other sticklers require angle brackets around return addresses and recipients. Improved handling of MXes that we cannot connect to, by defining a ReQueue_unconnected entry point in addition to the ReQueue one that we had already..

0.09 19 June 2003

We now bounce mail to domains that ( have no MX records OR there is only one MX record and it is the same as the domain name ) AND we could not resolve the one name. Previously it had been given the full benefit of the doubt.

0.10 24 June 2003

Better handling of slow peers. sysread does not block and eof cannot be used on sockets, did you know that? We check for socket openness by seeking and looking at the text of the resulting error message. Instead of using an OO interface. Also timeouts are now handled with a global variable instead of die because dieing from a signal handler does not get appear to get caught by an eval enclosing the point of execution at the time of the signal. Is this a bug?

Anyway TipJar::MTA.pm has been handling outgoing production e-mail for a couple weeks now.

To-do list and Known Bugs

Patches are welcome.

log rolling

there is no rotation of the log in the mylog() function. mylog does repoen the file by name on every logging event, though. Rewriting mylog to use Unix::Syslog or Sys::Syslog would be cool, but would add dependencies. Mailing the log to the postmaster every once in a while is easy enough to do from cron.

connection reuse and per-domain queues

have deferred messages organized by peer, when the deferral is because of connection problems, possibly by grouping the "immediate" messages by domain so we can reuse a connection instead of trying to make a new connection

ESMTP

take advantage of post-RFC-821 features

QMTP

use QMTP when available.

local deliveries

add MBOX and MailDir deliveries and some kind of configuration interface

AUTHOR

David Nicol, <davidnico@cpan.org>

SEE ALSO

TipJar::MTA::queue.