Preface

Simple Mail Transfer Protocol (SMTP) which is the standard for email transmissions across the Internet was designed in the good old days when nobody could even think of the possibility of e-mail being abused to send tons of unsolicited messages of dubious contents. Therefore it lacks mechanisms that could have prevented this abuse (spamming), or at least could have made it difficult. Attempts to introduce such mechanisms (such as SMTP-AUTH extension) are being made, but they are not in wide use yet and, probably, their introduction will not be enough to stop the e-mail abuse. Spamming is today’s grim reality and developers spend lots of time and efforts designing new protection measures against it. Mailfromd is one of such attempts.

The package is designed to work with any MTA supporting ‘Milter’ or ‘Pmilter’ protocol, such as ‘Sendmail’, ‘MeTA1’ or ‘Postfix’. It allows you to:

• Control whether messages come from trustworthy senders, using so called callout or Sender Address Verification (see SAV) mechanism.
• Prevent emails coming from forged addresses by use of SPF mechanism (see SPF Functions).
• Limit connection and/or sending rates (see Rate Limit).
• Use black-, white- and greylisting techniques.
• Invoke external programs or other mail filters.

Short history of mailfromd.

The idea of the utility appeared in 2005, and its first version appeared soon afterward. Back then it was a simple implementation of Sender Address Verification (see SAV) for ‘Sendmail’ (hence its name – mailfromd) with rudimentary tuning possibilities.

After a short run on my mail servers, I discovered that the utility was not flexible enough. It took less than a month to implement a configuration file that allowed the user to control program and data flow during the ‘envfrom’ SMTP state. The new version, 1.0, appeared in June, 2005.

Next major release, 1.2 (1.1 contained mostly bugfixes), appeared two months later, and introduced mail sending rate control (see Rate Limit).

The program evolved during the next year, and the version 2.0 was released in September, 2006. This version was a major change in the main idea of the program. Configuration file become a flexible filter script allowing the operator to control almost all SMTP states. The program supplied in the script file was compiled into a pseudo-code at startup, this code being subsequently evaluated each time the filter was invoked. This caused a considerable speed-up in comparison with the previous versions, where the run-time evaluator was traversing the parse tree. This version also introduced (implicitly, at the time), two separate data types for the entities declared in the script, which also played its role in the speed improvement (in the previous versions all data were considered strings). Lots of improvements were made in the filter language (see MFL) itself, such as user-defined functions, the switch statement, the catch statement for handling run-time errors, etc. The set of built-in functions extended considerably. A testsuite (using DejaGNU) was introduced in this version.

During this initial development period the limitations imposed by libmilter implementation became obvious. Finally, I felt they were stopping further development, and decided that mailfromd should use its own ‘Milter’ implementation. This new library, libgacopyz was the main new feature of the 3.0 release, which was released in November, 2006. Another major feature was the --dump-macros option and the macros subcommand to rc.mailfromd script, that were intended to facilitate configuration on the ‘Sendmail’ side.

The development of 3.x (more properly, 3.1.x) series concentrated mainly on bug-fixes, while the main development was done on the next branch.

The version 4.0 appeared on May 12, 2007. A full list of changes in this release is more than 500 lines long, so it is impractical to list them here. In particular, this version introduced lots of new features in MFL syntax and the library of useful MFL functions. The runtime engine was also improved, in particular, stack space become expandable which eliminated many run-time errors. This version also provided a foundation for MFL module system. The code generation was re-implemented to facilitate introduction of object files in future versions. Another new features in this release include SPF support and mtasim utility — an MTA simulator designed for testing mailfromd scripts (see mtasim). The test suite in this version was made portable by rewriting it in Autotest.

Another big leap forward was the 5.0 release, which appeared on December 26, 2008. It largely enriched a set of available functions (61 new functions were introduced, which amounts to 41% of all the available functions in 5.0 release) and introduced several improvements in the MFL itself. Among others, function aliases and optional arguments in user-defined functions were introduced in this release. The new “run operation mode” allowed to execute arbitrary MFL functions from the command line. This release also raised the Mailutils version requirements to at least 2.0.

Version 6.0, which was released in on 12 December, 2009, introduced a full-fledged modular system, akin to that of Python, and quite a few improvements to the language. such as explicit type casts, concatenation operator, static variables, etc.

Starting from version 7.0, the focus of further development of mailfromd has shifted. While previously it had been regarded as a mail-filtering server, since then it was developed as a system for extending MTA functionality in the broad sense, mail filtering being only one of features it provides.

Version 7.0 makes the MFL syntax more consistent and the language itself more powerful. For example, it is no longer necessary to use prefixes before variables to dereference them. The new ‘try--catch’ construct allows for elegant handling of exceptions and errors. User-defined exceptions provide a way for programming complex loops and recursions with non-local exits.

This version introduces a concept of dedicated callout server. This allows mailfromd to defer verifications for a later time if the remote server does not response within a reasonably short period of time (see SMTP Timeouts).

Six years later the version 8.0 was released. This version was a major rewrite of the mailfromd codebase. It introduced a separate callout daemon that made it possible to separate the mailfromd server machine from machines performing callout checks. The MFL language was extended by a number of built-in functions.

Since version 8.3 (2017-11-02) mailfromd uses ‘adns’¹ for DNS queries.

The version 8.7 released in July, 2020 introduced DKIM support.

The version 8.15 (2022-12-11) introduced dynamically loaded MFL modules. These modules use dynamically loaded libraries to extend the program functionality without having to modify its code. Several such modules were provided as separate projects. See mfmod.

The version 9.0 released in January, 2024 introduced full IPv6 support, separated module and search paths and added several new MFL functions.

Acknowledgments

Many people need to be thanked for their assistance in developing and debugging mailfromd. After S. C. Johnson, I can say that this program “owes much to a most stimulating collection of users, who have goaded me beyond my inclination, and frequently beyond my ability in their endless search for "one more feature". Their irritating unwillingness to learn how to do things my way has usually led to my doing things their way; most of the time, they have been right.”

A real test for a program like mailfromd cannot be done but in conditions of production environment. A decision to try it in these conditions is by no means an easy one, it requires courage and good faith in the intentions and abilities of the author. To begin with, I would like to thank my contributors for these virtues.

Jan Rafaj has intrepidly been using mailfromd since its early releases and invested lots of efforts in improving the program and its documentation. He is the author of many of the MFL library functions, shipped with the package. Some of his ideas are still waiting in my implementation queue, while new ones are consistently arriving.

Peter Markeloff patiently tested every mailfromd release and helped discover and fix many bugs.

Zeus Panchenko contributed many ideas and gave lots of helpful comments. He offered invaluable help in debugging and testing mailfromd on FreeBSD platform.

Sergey Afonin proposed many improvements and new ideas. He also invested a lot of his time in finding bugs and testing bugfixes.

John McEleney and Ben McKeegan contributed the token bucket filter implementation (see TBF).

Con Tassios helped to find and fix various bugs and contributed the new implementation of the greylist function (see greylisting types).

The following people (in alphabetical order) provided bug reports and helpful comments for various versions of the program: Alan Dobkin, Brent Spencer, Jeff Ballard, Nacho González López, Phil Miller, Simon Christian, Thomas Lynch.