Next: Copying This Manual, Previous: Time and Date Formats, Up: Top [Contents][Index]
The following sections describe procedures for upgrading between the consecutive Mailfromd releases. The absence of a section for a pair of versions x-y numbers means that no specific actions are required for upgrading from x to y.
• 8170-9000 | Upgrading from 8.17 to 9.00 | |
• 8160-8170 | Upgrading from 8.16 to 8.17 | |
• 8140-8150 | Upgrading from 8.14 to 8.15 | |
• 8130-8140 | Upgrading from 8.13 to 8.14 | |
• 870-880 | Upgrading from 8.7 to 8.8 | |
• 850-860 | Upgrading from 8.5 to 8.6 | |
• 820-830 | Upgrading from 8.2 to 8.3 (or 8.4) | |
• 700-800 | Upgrading from 7.0 to 8.0 | |
• 600-700 | Upgrading from 6.0 to 7.0 | |
• 5x0-600 | Upgrading from 5.x to 6.0 | |
• 500-510 | Upgrading from 5.0 to 5.1 | |
• 440-500 | Upgrading from 4.4 to 5.0 | |
• 43x-440 | Upgrading from 4.3.x to 4.4 | |
• 420-43x | Upgrading from 4.2 to 4.3.x | |
• 410-420 | Upgrading from 4.1 to 4.2 | |
• 400-410 | Upgrading from 4.0 to 4.1 | |
• 31x-400 | Upgrading from 3.1.x to 4.0 | |
• 30x-31x | Upgrading from 3.0.x to 3.1 | |
• 2x-30x | Upgrading from 2.x to 3.0.x | |
• 1x-2x | Upgrading from 1.x to 2.x |
The ‘.mf’ suffix, that has long been used for MFL files and has been deprecated since version 8.15 (see 8140-8150) is no longer supported. MFL files have suffix ‘.mfl’.
prog
The old syntax was:
begin do ... end
It is no longer supported. The correct way to declare begin
and end
handlers is
prog begin do ... end
Since version 8.15 if a module was not found in module search path, the search was retried using include search path. This is no longer the case: the two paths serve different purposes and don’t interact in any way. MFL modules are searched in module search path only.
Version 9.0 features full IPv6 support.
The following modules used in previous releases are now deprecated. They are retained for backward compatibility. Any existing code that uses any of these will compile and work as in previous releases, except that a warning will be printed to draw your attention to the fact. You are advised to remove uses of these modules, as they will be removed in future versions:
match_cidr
The function match_cidr
is reimplemented as a built-in
function. Remove any instances of require 'match_cidr'
or
any equivalent import
statements from your code.
See match_cidr.
is_ip
This module defines the is_ip
function:
Returns 1 if str is a string representation of an IPv4 address and 0 otherwise.
This function has been obsoleted by is_ip4str
function.
You are advised to use it, or a more general is_ipstr
,
function instead. See Internet address manipulation functions, for a detailed discussion of these functions,
revip
This module defines the following function:
Reverses octets in ip, which must be a valid string representation of an IPv4 address.
This function has been obsoleted by reverse_ipstr
, discussed
in Internet address manipulation functions.
No special actions are required after upgrading. However, one point
is worth noticing. Until now, it was an error to specify two
prog
definitions with the same handler name. This rule
however, applied only to generic milter handlers, special handlers
(begin
, end
, startup
, shutdown
) being exempt
from it. Starting with version 8.17, multiple prog
declarations with the same handler name are allowed for all handlers.
See Multiple handler definitions, for details.
This release introduces certain changes that require the installer’s attention.
MFL source files now have the suffix ‘.mfl’ instead of ‘.mf’ which was used in previous versions. This is to avoid confusion with Metafont files, which also have suffix ‘.mf’.
All installed MFL module files have been renamed.
As of version 8.15, this change is of advisory nature, and the
legacy ‘.mf’ suffix is still supported. Thus, for example
if module ‘X’ was required and the file ‘X.mfl’ was not
found, mailfromd
will look for file ‘X.mf’.
Nevertheless, module authors are urged to rename their module
files accordingly.
In particular, the default script file is now named mailfromd.mfl. If it does not exist in the system configuration directory, then the file mailfromd.mf is looked up. Installers are advised to rename mailfromd.mf to mailfromd.mfl.
Until now, mailfromd
searched for module files in
include search path, which was mostly due to historical
reasons. Version 8.15 introduces a separate module search
path for this purpose. See module search path, for details.
All modules shipped with mailfromd
are installed to
the new location: prefix/share/mailfromd/8.15.
For backward compatibility, if unable to find module file in
the module search path, mailfromd
retries the search
using include search path, so that existing installation will still
work. A warning message is issued for each module file found in
include search path, to help you locate files that need moving
to the new place.
This additional search phase will remain active during a transitional period of couple of new releases to help users accommodate to the change.
Special handlers (see begin/end) are defined using the same syntax as standard milter stage handlers, i.e.:
prog begin do … done
Old syntax (without the prog
keyword) is still supported, but
causes a deprecation warning. Module writers are advised to update
their sources accordingly.
Two new special handlers are provided. See startup/shutdown, for details.
Deprecated configuration statements lock-retry-count
and
lock-retry-timeout
were removed in this version. Please
use the locking
statement instead. See Mailutils Configuration File in GNU Mailutils Manual, for
a detailed decription. To make the long story short:
Deprecated keyword | Use this instead |
---|---|
lock-retry-count | retry-count |
lock-retry-timeout | retry-sleep |
So that, instead of
lock-retry-count 10; lock-retry-timeout 1;
you would write:
locking { retry-count 10; retry-sleep 1; }
No special actions are required. There are several important changes
in version 8.14, most notable ones being improved dkim_sign
function, DNS resolver tolerance for CNAME chains, and buffered I/O.
This change is important for those who use dkim_sign
with
Sendmail. See dkim_sign and sendmail, for a detailed discussion.
Mailfromd dropped support for CNAME chains in version 8.3
(see 820-830). This change was forced by the behavior of
the adns
resolver library, which had apparently been grounded
on somewhat rigorous reading of RFCs
103432
and
218133.
Reportedly, many existing DNS servers still employ CNAME chains leading to
TXT records (in particular, to DKIM TXT records), so this change made certain
DKIM signatures unverifiable for mailfromd
.
To fix this, starting from version 8.14 mailfromd
implements
a work-around that allows it to follow CNAME chains of limited
length. The default length is 2 (which means CNAME pointing to a
CNAME, pointing to a valid RR). It can be changed using the
dns.max-cname-chain
configuration statement
(see conf-resolver).
Buffered I/O can tremendously improve performance of getdelim
and getline
.
The global variables io_buffering
and io_buffer_size
(see io_buffering) define buffering mode and associated buffer
size for file descriptors returned by the subsequent calls to
open
or spawn
. Buffering mode of an already open file
descriptor can be changed using the setbuf
function.
The io_buffering
variable defines the buffering mode. By
default it is 0 (BUFFER_NONE
), which disables buffering for
backward compatibility with the previous versions. Another
possible values are: 1 (BUFFER_FULL
) and 2 (BUFFER_LINE
).
When set to BUFFER_FULL
, all I/O operations become fully buffered.
The buffer size is defined by the io_buffer_size
global variable.
BUFFER_LINE
is similar to BUFFER_FILE
when used for
input. When used for the output, the data are accumulated in buffer
and actually sent to the underlying transport stream when the newline
character is seen. The io_buffer_size
global variable sets the
initial value for the buffer size in this mode. The actual size can
grow as needed during the I/O.
The default value for io_buffer_size
is the size of the system page.
The symbolic constants BUFFER_NONE
, BUFFER_FULL
and
BUFFER_LINE
are defined in the status.mf module. E.g.:
require status begin do io_buffering = BUFFER_FULL done
will set up all the subsequent buffer-aware I/O functions for buffered reads and writes using the maximum buffer capacity.
The setbuf
function (see setbuf) changes the buffering
mode and/or buffer size for an already opened stream, e.g.:
setbuf(fd, BUFFER_FULL, 4096)
For detailed discussion of various buffering modes and their effect on the I/O, see I/O functions.
DKIM support (see DKIM) introduced in this version requires the
Nettle cryptographic
library34. It you
need DKIM, make sure Nettle is installed prior to compiling
mailfromd
. Otherwise, no special actions are required.
New configure option --with-dbm allows you to select any
DBM flavor supported by GNU mailutils as the default DBM implementation
for mailfromd
.
Versions 8.3 and 8.4 differ only in required minimal version of mailutils (3.3 and 3.4, correspondingly). Apart from that, the following instructions apply to both versions.
In version 8.3 I abandoned the legacy DNS resolver and switched to GNU adns. GNU ands is a standalone resolver library, which provides a number of advanced features. It is included in most distributions. The source code of the recent release is available from http://www.chiark.greenend.org.uk/~ian/adns/adns.tar.gz.
This change brought a number of user-visible changes. In particular, arbitrary limits on the sizes of the RRs are removed. Consequently, the following configurations statements are withdrawn:
runtime.max-dns-reply-a
runtime.max-dns-reply-ptr
runtime.max-dns-reply-mx
max-match-mx
max-callout-mx
Secondly, the new resolver is less tolerant to deviations from the standard. This means that remote DNS misconfigurations that would have slipped unnoticed in previous versions, will be noticed by mailfromd 8.3. For example, a CNAME record pointing to another CNAME is treated as an error.
A new command line option was added to mailfromd
and
calloutd
: --resolv-conf-file. This option instructs
the programs to read resolver settings from the supplied file, instead
of the default /etc/resolv.conf.
Another user-visible change is in handling of SPF checks. Previously, results if SPF checks were cached in a database. This proved to cause more problems than solutions and was removed in this version. As a result, the following MFL global variables have been withdrawn:
spf_ttl
spf_cached
spf_database
spf_negative_ttl
Version 8.0 is a major rewrite, that introduces a lot of new concepts and features. Nevertheless, it is still able to run the MFL scripts from version 7.0 without modifications.
Note the following important points:
listen
configuration statement withdrawn
Use the server milter
statement instead. See conf-server.
This option was a noop since version 7.0.91.
The ‘%’ characters is used as modulo operator. See Arithmetic operations.
debug_spec
built-in function changed signature.
See debug_spec.
listens
and portprobe
The listens
function was moved to the portprobe module. It is
actually an alias to the portprobe
function. If your filter uses
listens
, make sure to require
the portprobe
module.
See portprobe.
_pollhost
, _pollmx
, stdpoll
, strictpoll
These functions have been moved to the poll module, which must be required prior to using any of them.
message_header_count
function.
This function takes an optional string argument, supplying the header name. See message_header_count.
The release 7.0 removes the features which were declared as obsolete in 6.0 and introduces important new features, both syntactical, at the MFL level, and operational.
Unless your filter used any deprecated features, it should work correctly after upgrade to this version. It will, however, issue warning messages regarding the deprecated features (e.g. the use of ‘%’ in front of identifiers, as described below). To fix these, follow the upgrade procedure described in upgrade procedure.
The removed features are:
The MFL syntax has changed: it is no longer necessary to use ‘%’ in front of a variable to get its value. To reference a variable, simply use its name, e.g.:
set x var + z
The old syntax is still supported, so the following statement will also work:
set x %var + %z
It will, however, generate a warning message.
Of course, the use of ‘%’ to reference variables within a string literal remains mandatory.
Another important changes to MFL are user-defined exceptions (see User-defined Exceptions) and the try–catch construct (see try–catch).
Several existing MFL functions have been improved. In
particular, it is worth noticing that the open
function, when
opening a pipe to or from a command, provides a way to control where
the command’s standard error would go (see standard error).
The accept
function (or action) issues a warning if its use would
cancel any modifications to the message applied by, e.g.,
header_add
and similar functions. See Message modification queue, for a detailed discussion of this feature.
The most important change in mailfromd
operation is that
the version 7.0 is able to run several servers (see conf-server).
Dedicated callout servers make it possible to run sender
verifications in background, using a set of long timeouts, as
prescribed by RFC 2822 (see SMTP Timeouts). This diminishes
the number of false positives, allows for coping with servers showing
large delays and also reduces the number of callouts performed for
such servers.
This release no longer includes the smap
utility. It was
moved into a self-standing project, which by now provides much more
functionality and is way more flexible than this utility was. If you
are interested in using smap
, visit
http://www.gnu.org.ua/software/smap, for a detailed
information, including pointers to file downloads.
The 6.0 release is aimed to fix several logical inconsistencies that
affected the previous versions. The most important one is that until
version 5.2, the filter script file contained both the actual filter
script, and the run-time configuration for mailfromd
(in
form of ‘#pragma option’ and ‘#pragma database’ statements).
The new version separates run-time configuration from the
filter script by introducing a special configuration file
mailfromd.conf (see Mailfromd Configuration).
Consequently, the ‘#pragma option’ and ‘#pragma database’ statements become deprecated. Furthermore, the following deprecated pragmas are removed: ‘#pragma option ehlo’, ‘#pragma option mailfrom’. These pragmas became deprecated in version 4.0 (see 31x-400).
The second problem was that the default filter script file had ‘.rc’ suffix, which usually marks a configuration file, not the source. In version 6.0 the script file is renamed to mailfromd.mf. In the absence of this file, the legacy file mailfromd.rc is recognized and parsed. This ensures backward compatibility.
This release also fixes various inconsistencies and dubious features in the MFL language.
The support for unquoted literals is discontinued. This feature was marked as deprecated in version 3.0.
The following features are deprecated: ‘#pragma option’ (pragma-option and ‘#pragma database’ (pragma-database) directives, the legacy style of function declarations (old-style function declarations), calls to functions of one argument without parentheses (operational notation), the ‘#require’ statement (See import, for the new syntax) and implicit concatenation (implicit concatenation). See Deprecated Features, for more information about these.
This release also introduces important new features, which are summarized in the table below:
Feature | Reference |
---|---|
Configuration | See Mailfromd Configuration. |
Module system | See Modules. |
Explicit type casts | See explicit type casts. |
Concatenation operator | See Concatenation. |
Scope of visibility | See scope of visibility. |
Precious variables | See rset. |
Mailfromd
version ‘6.0’ will work with unchanged
scripts from ‘5.x’. When started, it will verbosely warn you
about any deprecated constructs that are used in your filter sources
and will create a script for upgrading them.
To upgrade your filter scripts, follow the steps below:
mailfromd: Warning: using legacy script file /usr/local/etc/mailfromd.rc mailfromd: Warning: rename it to /usr/local/etc/mailfromd.mf or use script-file statement in /usr/local/etc/mailfromd.conf to disable this warning mailfromd: /usr/local/etc/mailfromd.rc:19: warning: this pragma is deprecated: use relayed-domain-file configuration statement instead mailfromd: /usr/local/etc/mailfromd.rc:23: warning: this pragma is deprecated: use io-timeout configuration statement instead mailfromd: Info: run script /tmp/mailfromd-newconf.sh to fix the above warnings …
mailfromd
will create a shell script
/tmp/mailfromd-newconf.sh for fixing these warnings. Run it:
$ sh /tmp/mailfromd-newconf.sh
mailfromd --lint
again. If
it shows no more deprecation warnings, the conversion went correctly.
Now you can remove the upgrade script:
$ rm /tmp/mailfromd-newconf.sh
Notice, that the conversion script attempts to fix only deprecation warnings. It will not try to correct any other type of warnings or errors. For example, you may get warning messages similar to:
mailfromd: /etc/mailfromd.mf:7: warning: including a module file is unreliable and may cause subtle errors mailfromd: /etc/mailfromd.mf:7: warning: use `require dns' instead
This means that you use ‘#include’ where you should have used ‘require’. You will have to fix such warnings manually, as suggested in the warning message.
If, for some reason, you cannot upgrade your scripts right now, you
may suppress deprecation warnings by setting the environment variable
MAILFROMD_DEPRECATION
to ‘no’ before starting
mailfromd
. Nonetheless, I recommend to upgrade as soon as
possible, because the deprecated features will be removed in version
‘6.1’.
Upgrading from 5.0 to 5.1 does not require any changes in your filter scripts. Notice, however, the following important points:
mailfromd
supports Milter
protocol version 6, which is compatible with Sendmail 8.14.0 and
newer. While being backward compatible with earlier Sendmail
releases, it allows you to use the new ‘prog data’ handler
(see data). It also supports macro negotiation, a
feature that enables Mailfromd to ask MTA to export the
macros it needs for each particular handler. This means that if you
are using Sendmail 8.14.0 or higher (or Postfix 2.5 or higher), you no
longer need to worry about exporting macro names in sendmail.cf
file.
The same feature is also implemented on the server side, in
mtasim
and pmult
. Consequently, using
define-macros
in pmult
configuration file
is not strictly necessary. However, keep in mind that due to the
specifics of MeTA1, the number of symbols that may be
exported for each stage is limited (see pmult-macros).
__preproc__
and __statedir__
built-in constant is slightly different from what it used to be in
5.0. These constants now refer to the current values of the
preprocessor command line and program state directory,
correspondingly. This should not affect your script, unless you
redefine the default values on run time. If your script needs to
access default values, use __defpreproc__
and
__defstatedir__
, correspondingly (see Built-in constants).
The following example explains the difference between these:
$ cat pval.mf prog envfrom do echo "Default value: " __defstatedir__ echo "Current value: " __statedir__ done $ mailfromd --state-directory=/var/mfd --test pval.mf Default value: /usr/local/var/mailfromd Current value: /var/mfd
rate
function,
you might consider using the new function rateok
or
tbf_rate
instead. For a detailed discussion of these functions,
see Sending Rate.
#pragma dbprop
(see dbprop).
This version of Mailfromd requires GNU mailutils version 2.0 or later.
Upgrading from version 4.4 to 5.0 requires no additional changes. The major differences between these two versions are summarized below:
getopt
and varptr
. See getopt.
verp_extract_user
, sa_format_report_header
,
sa_format_score
.
The stack can be grown either by fixed size blocks, or exponentially. Upper limit can be specified. See stacksize.
Support for some deprecated features has been withdrawn. These are:
The deprecated --domain command line option has been withdrawn. The short option -D now defines a preprocessor symbol (see Preprocessor Options).
This version correctly handles name clashes between constants and variables, which remained unnoticed in previous releases. See variable--constant shadowing, for a detailed description of it.
To minimize chances of name clashes, all symbolic exception codes has
been renamed by prefixing them with the ‘e_’, thus, e.g. divzero
became e_divzero
, etc. The ioerr
exception code is renamed to
e_io
. See status.mf, for a full list of the new exception codes.
For consistency, the following most often used codes are available without
the ‘e_’ prefix: success, not_found, failure, temp_failure. This
makes most existing user scripts suitable for use with version 4.4
without any modification. If your script refers to any exception
codes other than these four, you can still use it by defining a
preprocessor symbol OLD_EXCEPTION_CODES
, for example:
$ mailfromd -DOLD_EXCEPTION_CODES
Upgrading from 4.2 to 4.3 or 4.3.1 does not require any changes to your configuration and scripts. The only notable change in these versions is the following:
The asynchronous syslog implementation was reported to malfunction
on some systems (notably on Solaris), so this release does not enable
it by default. The previous meaning of the --enable-syslog-async
configuration option is also restored. Use this option in order to
enable asynchronous syslog feature. To set default syslog
implementation, use DEFAULT_SYSLOG_ASYNC
configuration variable
(see syslog-async).
The following deprecated features are removed:
#pragma option ehlo
statement.
It became deprecated in version 4.0. See pragma-option-ehlo.
#pragma option mailfrom
statement.
It became deprecated in version 4.0. See pragma-option-ehlo.
It became deprecated in version 3.1. See 30x-31x.
They are deprecated since version 4.0. See 31x-400.
Upgrading to this version does not require any special efforts. You
can use your configuration files and filter scripts without any
changes. The only difference worth noticing is that starting from this
version mailfromd
is always compiled with asynchronous
syslog implementation. The --enable-syslog-async
configuration file option is still available, but its meaning has
changed: it sets the default syslog implementation to use
(see syslog-async). Thus, it can be used the same way it was in
previous versions. You can also select the syslog implementation at
run time, see –syslog-async option, for
more detailed information.
Upgrading to this version does not require any special efforts. You can use your configuration files and filter scripts without any changes. Notice only the following major differences between 4.1 and 4.0:
sprintf
function
(see String formatting) and printf
macro
(see printf).
retry
, io-retry
, and
connect-retry
.
Before building this version, please re-read the chapter See Building, especially the section Using non-blocking syslog.
Starting from the version 4.0, MFL no longer uses the predefined symbolic names for exception codes (previous versions used the ‘&’ prefix to dereference them). Instead, it relies on constants defined in the include file status.mfh (see status.mf).
However, the script files from 3.1 series will still work, but the following warning messages will be displayed:
Warning: obsolete constant form used: &failure Warning: remove leading '&' and include <status.mfh> Warning: Using built-in exception codes is deprecated Warning: Please include <status.mfh>
Another important difference is that pragmatic options ‘ehlo’
and ‘mailfromd’ are now deprecated, as well as their command line
equivalents --ehlo and --domain. These options
became superfluous after the introduction of mailfrom_address
and ehlo_domain
built-in variables. For compatibility with the
previous versions, they are still supported by mailfromd
4.0, but a warning message is issued if they are used:
warning: `#pragma option ehlo' is deprecated, consider using `set ehlo_domain "domain.name"' instead
To update your startup scripts for the new version follow these steps:
#pragma option mailfrom value
to
set mailfrom_address value
. Refer to
mailfrom_address, for a detailed discussion of this variable.
#pragma option ehlo value
to
set ehlo_domain value
. Refer to
ehlo_domain, for a detailed discussion of this variable.
#include_once <status.mfh>
sed
expression: ‘s/&\([a-z]\)/\1/g’.
hostname
,
resolve
, hasmx
or ismx
, add the following line
to the top of your script:
#require dns
See Modules, for a detailed description of the module system.
next
with pass
.
match_cidr
, add the following line
to the top of your script:
#require match_cidr
See Modules, for a description of MFL module system.
mailfromd
binary no longer supports
--config-file (-c) option. To use an alternative
script file, give it as an argument, i.e. instead of:
$ mailfromd --config-file file.rc
write:
$ mailfromd file.rc
For backward compatibility, the old style invocation still works but
produces a warning message. However, if mailfromd
encounters the -c option it will print a diagnostic message
and exit immediately. This is because the semantics of this option
will change in the future releases.
prog helo
do
# Save the host name for further use
set helohost $s
done
has to be rewritten as follows:
set helohost ""
prog helo
do
# Save the host name for further use
set helohost $s
done
dbmap
takes an
optional third argument indicating whether or not to count the
terminating null character in key (see dbmap). If your startup
script contained any calls to dbmap
, change them as follows:
in 3.0.x | in 3.1 |
---|---|
dbmap(db, key) | dbmap(db, key, 1) |
Update your startup scripts and/or crontab entries. The
mailfromd
binary is now installed in ${prefix}/sbin.
We also encourage you to update the startup script (run cp etc/rc.mailfromd /wherever-your-startup-lives), since the new version contains lots of enhancements.
If you are upgrading from version 1.x to 2.0, you will have to do the following:
prog envfrom do … done
See Handlers, for more information about the prog
statement.
rate
statements, convert them to
function calls (see rate), using the
following scheme:
Old statement: if rate key limit / expr New statement: if rate(key, interval("expr")) > limit
For example,
rate $f 180 / 1 hour 25 minutes
should become
rate($f, interval("1 hour 25 minutes")) > 180
mailfromd --compact --all
This is necessary since the format of mailfromd
databases
has changed in version 2.0: the key field now includes the trailing
‘NUL’ character, which is also reflected in its length. This
allows for empty (zero-length) keys. See Database Maintenance, for
more information about the database compaction.
https://www.rfc-editor.org/rfc/rfc1034#section-3.6.2
https://www.rfc-editor.org/rfc/rfc2181#section-10.1