4.25 Dynamically Loaded Modules

Native mailfromd modules described above rely on the functions provided by the mailfromd binary. For more sophisticated tasks you might need to use C functions, either for efficiency reasons or to make use of some third-party library. This is possible using special kind of modules called mfmod.

An mfmod consists of two major parts: a dynamically loaded library that provides its main functionality and a small interface mailfromd module. The convention is that for the module x the library is named mfmod_x.so¹⁹, and the interface module file is x.mfl.

At the time of this writing, three mfmods exist:

mfmod_pcre

Provides support for Perl-compatible regular expressions. It also contains a special function for scanning an email message for a match to a regular expression. See Mfmod_pcre in Mfmod_pcre.

mfmod_ldap

Functions for searching in LDAP directory. See Mfmod_ldap in Mfmod_ldap.

mfmod_openmetrics

Openmetrics support for mailfromd. See mfmod_openmetrics in mfmod_openmetrics reference.

The subsections below describe the internal structure of an mfmod in detail.

4.25.1 Loadable Library

External functions in the loadable library must be declared as

int funcname(long count, MFMOD_PARAM *param, MFMOD_PARAM *retval);

The MFMOD_PARAM type is declared in the header file mailfromd/mfmod.h, which must be included at the start of the source code.

mfmod type: MFMOD_PARAM type string number

This type is defined as follows:

typedef struct mfmod_param {
        mfmod_data_type type;
        union {
                char *string;
                long number;
                mu_message_t message;
        };
} MFMOD_PARAM;

The type fields defines the type of the data represented by the object. Its possible values are:

mfmod constant: mfmod_string

String data.

mfmod constant: mfmod_number

Numeric data.

mfmod constant: mfmod_message

A mailutils message object (mu_message_t).

The actual data are accessed as string, number, or message, depending on the value of type.

The first parameter in the external function declaration, count, is the number of arguments passed to that function. Actual arguments are passed in the MFMOD_PARAM array param. The function should never modify its elements. If the function returns a value to MFL, it must pass it in the retval parameter. For example, the following code returns the numeric value ‘1’:

retval->type = mfmod_number;
retval->number = 1;

To return a string value, allocate it using malloc, calloc or a similar function, like this:

retval->type = mfmod_string;
retval->string = strdup("text");

If a message is returned, it should be created using mailutils message creation primitives. Mailutils will call mu_message_destroy on it, when it is no longer used.

The return value (in the C sense) of the function is used to determine whether it succeeded or not. Zero means success. Returning -1 causes a runtime exception e_failure with a generic error text indicating the names of the module and function that caused the exception. Any other non-zero value is treated as a mailfromd exception code (see Exceptions). In this case an additional textual explanation of the error can be supplied in the retval variable, whose type must then be set to mfmod_string. This explanation string must be allocated using malloc.

To facilitate error handling, the following functions are provided (declared in the mailfromd/mfmod.h header file):

mfmod: int mfmod_error (MFMOD_PARAM *retval, int ecode, char const *fmt, ...)

Raises exception ecode with the error message formatted from the variadic arguments using printf-style format string fmt.

Example use:

if (error_condition)
   return mfmod_error(retval, "error %s occurred", error_text);

mfmod: int mfmod_error_argtype (MFMOD_PARAM *param, MFMOD_PARAM *retval, int n, int exptype)

Reports argument type mismatch error (e_inval with appropriately formatted error text). Arguments are:

param

retval

The two arguments passed to the interface function.

n

0-based index of the erroneous argument in param.

exptype

Expected data type of param[n].

You will seldom need to use this function directly. Instead, use the ASSERT_ARGTYPE macro described below.

mfmod: char const * mfmod_data_type_str(mfmod_data_type type)

Returns the MFL name of the mfmod data type type.

The following convenience macros are provided for checking the number of argument and their types and returning error if necessary:

mfmod macro: ASSERT_ARGCOUNT (MFMOD_PARAM *retval, long count, long expcount)

Assert that the number of arguments (count) equals the expected number (expcount). If it does not, return the e_inval exception with a descriptive error text.

retval and count are corresponding arguments from the calling function.

mfmod macro: ASSERT_ARGTYPE (MFMOD_PARAM *param, MFMOD_PARAM *retval, int n, int exptype)

Check if the data type of the nth parameter (i.e. param[n]) is exptype and return the e_inval exception if it does not.

As an example, suppose you want to write an interface to the system crypt function. The loadable library source, mfmod_crypt.c, will look as follows:

#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <mailfromd/mfmod.h>
#include <mailfromd/exceptions.h>

/*
 * Arguments:
 *   param[0] - key string to hash.
 *   param[1] - salt value.
 */
int
cryptval(long count, MFMOD_PARAM *param, MFMOD_PARAM *retval)
{
     char *hash;

     /* Check if input arguments are correct: */
     ASSERT_ARGCOUNT(retval, count, 2);
     ASSERT_ARGTYPE(param, retval, 0, mfmod_string);
     ASSERT_ARGTYPE(param, retval, 1, mfmod_string);

     /* Hash the key string. */
     hash = crypt(param[0].string, param[1].string);

     /* Return string to MFL */
     retval->type = mfmod_string;
     retval->string = strdup(hash);

     /* Throw exception if out of memory */
     if (retval->string == NULL)
          return -1;

     return 0;
}

The exact way of building a loadable library from this source file depends on the operating system. For example, on GNU/Linux you would do:

cc -shared -fPIC -DPIC -omfmod_crypt.so -lcrypt mfmod_crypt.c

The preferred and portable way of doing so is via libtool (see Shared library support for GNU in Libtool). Mailfromd provides a special command mfmodnew that creates infrastructure necessary for building loadable modules. See mfmodnew.

4.25.2 Interface Module

The interface module is responsible for loading the library, and providing MFL wrappers over external functions defined in it.

For the first task, the dlopen function is provided. It takes a single argument, the file name of the library to load. This can be an absolute pathname, in which case it is used as is, or a relative file name, which will be searched in the library search path (see mfmod-path). On success, the function returns the library handle, which will be used in subsequent calls to identify that library. On error, a runtime exception is signalled.

It is common to call the dlopen function in the startup section of the interface module (see startup/shutdown), so that the library gets loaded at the program startup. For example:

static number libh

prog startup
do
  set libh dlopen("mfmod_crypt.so")
done

The function dlcall is provided to call a function from the already loaded library. It is a variadic function with three mandatory parameters:

1. The handle of the loadable library as returned by dlopen.
2. Name of the external function to call.
3. Type string

The type string argument declares data types of the variable arguments. It contains a single letter for each additional argument passed to dlcall. The valid letters are:

s

The argument is of string type.

n

d

The argument is of numeric type.

m

The argument is of message type.

For example, the following will call the cryptval function defined in the previous section (supposing key and salt are two string MFL variables):

set x dlcall(libh, "cryptval", "ss", key, salt)

The last letter in type string can be ‘+’ or ‘*’. Both mean that any number of arguments are allowed (all of the type given by the penultimate type letter). The difference between the two is that ‘+’ allows for one or more arguments, while ‘*’ allows for zero or more arguments. For example, ‘n+’ means one or more numeric arguments, and ‘n*’ means zero or more such arguments. Both are intended to be used in variadic functions, e.g.:

func pringstddev(number ...)
  returns number
do
  return dlcall(libh, "stddev", "n*", $@)
done

The dlcall function returns the value returned by the library function it invoked. If the library function returns no meaningful value, it is recommended to use the void type cast around the dlcall invocation (see void type cast). E.g.:

func out(string text)
do
  void(dlcall(libh, "output", "s", text))
done

Without void type cast, the definition above will produce the following warning when compiled:

return from dlcall is ignored

4.25.3 Creating a Mfmod Structure

The mfmodnew provides a convenient start for writing a new mfmod. Given a name of the planned module, this command creates directory mfmod_name and populates it with the files necessary for building the new module using GNU autotools, as well as boilerplate files for the loadable library and interface module.

Let’s see how to use it to create the crypt module outlined in previous subsections.

First, invoke the command:

$ mfmodnew crypt
mfmodnew: setting up new module in mfmod_crypt

Let’s change to the new directory and see the files in it:

$ cd mfmod_crypt
$ ls
Makefile.am  configure.ac  crypt.mfl  mfmod_crypt.c

Now, open the mfmod_crypt.c file and add to it the definition of the cryptval function (see Loadable Library). Then, add the interface function definition from Interface Module to file crypt.mfl.

The last thing to do is to edit configure.ac. The crypt function requires the libcrypt library, so the following line should be added to the ‘Checks for libraries.’ section.

AC_CHECK_LIB([crypt], [crypt])

Now, run autoreconf, as follows:

$ autoreconf -f -i -s

It will bootstrap the autotools infrastructure, importing additional files as necessary. Once done, you can build the project:

$ ./configure
$ make

Notice, that if the autoreconf stage ends abnormally with a diagnostics like:

configure.ac:21: error: possibly undefined macro: AC_MFMOD

that means that autoconf was unable to find the file mfmod.m4, which provides that macro. That’s because the directory where this file is installed is not searched by autoreconf. To fix this, supply the name of that directory using the -I option. E.g. assuming mfmod.m4 is installed in /usr/local/share:

$ autoreconf -fis -I /usr/local/share/aclocal

4.25.3.1 mfmodnew invocation

The mfmodnew is invoked as:

mfmodnew [options] modname [dir]

where modname is the name of the new module and dir is the directory where to store the module infrastructure files. Normally you would omit dir altogether: in this case the utility will use mfmod_modname as the directory name.

Options are:

-C dir

Search for template files in dir, instead of the default location.

-e email

Supply the author’s email. The email is passed as argument to the AC_INIT macro in configure.ac. By default it is constructed as ‘username@hostname’. If it is incorrect, you can either edit configure.ac afterwards, or just supply the correct one using this option.

-q

Suppress informative messages.

-h

Display a short command line usage help.