![[image of the Head of a GNU]](/graphics/gnu-head-sm.jpg)
|
|
Wydawca |
Release Submission Daemon |
Sergey Poznyakoff |
Wydawca (split by chapter): 
wydawca.Upon startup, wydawca reads its settings from the
configuration file wydawca.conf. By default it is located
in $sysconfidr (i.e., in most cases /usr/local/etc, or
/etc), but an alternative location may be specified using
--config-file command line option (see config-file).
If any errors are encountered in the configuration file, the program reports them on its error output and exits with a non-zero status.
To test the configuration file without starting the server use
--lint (-t) command line option. It causes
wydawca to check configuration file for syntax errors and
other inconsistencies. If no errors were detected, the program exits
with code 0. Otherwise, the exit code is 78.
Using this option together with -d1 (--debug=1),
causes wydawca to produce a dump of the configuration parse
tree. Setting a higher debugging level (e.g. -d2 option) will
additionally prefix each statement in the dump with the file location
where it appeared.
Before parsing, configuration file is preprocessed using
m4 (see Preprocessor). To see the preprocessed
configuration without actually parsing it, use the -E command
line option. To avoid preprocessing it, use
--no-preprocessor option.
The rest of this section describes the configuration file syntax in
detail. You can receive a concise summary of all configuration
directives any time by running wydawca --config-help.
Wydawca configuration file consists of statements and comments.
There are three classes of lexical tokens: keywords, values, and separators. Blanks, tabs, newlines and comments, collectively called white space are ignored except as they serve to separate tokens. Some white space is required to separate otherwise adjacent keywords and values.
Comments may appear anywhere where white space may appear in the configuration file. There are two kinds of comments: single-line and multi-line comments. Single-line comments start with ‘#’ or ‘//’ and continue to the end of the line:
# This is a comment // This too is a comment
Multi-line or C-style comments start with the two characters ‘/*’ (slash, star) and continue until the first occurrence of ‘*/’ (star, slash).
Multi-line comments cannot be nested. However, single-line comments may well appear within multi-line ones.
Pragmatic comments are similar to usual single-line comments, except that they cause some changes in the way the configuration is parsed. Pragmatic comments begin with a ‘#’ sign and end with the next physical newline character. Wydawca version 4.0.3, understands the following pragmatic comments:
#include <file>#include fileInclude the contents of the file file. If file is an absolute file name, both forms are equivalent. Otherwise, the form with angle brackets searches for the file in the include search path, while the second one looks for it in the current working directory first, and, if not found there, in the include search path.
The default include search path is:
where prefix is the installation prefix.
New directories can be appended in front of it using -I (--include-directory) command line option (see include-directory).
#include_once <file>#include_once fileSame as #include, except that, if the file has already
been included, it will not be included again.
#line num#line num "file"This line causes wydawca to believe, for purposes of error
diagnostics, that the line number of the next source line is given by
num and the current input file is named by file.
If the latter is absent, the remembered file name does not change.
# num "file"This is a special form of #line statement, understood for
compatibility with the C preprocessor.
In fact, these statements provide a rudimentary preprocessing features. For more sophisticated ways to modify configuration before parsing, see Preprocessor.
A simple statement consists of a keyword and value separated by any amount of whitespace. Simple statement is terminated with a semicolon (‘;’).
Examples of simple statements:
daemon yes; pidfile /var/run/wydawca.pid;
A keyword begins with a letter and may contain letters, decimal digits, underscores (‘_’) and dashes (‘-’). Examples of keywords are: ‘group’, ‘file-sweep-time’.
A value can be one of the following:
A number is a sequence of decimal digits.
A boolean value is one of the following: ‘yes’, ‘true’, ‘t’ or ‘1’, meaning true, and ‘no’, ‘false’, ‘nil’, ‘0’ meaning false.
An unquoted string may contain letters, digits, and any of the following characters: ‘_’, ‘-’, ‘.’, ‘/’, ‘@’, ‘*’, ‘:’.
A quoted string is any sequence of characters enclosed in double-quotes (‘"’). A backslash appearing within a quoted string introduces an escape sequence, which is replaced with a single character according to the following rules:
| Sequence | Replaced with |
| \a | Audible bell character (ASCII 7) |
| \b | Backspace character (ASCII 8) |
| \f | Form-feed character (ASCII 12) |
| \n | Newline character (ASCII 10) |
| \r | Carriage return character (ASCII 13) |
| \t | Horizontal tabulation character (ASCII 9) |
| \v | Vertical tabulation character (ASCII 11) |
| \\ | A single backslash (‘\’) |
| \" | A double-quote. |
Table 4.1: Backslash escapes
In addition, the sequence ‘\newline’ is removed from the string. This allows to split long strings over several physical lines, e.g.:
"a long string may be\ split over several lines"
If the character following a backslash is not one of those specified above, the backslash is ignored and a warning is issued.
Two or more adjacent quoted strings are concatenated, which gives another way to split long strings over several lines to improve readability. The following fragment produces the same result as the example above:
"a long string may be" " split over several lines"
Depending on the context, the quoted string may be subject to variable expansion.
During variable expansion, references to variables in the string are replaced with their actual values. A variable reference has two basic forms:
$v
${v}
where v is the variable name. The notation in curly braces serves several purposes. First, it should be used if the variable reference is immediately followed by an alphanumeric symbol, which will otherwise be considered part of it (as in ‘${home}dir’). Secondly, this form allows for specifying the action to take if the variable is undefined or expands to an empty value.
The following special forms are recognized:
Use Default Values. If variable is unset or null, the expansion of word is substituted. Otherwise, the value of variable is substituted.
Assign Default Values. If variable is unset or null, the expansion of word is assigned to variable. The value of variable is then substituted.
The assigned value remains in effet during expansion of the current string.
Display Error if Null or Unset. If variable is null or unset, the expansion of word (or a message to that effect if word is not present) is output to the current logging channel. Otherwise, the value of variable is substituted.
Use Alternate Value. If variable is null or unset, nothing is substituted, otherwise the expansion of word is substituted.
These constructs test for a variable that is unset or null. Omitting the colon results in a test only for a variable that is unset.
If a string contains a reference to an undefined variable,
wydawca will report an error and abort. To
gracefully handle such cases, use the default value construct,
defined above.
A here-document is a special construct that allows to introduce strings of text containing embedded newlines.
The <<word construct instructs the parser to read all
the following lines up to the line containing only word, with
possible trailing blanks. Any lines thus read are concatenated
together into a single string. For example:
<<EOT A multiline string EOT
Body of a here-document is interpreted the same way as double-quoted string, unless word is preceded by a backslash (e.g. ‘<<\EOT’) or enclosed in double-quotes, in which case the text is read as is, without interpretation of escape sequences.
If word is prefixed with - (a dash), then all leading
tab characters are stripped from input lines and the line containing
word. Furthermore, if - is followed by a single space,
all leading whitespace is stripped from them. This allows to indent
here-documents in a natural fashion. For example:
<<- TEXT
All leading whitespace will be
ignored when reading these lines.
TEXT
It is important that the terminating delimiter be the only token on its line. The only exception to this rule is allowed if a here-document appears as the last element of a statement. In this case a semicolon can be placed on the same line with its terminating delimiter, as in:
help-text <<-EOT
A sample help text.
EOT;
A list is a comma-separated list of values. Lists are enclosed in parentheses. The following example shows a statement whose value is a list of strings:
alias (test,null);
In any case where a list is appropriate, a single value is allowed without being a member of a list: it is equivalent to a list with a single member. This means that, e.g.
alias test;
is equivalent to
alias (test);
The time interval specification is a string that defines an interval, much the same way we do this in English: it consists of one or more pairs ‘number’-‘time unit’. For example, the following are valid interval specifications:
"1 hour" "2 hours 35 seconds" "1 year 7 months 2 weeks 2 days 11 hours 12 seconds"
The pairs can occur in any order, however unusual it may sound to a human ear, e.g. ‘2 days 1 year’. If the ‘time unit’ is omitted, seconds are supposed.
A block statement introduces a logical group of statements. It consists of a keyword, followed by an optional value, and a sequence of statements enclosed in curly braces, as shown in the example below:
spool download {
source /home/ftp/incoming/ftp;
destination /home/ftp/pub;
}
The closing curly brace may be followed by a semicolon, although this is not required.
Before parsing its configuration file, wydawca preprocesses
it. The built-in preprocessor handles only file inclusion
and #line statements (see Pragmatic Comments), while the
rest of traditional preprocessing facilities, such as macro expansion,
is supported via m4, which is used as an external preprocessor.
The detailed description of m4 facilities lies far beyond
the scope of this document. You will find a complete user manual in
http://www.gnu.org/software/m4/manual.
For the rest of this subsection we assume the reader is sufficiently
acquainted with m4 macro processor.
The external preprocessor is invoked with -s flag, which instructs it to include line synchronization information in its output. This information is then used by the parser to display meaningful diagnostic. An initial set of macro definitions is supplied by the pp-setup file, located in $prefix/share/wydawca/version/include directory (where version means the version of Wydawca package).
The default pp-setup file renames all m4 built-in
macro names so they all start with the prefix ‘m4_’. This
is similar to GNU m4 --prefix-builtin options, but has an
advantage that it works with non-GNU m4 implementations as
well.
To examine the preprocessed configuration, use the -E
option. The output from m4 will be printed on the standard
output and the program will terminate.
Additional control over the preprocessor is provided via the following command line options:
Define the preprocessor symbol name as having value, or empty.
Add dir to the list of directories searched for preprocessor include files.
Disable preprocessor.
Use command instead of the default preprocessor.
If bool is ‘yes’, run in foreground. See foreground.
Set the default umask. The value argument must be an octal number.
Consider triplet expired if its oldest file was created more than time seconds ago. See time interval specification, for the syntax of time. Default is 300 seconds.
This parameter may also be set for each spool individually. See file-sweep-time.
Set default GPG home directory. The keys for signing outgoing messages are looked up in this directory. See gpg-sign, and gpg-sign.
At the time of this writing, FSF has published three versions of the upload directives, numbered 1.0 through 1.2. The version 1.0 is considered obsolete and was withdrawn in 2006. The only difference between versions 1.1 and 1.2 is in handling of files that existed prior to upload. The version 1.1 implied automatic archivation of the existing files and their replacement with the newly uploaded versions. The version 1.2 introduces a new keyword (‘replace’) for that purpose, which determines its further actions.
For a detailed information about version 1.1, see Standalone directives.
The version 1.2 and its differences from 1.1 are discussed in Standalone directives.
By default, wydawca supports both versions. The supported range of
versions can be abridged using the following configuration statements:
Sets minimal allowed directive file version. The vn argument must have the form ‘major.minor’ and can not be less than ‘1.1’.
Sets maximal allowed directive file version.
For example, the following statements configure wydawca to
accept only directive files of version 1.2:
min-version 1.2; max-version 1.2;
Wydawca refuses to run with the root privileges. You
should configure its user privileges by using user and,
optionally, group statements in its configuration file:
Run with UID and GID of the user name.
Retain the supplementary groups from the list. The latter must contain group names. For example:
group (nogroup, ftp);
Statements in this section configure the daemon mode.
Enable daemon mode.
Enables or disables the inotify watcher. By default, inotify is always enabled on GNU/Linux systems (unless disabled at the configure time). It can also be configured for each spool individually (See inotify. See inotify, for a detailed description of this feature.
Listen on this socket for incoming upload notifications (see upload notification). Allowed values for url are:
Listen on IPv46. address ip. Ip may be given either in a dotted quad notation or as a symbolic host name. Port is either a decimal port name, or a service name from /etc/services.
Listen on the UNIX socket file file, which is either an absolute or relative file name.
Declare a special service name, which, when used in a upload notification request, will be treated as a request to process all spools.
Limits the number of upload notification connections allowed to be open simultaneously. The default value is 16 connections.
Sets the idle timeout for upload notification connections. If a connection stays idle for more than the given interval, it will be closed forcibly. Default idle timout is 10 seconds.
See time interval specification, for the syntax of interval.
Store master process PID in file. Default pidfile location is localstatedir/run/wydawca.pid.
Access to the socket specified in listen statement is
controlled by the tcp-wrapper block statement:
tcp-wrapper {
enable arg:boolean;
daemon name:string;
allow-table file:string;
deny-table file:string;
allow-syslog-priority prio:string;
deny-syslog-priority prio:string;
}
This statement is available only if wydawca was compiled
with support for TCP wrappers.
Enable or disable the use of TCP wrappers.
Set the daemon name. It is the name before the colon in
the access control file, that marks the line controlling access to
wydawca. The default is ‘wydawca’.
File name of the positive access control file. By default /etc/hosts.allow.
File name of the negative access control file. By default /etc/hosts.deny.
Log allowed accesses via the given syslog priority.
Log denied accesses via the given syslog priority.
Allowed values for prio in the ‘allow-syslog-priority’ and ‘deny-syslog-priority’ statements are: ‘emerg’, ‘alert’, ‘crit’, ‘err’, ‘warning’, ‘notice’, ‘info’, and ‘debug’.
Unless told otherwise, wydawca uses syslog to print
its diagnostic messages. By default, the program uses the
‘daemon’ facility. The syslog statement allows to change that:
syslog {
facility local1;
tag wydawca;
print-priority yes;
}
Configures the syslog facility to use. Allowed values are: ‘auth’, ‘authpriv’, ‘cron’, ‘daemon’, ‘ftp’, ‘local0’ through ‘local7’, and ‘mail’. These names are case-insensitive and may be optionally prefixed with ‘log_’ (case-insensitive as well).
This statement sets the syslog tag, a string identifying each message issued by the program. By default, the name of the program (‘wydawca’) is used.
In addition to priority segregation, provided by syslog,
you can instruct wydawca to prefix each syslog message with
its priority. To do so, set:
print-priority yes;
Several statements in configuration file may need to access an
SQL database. Wydawca is able to use any number of
databases simultaneously, the only restriction being that they must be
MySQL databases (this restriction will be removed in future
releases).
A database is defined using sql block statement:
sql id {
config-file file;
config-group group;
host hostname;
database dbname;
user username;
password string;
ssl-ca string;
}
Here, id is a string uniquely identifying this database. It is used by other configuration statements (e.g. by dictionaries, see the next section) to refer to this database.
Set the name of the SQL configuration file to read.
Set the name of the group in the SQL configuration file, from where to read configuration options.
The statements above allow to keep all security-sensitive
information, such as SQL username and password, in an
external configuration file and thus to relax permission requirements
for wydawca.conf. The exact format of such external configuration
file depends on the flavor of SQL DBMS in use.
As of version 4.0.3 wydawca supports only
‘MySQL’, so the configuration file is what is called option
file in ‘MySQL’ parlance
(see option files).
For example, suppose your wydawca.conf contains the following:
sql default {
config-file /etc/wydawca.mysql;
config-group wydawca;
}
Then, the /etc/wydawca.mysql would contain the actual parameters for accessing the database, e.g.:
[wydawca] socket = /var/db/mysql.sock database = savane user = savane pass = guessme
Another way to specify database credentials is by using the statements described below. If you prefer this way, you will have to tighten the permissions of wydawca.conf so that no third person could see the SQL password. The recommended permissions are ‘0600’.
Set the hostname or IP address of the host running the database. Optional port-or-socket specifies port number (for TCP connections) or socket name (for UNIX sockets) to use. In the latter case, the hostname and the colon may be omitted. If, however, it is present, it must be ‘localhost’.
Specifies the database name.
Sets the database user name.
Sets the password for accessing the database.
Sets the pathname to the certificate authority file, if you wish to use a secure connection to the server via SSL.
An example sql statement follows:
sql default {
host db.example.org:3306;
database savane;
user root;
password guessme;
}
It is possible to combine both methods, e.g.:
sql default {
config-file /etc/wydawca.sql;
host db.example.org:3306;
database savane;
}
Then, wydawca will attempt to obtain the missing
information (username and password, in this case) from the
/etc/wydawca.sql file.
A dictionary defines the ways to retrieve user information necessary to verify the submission. This information can be, for example, the user’s PGP key or his permissions on a project.
A dictionary is defined in configuration file using the following syntax:
dictionary dict-id {
type type;
query string;
params (param1,param2,…);
}
The dictionary statement can appear either in the global scope of
the configuration file, or inside a spool statement
(see spool). Global definitions affect all spools
in the configuration file, and ones inside a spool
statement override them for that particular spool.
There are two dictionaries, identified by the value of dict-id tag:
Keeps email addresses and real names of administrators (or owners) of a project. It may return any number of rows, each one consisting of two columns: an email address and a user name, in this order.
Keeps system user names, real names, emails and GPG keys of the users that are allowed to make uploads for the project.
The sub-statements of dictionary are:
Defines the type of this dictionary. Name is one of the following:
The data are supplied in the configuration file.
Retrieve data from an SQL database. Currently only
MySQL is supported.
Retrieve data using an external program. This dictionary type is reserved for future use.
See below for a detailed description of these dictionary types.
Sets the query used for retrieving the data. The string is subject to variable expansion (see variable expansion). The following variables are defined in this context:
projectThe system name of the project for which the triplet is
submitted. The project name is obtained from the directory
directive. If the value of this directive contains subdirectories,
the first (topmost) directory is used as ‘project’.
spoolThe name of the distribution spool where this upload originates (see spool).
urlThe URL of the spool, as set in the url statement
of the spool block (see url).
dirDirectory (relative to the project distribution root) where the files are going to be uploaded.
dest_dirSpool destination directory (see destination).
source_dirSpool source directory (see source).
useruser:nameThe system name of the user that submitted the triplet. This is defined only for ‘project-owner’ dictionaries.
commentThe value of the ‘comment’ field from the directive file.
Supplies additional parameters.
Dictionaries of ‘sql’ type retrieve information from an SQL database (as of version 4.0.3, only ‘MySQL’ databases are supported).
The query statement supplies the SQL query to
execute. Normally, it should be a SELECT query.
The params statement must supply a single parameter –
the identifier of one of the preceding sql blocks (see sql),
which determines database name and user credentials needed to access it.
The following sub-nodes contain sample definitions for the
sql dictionaries. They are based on the database structure used in
Savane system.
This dictionary retrieves email addresses and real names of administrators (or owners) of a project. It may return any number of rows, each one consisting of two columns: an email address and a user name, in this order.
dictionary project-owner {
type sql;
params (default);
query "SELECT user.email, user.realname "
"FROM user,user_group,groups "
"WHERE user_group.user_id=user.user_id "
"AND user_group.group_id=groups.group_id "
"AND user_group.admin_flags = 'A' "
"AND groups.unix_group_name = '${project}'";
}
This dictionary assumes that the ‘user’ table has a special column, ‘upload_flags’, whose value is ‘Y’ for those users who can do uploads for this project:
dictionary project-uploader {
type sql;
params (default);
query "SELECT user.email, user.realname "
"FROM user,user_group,groups "
"WHERE user_group.user_id=user.user_id "
"AND user_group.group_id=groups.group_id "
"AND user_group.upload_flags = 'Y' "
"AND groups.unix_group_name = '${project}'";
}
Builtin dictionaries are small dictionaries that keep all data
in their params list. They are designed mainly for testing
purposes.
Look ups in builtin dictionaries are performed as follows:
The query value is expanded (see query). The resulting
value is used as a key for lookup in params list.
The list scanned as follows:
Let i be the index of the current element in params.
Set i to 0.
Get the ith element.
Exact comparison. The key must be exactly equivalent to the dictionary field.
Dictionary field is treated as an fnmatch globbing pattern. See globbing pattern in glob man page.
Dictionary field is treated as a regular expression. Unless configured otherwise by flags (see below), POSIX extended regular expressions are used (see Extended regular expressions in GNU sed).
If that word ends with a comma, the characters following it are flags, defining the type of matching. Allowed flags are:
| Flag | Meaning |
|---|---|
| i | Ignore case |
| b | Use basic regular expressions |
For example, the string ‘/exact,i’ specifies case-insensitive exact comparison, the string ‘/regex,bi’ specifies case-insensitive basic regular expression matching, etc.
Go to step ‘INCR’.
Compare the element with the key, using currently selected comparison method.
i+1
through i+n to the result set. The value for
n is selected as follows:
| Dictionary | n |
|---|---|
| project-owner | 2 |
| project-uploader | 4 |
i = i + n
Set i = i + 1.
If i is greater than the number of elements in param,
then stop. Otherwise, go to step ‘GETEL’.
For example, the following defines the ‘project-owner’ dictionary, containing data for projects ‘foo’ and ‘bar’:
dictionary project-owner {
type builtin;
query "${project}";
params ("/exact",
"foo", "foo-owner@domain.net", "Foo Admin",
"bar", "smith@other.net", "John Smith");
}
As of version 4.0.3 this dictionary is not yet implemented.
Wydawca operates on three kinds of directories: spool source
directories (see source), destination directories
(see destination) and archive directories
(see archivation). By default, wydawca assumes that all
directories specified in its configuration file already exist and have
proper ownership and modes. It will abort if it is not so.
You can configure wydawca to create these directories as
needed, and to set up their ownership and modes automatically.
If set to ‘yes’, this statement instructs wydawca to
create any missing directories.
Specifies the mode for created directories (in octal). If the directory already exists, its mode will be checked and if necessary changed to mode.
This statement is overridden by per-directory statements:
source-mode and destination-mode statements in
spool block (see spool) and directory-mode statement
in archive block (see archivation).
Configures owner user and group IDs for source, destination and archive directories.
The uid argument is either a numeric UID prefixed with a plus sign, or a symbolic user name, which will be converted to the numeric UID using the system user database. If a number without the ‘+’ prefix is supplied, it will first be looked in the password database as the user name, and, if no such user is found, it will be used as the numeric UID.
The same holds for the gid argument.
This statement is overridden by per-directory statements:
source-owner and destination-owner statements in
spool block (see spool) and directory-owner statement
in archive block (see archivation).
Notice, that both directory-mode and directory-owner
apply only to the last component of the created directory
(‘basename’). Any intermediate directories are created with
default mode and ownership.
There may be cases when project maintainers need to overwrite
existing distributed files with another ones, having the same names.
(Note, however, that this practice is not encouraged). In that case,
wydawca needs to first archive the already existing
file, and then put the new one in its place. Moreover, the directive
file format allows maintainers to explicitly require archivation of
their existing files.
Wydawca supports two basic archivation methods: to a
tar file, and to a separate directory. The method to be used
is configured using archive statement. This statement can
appear either in the global scope, in which case it affects all
spools, or within a spool block (see spool),
where it affects only the given spool.
archive type {
# Name of archive file or directory
name file-or-dir;
# Define backup type
backup type;
# mode for the archive directory
directory-mode mode;
# owner user and group for the archive directory
directory-owner uid gid;
}
The type argument specifies the archivation type:
Disable archivation.
Add to a tar archive.
Store file in a separate directory.
Specify the name of the tar archive (if type ‘tar’ is used) or destination directory (if type ‘directroy’ is used).
If the archivation type tar is used, the name
statement sets the full name of the tar archive to use, e.g.:
archive tar {
name /var/spool/uploads/archive.tar;
}
The file being archived is appended to the archive using
tar -r (see Appending Files to an Archive in GNU tar: an
archiver tool). Any archived instance can subsequently be retrieved
using GNU tar --occurrence option (see Multiple Files with the Same Name in GNU tar: an archiver tool).
By default, wydawca will search for tar binary in
your search path. If you wish to use a particular binary, you may
specify its full file name using tar-program statement.
The ‘directory’ archivation type means that archive copies will
be stored in a directory specified by the name statement.
If it begins with a slash (i.e. represents an absolute
file name), an exact copy of the distribution directory hierarchy will
be created under it. For example, given this configuration:
archive directory {
name /var/backups/gnu;
}
all files from /home/ftp/gnu/tar will be archived in /var/backups/gnu/tar, and files from /home/ftp/gnu/tar/old will be archived in /var/backups/gnu/tar/old, etc.
If the directory name does not begin with a slash, it will be created under the corresponding distribution directory. For example, the following archivation settings:
archive directory {
name .archive;
}
mean that files from /home/ftp/gnu/tar will be archived in the directory /home/ftp/gnu/tar/.archive, files from /home/ftp/gnu/tar/old — in /home/ftp/gnu/tar/.archive/old, etc.
When using the ‘directory’ archivation type, it may happen that the archive file with the same name as the one about to be created already exists. This statement specifies how to handle the existing copy, in other words, how to backup it. The type argument corresponds to the ‘version-control’ Emacs variable. The following table describes its possible values:
Always make numbered backups.
Make numbered backups of files that already have them, and simple backups of the others.
Always make simple backups.
If no backup method is given, ‘existing’ is assumed
Sets directory mode for creating the directory (octal). If the directory already exists, its mode will be checked and if necessary changed to mode.
This statement overrides the global directory-mode statement
(see directory setup).
Configures owner user and group IDs for created archive directories. If the archive directory already exists, its ownership will be checked and if necessary reverted to uid:gid.
See directory-owner, for a discussion of the syntax for uid and gid.
This statement overrides the global directory-mode statement
(see directory setup).
Signature files (i.e. the ones ending with ‘.sig’) are usually
located in the same directory as the files they sign. To enforce this
rule, wydawca implements implicit signature
archivation facility. It works as follows. When archivation of
file is requested by archive: file statement in the
directive file, wydawca also checks if the file named
file.sig exists. If so, it is archived along with
file.
If implicit signature archivation is not needed, use
the archive-signatures statement to disable it, e.g.:
archive-signatures no;
A distribution spool defines the location of the source directory and the corresponding distribution (or destination) directory. It may also set archivation type, various dictionaries and notifications for that directory, thus overriding the global settings.
The spool block statement defines a distribution spool:
spool tag {
url url;
alias (aliases);
inotify bool;
source dir;
source-mode mode;
source-owner uid gid;
destination dir;
destination-mode mode;
destination-owner uid gid;
file-sweep-time interval;
dictionary { … }
archive { … }
notify-event { … }
}
The tag argument defines a unique identifier for this spool. It will be used in log messages and is available for variable expansion (see variable expansion) as the ‘$spool’ variable.
Defines a list of aliases, i.e. alternative tag names for this spool.
Enables or disables the inotify watcher for this spool. By default, inotify is always enabled on GNU/Linux systems (unless explicitly disabled at the configure time). See inotify, for a detailed description of this feature.
Defines download URL, associated with this spool. Its value may be used as the variable ‘$url’ in mail notifications.
Specifies the location of the source directory.
Sets directory mode for creating the source directory (octal). If the directory already exists, its mode will be checked and if necessary changed to mode.
This statement overrides the global directory-mode statement
(see directory setup).
Configures owner user and group IDs for the source directory. If the directory already exists, its ownership will be checked and if necessary reverted to uid:gid.
See directory-owner, for a discussion of the syntax for uid and gid.
This statement overrides the global directory-mode statement
(see directory setup).
Specifies the type and location of the destination directory. The
dir argument must be either an absolute name of a directory on
the local file system, or a special URL. Wydawca
version 4.0.3 supports two destination URL
schemes:
Equivalent to dir-name alone. Defines a destination directory located on the local file system.
Defines a null upload spool. Null spools implement all tests described in overview, but do not do any actual copying. The uploaded files are simply removed after checks are over. Null spools are useful mainly for diagnostic purposes.
The following two statements apply only if the destination is a local directory (‘file://’ or ‘dir://’ URL scheme):
Sets directory mode for creating the destination directory (octal). If the directory already exists, its mode will be checked and if necessary changed to mode.
This statement overrides the global directory-mode statement
(see directory setup).
Configures the owner user and group IDs for the destination directory. If the directory already exists, its ownership will be checked and if necessary reverted to uid:gid.
See directory-owner, for a discussion of the syntax for uid and gid.
This statement overrides the global directory-mode statement
(see directory setup).
The following statements, if present, override the corresponding global definitions for this spool.
Configure spool-specific archivation. See archivation, for its description.
Configure spool-specific dictionary. See dictionaries, for a detailed discussion of this statement.
Set expiration time for triplets in this spool. A triplet is considered expired if its oldest file was created more than time seconds ago. This statement overrides the global ‘file-sweep-time’ setting (see file-sweep-time).
Configure spool-specific event notification. See notification, for a detailed discussion of this statement.
The source and destination statements are mandatory.
For example, the following definition says that valid uploads to /home/ftp/incoming/ftp should be transferred to /home/ftp/gnu:
spool ftp {
url ftp://ftp.gnu.org.ua;
source /home/ftp/incoming/ftp;
destination /home/ftp/gnu;
}
This spool defines no particular archivation type, dictionary or notifications, so it will inherit these settings from the global configuration.
The following example shows the same spool, that additionally sets its own archivation method:
spool ftp {
url ftp://ftp.gnu.org.ua;
source /home/ftp/incoming/ftp;
destination /home/ftp/gnu;
archive directory {
name .archive;
backup numbered;
}
}
After the submission has been verified, wydawca may also run
an additional check to verify whether the main file (normally,
a tarball) is OK to be distributed. To set up such distribution
verification, add the following statement either in the global scope,
or within a ‘spool’ declaration:
Define the distribution verification script. The text must be a valid sh program. It is executed without arguments, in a temporary directory which contains a copy of the main distribution file. The script can refer to the following environment variables:
Spool tag.
Spool source directory, as set by the source statement
(see tag).
Spool destination directory (see destination).
Spool URL (see url).
Base name of the triplet.
File name of the main distribution file.
Apart from these, the script inherits wydawca environment.
The submission is accepted only if the script returns 0. Otherwise, it is rejected and the ‘check-failure’ event (see event notification) is generated.
In case of non-zero return, the script may return additional diagnostics on the standard output. This diagnostics will be available for use in notification messages via the ‘$check:diagn’ variable.
Additionally, the actual return code of the script, in decimal, is available in the ‘$check:result’ variable. If the script terminates on a signal, the value of this variable is ‘SIG+n’, where n is the signal number.
If both global and spool ‘check-script’s are defined,
wydawca executes both scripts as if they were connected
by a logical ‘&&’, i.e. per-spool script is executed only if
the global one returned success (‘0’). The submission is accepted
only if both scripts returned ‘0’.
Since the script usually contains several lines, the ‘config-script’ value is usually supplied using a here-document construct (see here-document).
The following example illustrates the use of ‘config-script’ to catch possible security holes in the distributed Makefile.in files7
check-script <<EOT
case ${WYDAWCA_DIST_FILE} in
*.tar|*.tar.*)
if tar -xOf ${WYDAWCA_DIST_FILE} --occurrence=1 \
--wildcards --no-wildcards-match-slash '*/Makefile.in' | \
grep -q 'perm -777'; then
fmt <<_EOF_
The top-level Makefile.in in ${WYDAWCA_DIST_FILE} changes mode of
all the directories below the build tree to 777 before creating
the tarball. This constitutes a security hole (see CVE-2009-4029[1],
for more details).
Please, rebuild the package using a newer Automake (at least v. 1.11.1)
and resubmit.
_EOF_
cat <<_EOF_
--
[1] http://article.gmane.org/gmane.comp.sysutils.autotools.announce/131
_EOF_
exit 1
fi
;;
*)
;;
esac
exit 0
EOT;
Periodically wydawca produces statistic dumps. These dumps
are displayed on the diagnostic channel ‘info’ (and optionally
mailed to the admimistrator). The frequency with which they are
produced is defined by the stat-report-schedule configuration
statement.
Schedules generation of statistic reports. The time argument is a time specification in ‘crontab’ format (see crontab in crontab(5) manual page). By default, reports are generated hourly.
To create reports each three hours, set
stat-report-schedule "0 */3 * * *";
To create them at midnight, use
stat-report-schedule "@midnight";
See Event timestamps in WY_stat, if statistic reports appear to be generated one second prior to their scheduled time.
Statistic report is suppressed if there were no uploads since the last report.
The following example illustrates what you might get if you configured full statistic reports:
errors: 0 warnings: 2 bad signatures: 0 access violation attempts: 0 complete triplets: 6 incomplete triplets: 2 bad triplets: 0 expired triplets: 0 triplet successes: 6 files uploaded: 12 files archived: 2 symlinks created: 0 symlinks removed: 0
Each item in this report is configurable, and a unique configuration keyword is associated with it. The statistic items and their corresponding keywords are described in the table below:
errorsAny error that occurred during the run.
warningsAny warning condition occurred during the run.
bad-signaturesA PGP signature not matches the public key for the user that issued it.
access-violationsA user is attempting to upload files for some project, but it is not authorized to do so.
complete-tripletsA complete triplet is registered.
incomplete-tripletsAn incomplete triplet is registered, i.e. such that misses one or more
of its files. Notice, that a directive file alone is counted as a
complete triplet, provided that its signature verifies correctly and
that it does not contain file directive.
bad-tripletsA triplet contains files owned by different users.
expired_tripletsA triplet has expired.
triplet_successA triplet is processed successfully
uploadsAn upload is processed successfully. An upload is defined as a move of a file and its detached signature from the source to the destination directory.
archivesAn archivation is performed
symlinksA symlink is created.
rmsymlinksA symlink is removed.
There are two ways to enable statistic reports. The
built-in statistic output is enabled using the
statistics keyword.
The amount of information included in statistic report is
configured using the statistics statement. This statement takes
a list of arguments, each one being one of the keywords, described
above. For example, the following statement causes only the
information about errors and warnings to be printed:
statistics (errors, warnings);
The output produced looks like:
errors: 0 warnings: 2
A special keyword ‘none’ can be used to suppress this output altogether (which is the default), as in
statistics none;
Another special keyword is ‘all’. It enables full statistic report. This keyword may also be followed by any number of statistic item names, which are in this case excluded from the summary. For example, to output all statistic data, except errors and warnings one would set:
statistics (all, errors, warnings);
More elaborate output can be produced using the mod_logstat
loadable module. See mod_logstat, for a detailed discussion.
While running, wydawca keeps track of certain events
occurring, such as, for example, broken PGP signatures or
file uploads attempted by unauthorized users. It can issue
notifications about such events using the supplied loadable modules.
Configuration of notifications consists of two parts. First the required loadable module must be loaded and configured. Then, configure the notification itself.
A loadable module is a piece of software that provides
notification mechanism for wydawca. It is built as a UNIX
dynamically loaded library and placed in one of the preconfigured
directories which constitute a library load path. To load a
module, the following statement is used:
Load the module name from file. Other places of the configuration file can refer to the module as name.
The file argument is a file name of the module (normally, a ‘file.so’ or ‘file.la’ file).
Unless file in the ‘module’ statement is an absolute file name, it will be searched in the library load path, which is defined as:
Wydawca module directory: ‘$prefix/lib/wydawca’.
module-load-path
directive (see below).
LTDL_LIBRARY_PATH.
LD_LIBRARY_PATH).
The value of LTDL_LIBRARY_PATH and LD_LIBRARY_PATH must be a
colon-separated list of absolute directory names, for example
‘/usr/lib/mypkg:/lib/foo’.
In any of these directories, wydawca first attempts to find
and load the given filename. If this fails, it tries to append the following
suffixes to it:
The statements that modify the module search path are:
This directive adds the directories listed in its argument to the module load path. Example:
module-load-path (/usr/lib/wydawca,/usr/local/wydawca/lib);
Same as above, but the directories from list are added to the beginning of the module search list, rather than to its end. The order of directories in list is preserved in both cases.
Once loaded, the module can be initialized. This is done in the following block statement:
Initialize the module identified by name. The module must have been previously loaded using the ‘module’ statement, as described above. The statements between curly braces are module-specific configuration statements. See the module descriptions below for a detailed discussion of these.
To list module-specific configuration directives with a short usage instructions, use the --module-help statement:
wydawca --module-help=file
If the file argument is the base module name (e.g. ‘mod_mailutils’), it will be looked in the default library load path (see library search path). If it contains directory components, the file will be loaded from the specified directory.
A number of events are tracked during the execution. Any of them can be used to trigger the notification mechanism. It is configured using the following statement:
notify-event {
# Event on which to notify
event eid;
# Name of the module to invoke on event
module modname;
# Module-specific configuration data
module-config {
...
}
}
Trigger the notification when the event identified by eid occurs. The identified eid is one of the following:
Successful upload.
An unauthorized user attempted to upload files for their project.
The directive signature does not match the public key of the uploader.
The detached signature does not match the public key of the uploader.
Distribution verification failed. See verification, for a detailed description.
This event produces statistics about the recent jobs performed
by wydawca. In daemon mode, it is scheduled periodically
as controlled by the stat-report-schedule statement. In cron mode
it is emitted when all spools have been processed.
For compatibility with wydawca versions prior to 3.1.95, the
event name ‘finish’ can be used instead of ‘statistics’.
See statreports, for a detailed discussion. See also mod_logstat.
Identify the module responsible for the notification. The
modname argument must have been previously initialized in a
module statement (see modules).
This block provides module-specific configuration for modname.
Its content depends on the module used for notification. The version
4.0.3 of wydawca is shipped with two notification
modules: mod_mailutils for notifications via electronic mail,
and mod_logstat for logging the information via syslog.
These modules are described in detail later.
mod_mailutils– Mail NotificationMail notification is configured using the mod_mailutils
module. To load the module, add the following statement:
module mailutils mod_mailutils.so;
The module-init section can contain the following statements:
Set sender address for outgoing mails. E.g.:
from-address ftp-uploads@gnu.org.ua;
It is not strictly necessary to specify the sender address. In the
absence of from-address statement, the sender email will be
constructed from the name of the user wydawca runs as
(see user privileges) and the full domain name of the machine it
runs at.
Sets the admin email address or addresses. The statistic notifications and any notifications configured to be sent to admins will be forwarded to this address. The email argument is either a RFC 822 email address, or a list of such addresses. For example, the following statement configures a single admin address:
admin-address root@gnu.org.ua;
The example below illustrates how to configure multiple addresses:
admin-address "root@gnu.org.ua,ftp-adm@gnu.org.ua";
Yet another way to configure them is:
admin-address (root@gnu.org.ua, ftp-adm@gnu.org.ua);
To send messages, mod_mailutils uses a special logical entity
called a mailer. It is set in the module-init block using
the mailer keyword.
Set mailer URL.
A mailer URL consists of a scheme specification, followed by ‘://’ separator and additional data. The URLs supported by Wydawca version 4.0.3 are described in the table below. As usual, square brackets indicate optional parts:
Use an SMTP server on host to relay messages. The host part is either an IP address in dotted-quad notation or as a symbolic host name. In the latter case, DNS system is be used to resolve it. Optional port specifies port number or symbolic name (as defined in /etc/services). It defaults to 25. For example:
mailer smtp://remote.server.net:24;
Use sendmail-compatible program progname. Sendmail-compatible means that the program must be able to read an RFC-822 message from its standard input and must support the following command line options:
Do not treat ‘.’ as message terminator.
Use addr as the sender address.
Get recipient addresses from the message.
Example:
mailer sendmail:///usr/sbin/exim;
This is a special form of the ‘sendmail’ mailer. It uses the
sendmail binary from the _PATH_SENDMAIL macro in your
/usr/include/paths.h. It is the default mailer.
A prog mailer. This is a generalization of ‘sendmail’ mailer that allows to use arbitrary external programs as mailers.
The full file name of the program is given in progname part. The query part is a list of arguments, separated by ‘&’ signs. Arguments may contain the following macro-substitutions:
Expands to the sender email address.
Expands to the recipient email addresses.
The program progname must read an RFC-822 message from its standard input.
An example of ‘prog’ mailer definition:
mailer "prog:///bin/nullmail?localhost&-F${sender}&${rcpt}
When sending a mail, wydawca will invoke:
/bin/nullmail localhost -Fsender rcpt
where sender means the sender address, and rcpt stands for the recipient email address.
Equivalent to the ‘prog’ mailer, described above, but written in a more natural fashion. In this notation, the example definition above becomes:
mailer "|/bin/nullmail localhost -F${sender} ${rcpt}"
Each notification message is built from a message template, by
expanding variables (see variable expansion) within it.
The message text may be specified either in place within the
configuration directive it belongs to (see notification), or
defined by define-message statement.
Define message name to be text. This message can be
referred to from other configuration statements by @name
notation.
The message text must be formatted as a valid RFC-822 message, i.e. it must consist of two parts, message headers and body, separated by a single empty line. Therefore text is usually a here-document construct (see here-document). For example:
define-message my-message <<EOT From: Wydawca Subject: test This is a test message. EOT;
If you do not wish to supply any headers (which is unlikely, because a
mail should at least have a Subject header), simply begin the
message text with an empty line, like this:
define-message my-message <<EOT This is a test message. EOT;
The mail-statistics statement in the module-init section
for mod_mailutils configures the statistic reports
sent to the system administrator.
mail-statistics {
message text-or-id;
statistics item-list;
gpg-sign key;
}
To arrange for sending the reports, the configuration must contain the following statement:
notify-event {
event statistics;
module mailutils;
}
Define the message text. The argument is either the message text
template, or a reference to a template previously defined by a
define-message (see templates). The reference syntax is:
message @name;
where name is the message name as used in define-message.
The argument is a list of statistic item names as described in statistics. A report will be sent only if statistic counters for at least one of the requested items are not zero. For example, the following statement requires sending notifications only if there occurred any errors or access violation attempts, or any bad signature was uploaded:
statistics (errors, access-violations, bad-signatures);
If this statement is present, the message will be signed using the supplied GPG key. The key is looked up in the GPG home directory (see gpg-homedir).
The statistics message is sent to addresses configured by
admin-address statement (see admin-address).
The variables available for use in statistic reports are:
| Variable | Replaced with |
|---|---|
| date | Current date and time in the current locale. |
| stat:errors | Number of errors detected. |
| stat:warnings | Number of warnings reported. |
| stat:bad_signatures | Number of bad signatures detected. |
| stat:access_violations | Number of access violation attempts. |
| stat:complete_triplets | Number of complete triplets processed. |
| stat:incomplete_triplets | Number of incomplete triplets left in the source directory. |
| stat:bad_triplets | Number of bad triplets seen. |
| stat:expired_triplets | Number of expired triplets. |
| stat:triplet_success | Number of successfully processed triplets. |
| stat:uploads | Number of successful uploads. |
| stat:archives | Number of archivations performed. |
| stat:symlinks | Number of symbolic links created. |
| stat:rmsymlinks | Number of symbolic links removed. |
| stat:check_failures | Number of verification failures (see verification). |
An example definition of the admin notification template follows:
mail-statistics {
statistics (errors,warnings,bad_signatures,
access_violations);
message <<EOT
Subject: Wydawca stats
This is to notify you that my run on ${date}
caused the following results:
errors ............................. ${stat:errors}
warning ............................ ${stat:warnings}
bad signatures ..................... ${stat:bad_signatures}
access violation attempts .......... ${stat:access_violations}
Regards,
Wydawca
EOT;
}
module-config for mod_mailutilsWhen mod_mailutils is used in the notify-event
block, the following statements can be used in module-config
to configure it:
notify-event {
module mailutils;
# module configuration
module-config {
# Notify this recipient
recipient who;
# Sign message with this key
gpg-sign key;
# Text of the notification or identifier of a defined message
# template
message text-or-id;
}
}
Determines who should receive the notification. The following values for who are allowed:
readmessageRead recipients from the ‘To’, ‘Cc’ and ‘Bcc’ headers of the message. This is the default.
adminThe system administrator, as defined in admin-address statement
(see admin-address).
ownerAdministrators of the project for which the files where uploaded. Their addresses are retrieved from the ‘project-owner’ dictionary (see dictionaries).
userUser name of the user who uploaded files.
If this statement is present, the message will be signed using the supplied GPG key. The key is looked up in the GPG home directory (see gpg-homedir).
Define the message text. The argument is either the message text
template, or a reference to a template previously defined by a
define-message (see templates).
The following macro-variables are expanded in the message texts:
| Variable | Replaced with |
|---|---|
| project | Project system name. |
| url | URL of the distribution site. |
| spool | Name of the spool (see spool). |
| dir | Directory (relative to the project distribution root) where the files where uploaded. |
| dest-dir | Value of the destination keyword.
|
| source-dir | Value of the source keyword.
|
| triplet:dist | File name of the main distribution file. |
| triplet:sig | File name of the detached signature file. |
| triplet:dir | File name of the directive file. |
| triplet:ls:full | A full listing of the uploaded triplet8. |
| triplet:ls:upload | Listing of the uploaded files (see below). |
| triplet:ls:dist | Listing of the main distribution file (see below). |
| triplet:ls:sig | Listing of the detached signature file (see below). |
| triplet:ls:dir | Listing of the directive file (see below). |
| user | System name of the user who uploaded the triplet. |
| user:name | System name of the user who uploaded the triplet. |
| user:real-name | Real name of the user who uploaded the triplet. |
| user:email | Email of the user who uploaded the triplet. |
| email:admin | Full9. email address of the systems administrator, as set by the ‘admin-address’ (see admin-address). |
| email:owner | Full email address of the project administrator (owner). |
| email:user | Full email address of the user who did the upload. Equivalent to ‘"${user:real-name}" <${user:email}>’. |
| check:result | Code returned by external checker, in decimal. See check-result, for a detailed description. |
| check:diagn | Diagnostics text returned by external checker. See verification, for a detailed description. |
Listings referred to in the table above, are similar to those
produced by the ls command, and include information
on file permissions, ownership, size and modification date. For
example, here is a possible ${triplet:ls:full} listing:
-rw-r--r-- gray users 2707278 2007-09-06 22:14:35 tar-1.18.tar.gz -rw-r--r-- gray users 189 2007-09-06 22:14:35 tar-1.18.tar.gz.sig -rw-r--r-- gray user 62 2007-09-06 22:14:35 tar-1.18.tar.gz.directive.asc
The example in the following subsection shows how to configure success notification for the user.
This subsection provides a complete example for
mod_mailutils configuration.
module mailutils mod_mailutils.la;
module-init mailutils {
admin-address "root@example.net";
from-address "wydawca@example.net";
mailer "sendmail:";
mail-statistics {
statistics all;
message <<- EOT
Subject: upload statistics
This is to notify you that the run of wydawca on ${date}
caused the following results:
errors ............................. ${stat:errors}
warning ............................ ${stat:warnings}
bad signatures ..................... ${stat:bad_signatures}
access violation attempts .......... ${stat:access_violations}
complete triplets .................. ${stat:complete_triplets}
incomplete triplets ................ ${stat:incomplete_triplets}
bad triplets ....................... ${stat:bad_triplets}
expired triplets ................... ${stat:expired_triplets}
triplet successes .................. ${stat:triplet_success}
files uploaded ..................... ${stat:uploads}
files archived ..................... ${stat:archives}
symlinks created ................... ${stat:symlinks}
symlinks removed ................... ${stat:rmsymlinks}
verification failures .............. ${stat:check_failures}
Regards,
Wydawca
EOT;
}
}
notify-event {
event statistics;
module mailutils;
}
notify-event {
event success;
module mailutils;
module-config {
recipient user;
message <<- EOT
Subject: Upload of ${project} successful
Upload of ${project} to ${url}/${dir} finished successfully.
Files uploaded:
${triplet:ls:upload}
Regards,
Wydawca
The Project Submission Robot
EOT;
}
}
For the sake of brevity, this example defines only two
notify-event statements. More statements for others events can
be added as needed.
mod_logstat – statistics loggingThe module mod_logstat logs the supplied message at the
‘statististics’ event.
The simplest configuration for this module is:
module logstat mod_logstat.so;
notify-event {
event statistics;
module logstat;
}
This will produce on the default logging channel the detailed statistics, as discussed in statistics.
There is no specific module-init statements. The module should
be called from notify-event block on the ‘statistics’ event.
The module’s module-config statement can contain the following
statements:
Configures what statistics items should be included in the output. See statistics, for a detailed discussion of list.
This statement is ignored if the message statement is present.
Specifies the message to be logged. The text argument can contain references to statistic variables (see statistic variables).
If no message statement is present, the following default is
assumed:
message <<EOT
errors: ${stat:errors}
warnings: ${stat:warnings}
bad signatures: ${stat:bad_signatures}
access violation attempts: ${stat:access_violations}
complete triplets: ${stat:complete_triplets}
incomplete triplets: ${stat:incomplete_triplets}
bad triplets: ${stat:bad_triplets}
expired triplets: ${stat:expired_triplets}
triplet successes: ${stat:triplet_success}
files uploaded: ${stat:uploads}
files archived: ${stat:archives}
symlinks created: ${stat:symlinks}
symlinks removed: ${stat:rmsymlinks}
check failures: ${stat:check_failures}
EOT;
Support for IPv6 will be added in future versions.
See http://article.gmane.org/gmane.comp.sysutils.autotools.announce/131.
It is equivalent to:
${triplet:ls:dist}
${triplet:ls:sig}
${triplet:ls:dir}
Full here means an email address with eventual personal part
Wydawca (split by chapter):
This document was generated on January 6, 2021 using makeinfo.
Verbatim copying and distribution of this entire article is permitted in any medium, provided this notice is preserved.