##
##  OSSP lmtp2nntp - Mail to News Gateway
##  Copyright (c) 2001-2003 Ralf S. Engelschall <rse@engelschall.com>
##  Copyright (c) 2001-2003 The OSSP Project <http://www.ossp.org/>
##  Copyright (c) 2001-2003 Cable & Wireless Germany <http://www.cw.com/de/>
##
##  This file is part of OSSP lmtp2nntp, an LMTP speaking local
##  mailer which forwards mails as Usenet news articles via NNTP.
##  It can be found at http://www.ossp.org/pkg/tool/lmtp2nntp/.
##
##  This program is free software; you can redistribute it and/or
##  modify it under the terms of the GNU General Public  License
##  as published by the Free Software Foundation; either version
##  2.0 of the License, or (at your option) any later version.
##
##  This program is distributed in the hope that it will be useful,
##  but WITHOUT ANY WARRANTY; without even the implied warranty of
##  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
##  General Public License for more details.
##
##  You should have received a copy of the GNU General Public License
##  along with this file; if not, write to the Free Software
##  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
##  USA, or contact the OSSP project <ossp@ossp.org>.
##
##  lmtp2nntp.pod: LMTP to NNTP (manual page)
##

=pod

=head1 NAME

B<OSSP lmtp2nntp> - mail to news gateway

=head1 SYNOPSIS

B<lmtp2nntp>
[B<--childsmax|-C> I<childsmax>]
[B<--daemonize|-D>]
[B<--kill|-K>]
[B<--pidfile|-P> I<filename>]
[B<--acl|-a> I<addr>/I<mask> (LMTP daemon ACL)]
[B<--bind|-b> I<addr>[I<:port>]|C<->|I<path[:perms]> (LMTP daemon bind)]
[B<--client|-c> I<addr>[I<:port>] (NNTP client bind)]
[B<--destination|-d> I<addr>[I<:port>] (NNTP client destination)]
[B<--groupmode|-g> I<groupmode>]
[B<--headerrule|-h> [I<pri>]:[I<regex>]:I<header>:[I<val>]]
[B<--include|-i> I<configfile>]
[B<--l2spec|-l> I<l2spec>]
[B<--mailfrom|-m> I<regex>]
[B<--nodename|-n> I<name>]
[B<--operationmode|-o> I<abc/a.d.e|post|feed>]
[B<--restrictheader|-r> I<regex>]
[B<--size|-s> I<bytes>]
[B<--testfile|-t> I<filename>]
[B<--timeoutlmtp> I<sec>]
[B<--timeoutlmtpaccept> I<sec>]
[B<--timeoutlmtpread> I<sec>]
[B<--timeoutlmtpwrite> I<sec>]
[B<--timeoutnntp> I<sec>]
[B<--timeoutnntpconnect> I<sec>]
[B<--timeoutnntpread> I<sec>]
[B<--timeoutnntpwrite> I<sec>]
[B<--user|-u> I<uid|name>]
[B<--version|-v>]
I<--newsgroup> I<newsgroup|wildmat> [I<newsgroup> ...]

=head1 DESCRIPTION

The B<OSSP lmtp2nntp> program is an LMTP service for use in conjunction
with an MTA (like Sendmail), providing a reliable real-time mail to news
gateway. Input messages get their headers reformatted according to configurable rewrite rules.
The article is then posted or fed into
a remote NNTP service (like INN). Delivery must take place immediately
or the transaction fails. B<OSSP lmtp2nntp> relies on the queuing
capabilities of the MTA in order to provide a fully reliable service.
The program returns proper delivery status notification which
indicates successful completed action, persistent transient failure or
permanent failure.

Configuration files can be specified to reduce the complexity of command
lines. Their content has a simple "command WHITESPACE argument" syntax
where commands and their arguments map exactly to the long options, with
dashes omitted.  There is no artifical difference between using command
line and using configuration files. The only practical impact is that
the shell expands metacharacters while those have no special meaning
inside the configuration file.  However, shell style line concatenation
through BACKSLASH and comments through HASH are supported.

Options belong to one of the three classes SINGLE, FLAG or MULTI. Any
option can be specified more than once. Those belonging to SINGLE and
FLAG class can appear multiple times but only the last occurrence in
terms of parsing sequence counts while those belonging to MULTI class
handle their arguments as a array. SINGLE and MULTI require an argument,
FLAG does not allow an argument.

The following command line options and arguments are available:

=over 4

=item B<--childsmax|-C> I<childsmax>

Childs the daemon spawns at max. Default is 10. SINGLE.

=item B<--daemonize|-D>

Daemonize and detach from current terminal. FLAG.

=item B<--kill|-K>

Kill a previously run daemon. After processing this option the program is terminated so
this option effectivly renders most other options invalid not including
specification of a pidfile and logging. The pid must be listed in pidfile.
FLAG.

=item B<--pidfile|-P> I<filename>

Pidfile holding the process ID. Written when daemonizing. Read when killing a previously run daemon. Care
must be taken when using relative path names as daemonizing changes the
current working directory to '/' before the file is opened. SINGLE.

=item B<--acl|-a> I<addr>/I<mask> (LMTP daemon ACL)

Access control list specifying TCP INET addresses and masks where incoming
LMTP connections are accepted from.
Omitting a mask defaults to a host comparison. The mask
is a CIDR style bitmask where /0 means no comparison and enforces a match.
Omitting the wholly option defaults to 0.0.0.0/0 and [::] which allows access from
any IPv4 or IPv6 host.  It is possible to specify both inclusive and exclusive addresses,
the latter have to be prefixed with an exclamation mark. In order to pass the ACL
a client must match any inclusion and not match any exclusion. If you specify
exclusions only, a fake inclusion of 0.0.0.0/0 and [::] is appended internally.  Any
addr can be a name which will be resolved on program launch time. MULTI.

=item B<--bind|-b> I<addr>[I<:port>]|C<->|I<path[:perms]> (LMTP daemon bind)

Bind address accepting incoming LMTP connections. Supported are "C<->" for
stdio, I<path>[:perms] for Unix Domain socket with optional chmod-like
permissions and I<addr>[I<:port>] for TCP INET socket. Omitting this option
defaults to stdio. The path for a UNIX domain socket must start with a slash.
The addr can be a name which will be resolved on program launch time.
SINGLE.

=item B<--client|-c> I<addr>[I<:port>] (NNTP client bind)

Client connections for outgoing NNTP communication bind to this address. If an
address is specified but port is omitted the kernel chooses an ephemeral port.
If you want to specify a port but no address then replace address with all zeros.
If completely omitted, no assumptions are made which causes the kernel to choose
an address based on routing information and an ephemeral port.  The addr can
be a name which will be resolved on program launch time. SINGLE.

=item B<--destination|-d> I<addr>[I<:port>] (NNTP client destination)

Destination hostname or address and optional TCP port of a NNTP service.
Unless a port is specified, getserbyname(nntp) is queried with fallback to
119/tcp. If C<-d> option is ommited,
C<news> is used and if this doesn't
resolve, C<localhost> is assumed.
Any addr can be a name which will be resolved on program launch time. It is
assumed that multiple servers are used to increase the reliability of the news
system and to speed up distribution by posting the same article to more than
one server.  In regard to this program they must provide the same groups and
talk to each other. MULTI.

=item B<--groupmode|-g> I<groupmode>

Groupmode configures news group(s).
Possible values are C<arg> (default), C<envelope> and
C<header>. In C<arg> mode, the C<newsgroup>s specified as command line
arguments are ultimate destinations for the received messages.  Addresses from
envelope and headers are ignored.  In C<envelope> mode the newsgroup(s) are
taken from the LMTP envelope, in C<header> mode the newsgroup(s) are taken
from the header.  In all modes C<Newsgroups:> header is rewritten. In
C<envelope> and C<header> mode groups must still be specified as newsgroup
options. However, in these modes the newsgroup options are filters
representing allowed groups. Filters can be specified as wildmat's.
MULTI.

=item B<--headerrule|-h> [I<pri>]:[I<regex>]:I<header>:[I<val>]

Header rewriting rule. A message received by the LMTP server is split
into header and body. The header is further split into a list where the
headernames are unique keys to access single- or multivalues. The values
are kept in sequence as they appeared in the original message. The
gateway processes each rule in priority order. Small I<pri> numbers are
processed first. The default prioriy is 500.  Each rule can modify a
header and thus change the input of the remaining rules.  If a I<regex>
is given, the rule processes all currently existing headers that match
the regular expression. A new header I<header> with I<val> will be
created.  With no I<regex> given, a new header with name I<header> and
data I<val> is created.  In both cases, the new header will replace an
existing header with the same name. Also, if the name or value ever
becomes empty, the header is removed.  Omitting I<regex> and nothing
found to replace, the new header will be appended. Processed headers
will always be single-valued.  Omitting the I<val> means the (matching)
I<header>(s) will be deleted.  The headderrules support PCRE (Perl
compatible reguar expression) and if a I<regex> was used, the matching
elements are available to I<header> and I<value> as "DOLLAR NUMBER".
The I<value> is also processed through a variable expansion library
which has access to all command line options, many internal variables
and all currently existing headers. The expansion allows powerful
constructs like joining multivalues into a single value.  MULTI.

Functional diagram:

    foreach rule
        if rule has regex
            foreach header
                if regex matches
                    create new header
                    expand regex references into headername
                    if headername is empty
                        delete header
                    if headervalue is empty
                        delete header
                    expand regex references into headervalue
                    expand variables into headervalue
                    if headervalue is empty
                        delete header
                    replace existing header
        else (= rule has no regex)
            create new header
            if headervalue is empty
                delete header
            expand variables into headervalue
            replace existing or append new header

Further reading:

- regular expressions see 'pcre' manual page
- variable expansion see 'lib_var/var.pod' and 'lib_var/var_play'

Variables:

The variables use a hierarchical name space scheme. All options are
available through '${option.I<optionname>}'. For options of class MULTI
this yields the first element. Other elements can be accessed through
'${option.I<optionname>[I<index>]}'. All headers can be accessed through
'${msg.header.I<headername>}' and again, indexes are supported for
multivalued headers.

Inside variables, functions can be accessed through '%functionname'.
Currently, only %createmessageid is available which creates a value
properly suitable for a Message-ID: header.

=item B<--include|-i> I<configfile>

Include a configuration file. There is no artifical limitation on the
number and levels of configuration files supported. However, no attempt
is made to prevent recursion. Order of inclusion matters for SINGLE and
FLAG option classes. MULTI.

=item B<--l2spec|-l> I<l2spec>

L2 channel tree specification.  The full functionality of lib_l2 is
exposed to the user, see 'lib_l2/l2.pod' when it becomes available and
have a look at the example configuration file in the meantime. SINGLE.

=item B<--mailfrom|-m> I<regex>

"Mail From:" envelope restriction to limit sender addresses. If omitted, anyone can send
mail. The value to be compared includes the angle brackets. Use a PCRE (Perl
compatible reguar expression) for I<mailfrom>. SINGLE.

=item B<--nodename|-n> I<name>

Own FQDN used in LMTP and NNTP protocols. This overrides the nodename returned
by uname(3). SINGLE.

=item B<--operationmode|-o> I<abc/a.d.e|post|feed>

Set fake status or operationmode.
Possible values for I<operationmode> are C<post>, C<feed> or a string in
"LLL/D.D.D" format used to fake a LMTP return code.  In C<post> mode articles
are sent to the NNTP server(s) using POST command. Before posting, a duplicate
check using STAT command is issued. In C<feed> mode articles are sent to the
NNTP server(s) using IHAVE command.  Specifying a return code and DSN replaces
the post/ feed logic by a noop and assumes the given string must be returned
to the LMTP side.  The slash is replaced by a space internally. The default is
"553/5.7.1" meaning "Requested action not taken: mailbox name not allowed/
Delivery not authorized, message refused".  This is useful for debugging LMTP
setups without engaging NNTP.  Fake mode makes it possible to run without any
B<-d> option. However, if B<-d> option is given the NNTP client tries to
connect but it's return codes are ignored. SINGLE.

=item B<--restrictheader|-r> I<regex>

Restrict messages by header. Messages with a matching restrictheader are rejected.  If
omitted no restrictions apply. Matching is done before headers are
rewritten. Use a PCRE (Perl compatible reguar expression) for
I<regex>. SINGLE.

=item B<--size|-s> I<bytes>

Size limitation on message in bytes. Default is 8388608 (8M).

=item B<--testfile|-t> I<filename>

Testfile for headerrule. Allows debugging without LMTP or NNTP activity.
The file must be in RFC822 E-Mail format. Use this option for debugging
only, it disables both the mail server and news client.

=item B<--timeoutlmtp> I<sec>

LMTP server default timeout. Sets timeout for accept, read and write at
once.  Setting sec to zero means to wait infinite.  Note that all LMTP
timeouts only apply to socket operations, stdio always waits infinite.
SINGLE.

=item B<--timeoutlmtpaccept> I<sec>

LMTP server accept timeout.  Default is 0.SINGLE.

=item B<--timeoutlmtpread> I<sec>

LMTP server read timeout. Default is 10. SINGLE.

=item B<--timeoutlmtpwrite> I<sec>

LMTP server write timeout. Default is 10. SINGLE.

=item B<--timeoutnntp> I<sec>

NNTP client default timeout. Sets timeout for connect, read and write at
once.  Setting sec to zero means to wait infinite.  SINGLE.

=item B<--timeoutnntpconnect> I<sec>

NNTP client connect timeout. Default is 360. SINGLE.

=item B<--timeoutnntpread> I<sec>

NNTP client read timeout. Default is 60. SINGLE.

=item B<--timeoutnntpwrite> I<sec>

NNTP client write timeout. Default is 60. SINGLE.

=item B<--user|-u> I<uid|name>

User identity to be set for program execution. SINGLE.

=item B<--version|-v>

Version information is printed, then program exits. FLAG.

=item I<--newsgroup> I<newsgroup|wildmat> [I<newsgroup> ...]
=item I<newsgroup|wildmat> [I<newsgroup> ...]

Newsgroup name or match. Depending on groupmode, this is a newsgroup to
post or feed the message to or it is a wildmat filter.
Crosspostings succeed if delivery to I<any> group
succeeds. MULTI.

=back

=head1 SIGNALS

Sending a USR1 signal to the program will flush the logging stream.

=head1 DIAGNOSTICS

If invoked it returns 0 on successful execution (not neccessarily
meaning successful delivery!) or 1 on failed execution. Returning proper
delivery status notification is part of the LMTP protocol.

=head1 STANDARDS

=over 4

=item RFC0821

Simple Mail Transfer Protocol.  J. Postel. Aug-01-1982. (Format: TXT=124482
bytes) (Obsoletes RFC0788 ) (Obsoleted by RFC2821) (Also STD0010) (Status:
STANDARD )

=item RFC0822

Standard for the format of ARPA Internet text messages.  D. Crocker.
Aug-13-1982. (Format: TXT=109200 bytes) (Obsoletes RFC0733) (Obsoleted by
RFC2822) (Updated by RFC1123, RFC1138, RFC1148, RFC1327, RFC2156) (Also
STD0011) (Status: STANDARD)

=item RFC0977

Network News Transfer Protocol.  B. Kantor, P. Lapsley. Feb-01-1986. (Format:
TXT=55062 bytes) (Status: PROPOSED STANDARD)

=item RFC1035

Domain names - implementation and specification.  P.V. Mockapetris.
Nov-01-1987. (Format: TXT=125626 bytes) (Obsoletes RFC0973, RFC0882, RFC0883)
(Updated by RFC1101, RFC1183, RFC1348, RFC1876, RFC1982, RFC1995, RFC1996,
RFC2065, RFC2136, RFC2181, RFC2137, RFC2308, RFC2535, RFC2845) (Also STD0013)
(Status: STANDARD)

=item RFC1652

SMTP Service Extension for 8bit-MIMEtransport.  J. Klensin, N. Freed, M. Rose,
E. Stefferud, D. Crocker. July 1994.  (Format: TXT=11842 bytes) (Obsoletes
RFC1426) (Status: DRAFT STANDARD)

=item RFC1854

SMTP Service Extension for Command Pipelining.  N. Freed. October 1995.
(Format: TXT=14097 bytes) (Obsoleted by RFC2197) (Status: PROPOSED STANDARD)

=item RFC1893

Enhanced Mail System Status Codes.  G. Vaudreuil. January 1996. (Format:
TXT=28218 bytes) (Status: PROPOSED STANDARD)

=item RFC1894

An Extensible Message Format for Delivery Status Notifications.  K. Moore, G.
Vaudreuil. January 1996. (Format: TXT=77462 bytes) (Updated by RFC2852)
(Status: PROPOSED STANDARD)

=item RFC2034

SMTP Service Extension for Returning Enhanced Error Codes.  N. Freed. October
1996. (Format: TXT=10460 bytes) (Status: PROPOSED STANDARD)

=item RFC2606

Reserved Top Level DNS Names.  D. Eastlake, A. Panitz. June 1999. (Format:
TXT=8008 bytes) (Also BCP0032) (Status: BEST CURRENT PRACTICE)

=item RFC2821

Simple Mail Transfer Protocol.  J. Klensin, Editor. April 2001.  (Format:
TXT=192504 bytes) (Obsoletes RFC0821, RFC0974, RFC1869) (Status: PROPOSED
STANDARD)

=item RFC2980

Common NNTP Extensions.  S. Barber. October 2000. (Format: TXT=57165 bytes)
(Status: INFORMATIONAL)

=back

=cut