robots.org.uk mail system

This is a summary of how to configure an email system on a computer running Debian GNU/Linux 3.1 ("sarge").

Contents

robots.org.uk mail system

Design

The ingredients used are:

Exim: mail transport agent
PostgreSQL: database
Mailman: mailing list manager
SpamAssassin: scans messages for spam
ClamAV: scans messages for viruses, worms and other malware

TODO

This is a work-in-progress... things to change:

explain backports & volatile
remove redundant {yes}{no} from expansions
move all references to /srv/mail into database views
strip spamassassin's headers from incoming mail
dovecot
stop real_local_router from accepting mail for virtual domains
update for Debian 4.0 ("etch")
replace warn message with warn add_header in ACLs (post-etch)

Database setup

PostgreSQL configuration

Install postgresql-8.1 and postgresql-contrib-8.1 from backports.

Grant permission for the users Debian-exim and root to connect to the mail database using the maildaemon database role. In /etc/postgresql/8.1/main/pg_hba.conf, add:

local mail maildaemon ident mailmap

In /etc/postgresql/8.1/main/pg_ident.conf, add:

mailmap root maildaemon
mailmap Debian-exim maildaemon

Optional steps for the paranoid: tighten default rules in pg_hba.conf; comment out host entries, and the default ident sameuser entry, replacing it with an ident pgmap rule that only allows user postgres to connect as database user postgres.

Run /etc/init.d/postgresql-8.1 reload to bring these changes into effect.

Mail database creation

# sudo -u postgres createdb mail
# sudo -u postgres psql mail

Here are the SQL commands to set up the mail database. Use the create_account function to add users with their passwords in the correct hashed format.

Initial Exim configuration

Install exim4-daemon-heavy. Run dpkg-reconfigure exim4-config. Choose these options:

split configuration into small files
"internet site" mail configuration
system mail name: the FQDN of the default IP address assigned to the network interface from which outgoing connections will be made (usually the same as your system's hostname followed by a dot and the domain name)
ip-addresses to listen on: leave blank unless you know better
other destinations for which mail is accepted: leave blank

After making any changes to Exim's configuration, run /etc/init.d/exim4 reload.

If you use the vim editor, put # vim: filetype=exim at the end of exim's config files to have vim automatically enable syntax highlighting when you open them.

Create /etc/exim4/conf.d/main/db. If your PostgreSQL server runs on a different port, change 5432 to match.

hide pgsql_servers = (/var/run/postgresql/.s.PGSQL.5432)/mail/maildaemon/

If you are using password authentication, make sure this file is not world-redable! Also make sure that the generated /var/lib/exim4/config.autogenerated file is protected by changing CFILEMODE in /etc/exim4/update-exim4.conf to 600 and reloading Exim.

Incoming mail—"virtual" domains

The setup revolves around creating a virtual_domains domain list. The list is populated by a database lookup.

Appending virtual_domains to the local_domains domain list causes Exim to check the virtual_domains list when deciding whether a message is destined for a local domain or not. This can be done by overriding the definition of MAIN_LOCAL_DOMAINS.

In /etc/exim4/conf.d/main/00_localmacros, add:

MAIN_LOCAL_DOMAINS = DEBCONFlocal_domainsDEBCONF : +virtual_domains

Create /etc/exim4/conf.d/main/virtual:

localpartlist virtual_localparts_ignore = postmaster : abuse

domainlist virtual_domains = \
    ${lookup pgsql {SELECT DISTINCT domain FROM accounts WHERE domain = '${quote_pgsql:$domain}'}} \
    : ${lookup pgsql {SELECT DISTINCT domain FROM aliases WHERE domain = '${quote_pgsql:$domain}'}}

Create /etc/exim4/conf.d/router/345_virtual_alias:

virtual_alias:
        debug_print = "R: virtual_alias for $local_part@$domain"

        local_parts = ! +virtual_localparts_ignore
        domains = +virtual_domains

        driver = redirect
        data = ${lookup pgsql {SELECT target FROM aliases \
                WHERE "user" = '${quote_pgsql:$local_part}' \
                        AND domain = '${quote_pgsql:$domain}'}}
  
        forbid_file
        forbid_filter_existstest
        forbid_filter_logwrite
        forbid_filter_lookup
        forbid_filter_perl
        forbid_filter_readfile
        forbid_filter_readsocket
        forbid_filter_reply
        forbid_filter_run
        forbid_include
        forbid_pipe

Create /etc/exim4/conf.d/router/350_virtual_account:

virtual_account:
        debug_print = "R: virtual_account for $local_part@$domain"

        local_parts = ! +virtual_localparts_ignore
        domains = +virtual_domains

        driver = redirect
        data = ${lookup pgsql \
                {SELECT DISTINCT '/srv/mail/' || domain || '/' || "user" || '/' \
                FROM accounts \
                WHERE "user" = '${quote_pgsql:$local_part}' \
                        AND domain = '${quote_pgsql:$domain}'}}

        user = mail
        group = mail

        directory_transport = virtual_delivery

        cannot_route_message = "Unknown user $local_part@$domain"

Create /etc/exim4/conf.d/router/355_virtual_default:

virtual_default:
        debug_print = "R: virtual_default for $local_part@$domain"

        local_parts = ! +virtual_localparts_ignore
        domains = +virtual_domains

        driver = redirect
        data = ${lookup pgsql {SELECT target FROM aliases \
                WHERE "user" = '*' AND domain = '${quote_pgsql:$domain}'}}
        more = false

        forbid_file
        forbid_filter_existstest
        forbid_filter_logwrite
        forbid_filter_lookup
        forbid_filter_perl
        forbid_filter_readfile
        forbid_filter_readsocket
        forbid_filter_reply
        forbid_filter_run
        forbid_include
        forbid_pipe

Create /etc/exim4/conf.d/transport/virtual_delivery:

    virtual_delivery:
        debug_print = "T: virtual_delivery for $local_part@$domain"
        driver = appendfile
        envelope_to_add = true
        return_path_add = true
        check_string = ""
        escape_string = ""
        maildir_format

        quota = ${lookup pgsql {SELECT quota * 1024 * 1024 \
                FROM accounts \
                WHERE "user" = '${quote_pgsql:$local_part}' \
                        AND domain = '${quote_pgsql:$domain}'}}
        quota_warn_threshold = 90%
        maildir_use_size_file = true
        quota_warn_message = "\
    To: $local_part@$domain\n\
    From: postmaster@$primary_hostname\n\
    Subject: You have reached 90% of your mail quota\n\
    \n\
    This message is automatically created by mail delivery software.\n\
    \n\
    Your mailbox has reached 90% of its capacity. If you do not free\n\
    up some space by deleting old mail, you will not receive any new\n\
    mail that may be sent to you.\n\
    \n\
    Regards,\n\
    postmaster@$primary_hostname"

Create /etc/exim4/conf.d/retry/30_exim4-config:

# Bounce over-quota warnings immediatly
*                      quota

TLS (aka SSL) encryption

Put your certificate at /etc/exim4/exim.crt and your private key at /etc/exim4/exim.key. Ensure that the private key can only be read by Exim by changing its group owner to Debian-exim and its permissions to 0640.

Edit /etc/exim4/conf.d/main/00_localmacros:

MAIN_TLS_ENABLE = yes

Add information about the cipher used, the Distinguished Name of the peer communicated with, and whether the peer's certificate could be verified to Exim's mainlog:

Edit /etc/exim4/conf.d/main/02_exim4-config_options:

log_selector = +tls_certificate_verified +tls_cipher +tls_peerdn

Outgoing mail—SMTP authentication

Create /etc/exim4/conf.d/auth/postgres:

# $2 is the supplied username; $3 is the supplied password
virtual_auth_server_plain:
        driver = plaintext
        public_name = PLAIN
        server_advertise_condition = ${if eq{$tls_cipher}{}{no}{yes}}
        server_condition = ${if crypteq{$3} \
                {${lookup pgsql {SELECT password FROM exim_auth WHERE email = '${quote_pgsql:$2}'}}} \
                {yes} \
                {no}}
        server_set_id = $2

Clients can then send mail via your server, using their email addresses as their username.

Mailing lists

Mailman configuration

Install the mailman package. Read /usr/share/doc/mailman/README.Debian.gz. Disregard the paragraphs about Exim; the following files configure Exim to automatically recognise mailing lists.

Stop the newlist command from printing irrelevant info about how to configure your MTA to recognise a newly-created list. Edit /etc/mailman/mm_cfg.py:

MTA=None

Exim configuration

Create /etc/exim4/conf.d/main/mailman:

MAILMAN_HOME = /var/lib/mailman
MAILMAN_WRAP = MAILMAN_HOME/mail/mailman

MAILMAN_UID = list
MAILMAN_GID = list

MAILMAN_LOCALPART_SUFFIXES = -admin: -bounces: -confirm : -join : -leave \
        : -owner : -request : -subscribe : -unsubscribe

The mailman router is run after the various virtual routers are run. This ensures that a mailing list named foo does not clobber delivery to foo@virtual.example.com. It also means that mailing list addresses live 'in' the mail server's primary hostname. For example, the list mylist's address is mylist@hostname.example.com. If you want to create a list for a virtual domain, create a virtual alias that redirects to the list's real domain name.

Create /etc/exim4/conf.d/router/360_mailman:

mailman:
        debug_print = "R: mailman for $local_part@$domain"
        driver = accept
        require_files = MAILMAN_HOME/lists/$local_part/config.pck
        local_part_suffix_optional
        local_part_suffix = MAILMAN_LOCALPART_SUFFIXES
        transport = mailman

Create /etc/exim4/conf.d/transport/mailman:

mailman:
        debug_print = "T: mailman for $local_part@$domain"
        driver = pipe
        command = MAILMAN_WRAP \
                '${if def:local_part_suffix \
                        {${sg {$local_part_suffix} {-(\\w+)(\\+.*)?} {\$1}}} \
                        {post}}' \
                $local_part

        user = MAILMAN_UID
        group = MAILMAN_GID

        home_directory = MAILMAN_HOME
        current_directory = MAILMAN_HOME

Content scanning—antispam/malware

Install clamav-daemon from volatile.

Allow ClamAV permission to scan files in Exim's mail spool:

# adduser clamav Debian-exim

Install spamassassin from backports.

By default, spamd is configured to run as root so that it can run on behalf of any user. We don't want this; we will run it as its own user.

# adduser --system --group --home /var/run spamassassin

Edit /etc/default/spamassassin:

ENABLED=1
OPTIONS="-u spamassassin --nouser-config --max-children 3"

Edit /etc/spamassassin/local.cf:

# These are not suitable for system-wide scanning
use_auto_whitelist 0
use_bayes 0

clear_report_template
report "hits=_HITS_: _TESTSSCORES(, )_ with SpamAssassin _VERSION_ (_SUBVERSION_) on _HOSTNAME_"

Start SpamAssassin and ClamAV:

# /etc/init.d/start spamassassin
# /etc/init.d/restart clamav-daemon

Exim configuration

Edit /etc/exim4/conf.d/main/00_localmacros:

CHECK_RCPT_LOCAL_ACL_FILE = /etc/exim4/acl-rcpt
CHECK_DATA_LOCAL_ACL_FILE = /etc/exim4/acl-data

# enables sender address verification for messages from remote domains
# see acl_check_rcpt and section 39.31 of the Exim specification
CHECK_RCPT_VERIFY_SENDER = yes

Anything that recieves more than 10 points from SpamAssassin will be rejected at SMTP-time. Users can filter off mail with lower scores by matching the X-Spam-Bars header. Create /etc/exim4/conf.d/main/antispam:

# Desired values are multiplied by ten
SPAM_FLAG_SCORE = 50
SPAM_REJECT_SCORE = 100

av_scanner = clamd:/var/run/clamav/clamd.ctl

Create /etc/exim4/acl-rcpt:

# Accept messages from authorised senders *before* applying antispam tests.
#
accept
        hosts = +relay_from_hosts
accept
        authenticated = *

# Greylist hosts known to send spam and exploits
#
defer
        dnslists = dnsbl.sorbs.net : sbl-xbl.spamhaus.org : bl.spamcop.net
        condition = ${if eq {${lookup pgsql{SELECT greylist_process \
                ('${quote_pgsql:$sender_host_address}', \
                '${quote_pgsql:$sender_address}', \
                '${quote_pgsql:$local_part@$domain}')}}} {t} {true}{false}}
        message = Please try later
        log_message = greylisted

Create /etc/exim4/acl-data:

# malware
deny message = Message contains malware ($malware_name)
        demime = *
        malware = *

# spam
deny message = Message classified as spam ($spam_score points)
        spam = mail:true
        condition = ${if >{$spam_score_int}{SPAM_REJECT_SCORE}}

warn message = X-Spam-Bars: $spam_bar
        spam = mail:true

warn message = X-Spam-Report: $spam_report
        spam = mail:true

Vacation

Used for out-of-office messages, etc. The vacation auto-reply won't be sent more than once every three days.

Create /etc/exim4/conf.d/router/347_virtual_vacation:

virtual_vacation:
        debug_print = "R: virtual_vacation for $local_part@$domain"

        local_parts = ! +virtual_localparts_ignore
        domains = +virtual_domains

        driver = accept
        condition = ${lookup pgsql {SELECT DISTINCT message FROM vacation WHERE "user" = \
                '${quote_pgsql:$local_part}' AND domain = '${quote_pgsql:$domain}'}}

        transport = virtual_vacation
        unseen

Create /etc/exim4/conf.d/transport/virtual_vacation:

virtual_vacation:
        debug_print = "T: virtual_vacation for $local_part@$domain"

        driver = autoreply
        user = mail
        group = mail

        return_message
        once = /srv/mail/$domain/$local_part/vacation.dbm
        once_repeat = 3d

        from = $local_part@$domain
        to = $sender_address
        subject = Re: $h_subject
        text = ${lookup pgsql{SELECT message FROM vacation \
                WHERE "user" = '${quote_pgsql:$local_part}' \
                AND domain = '${quote_pgsql:$domain}'}}

Subaddressing

We redirect mail for sam+foo@example.com to sam@example.com. This makes it easy for users to create arbitrary mailboxes on the fly. The original destination address is preserved in the message's To header, so users can use it for filtering, or to see which sites sold their email addresses to spammers, etc.

Create /etc/exim4/conf.d/router/305_subaddress:

subaddress:
        debug_print = "R: subaddress for $local_part$local_part_suffix@$domain"

        driver = redirect
        data = $local_part@$domain

        local_part_suffix = +*

Mail for system users

We want mail for system users to be directed to root's mailbox. We do this so that such mail does not languish unread in /var/mail for years.

The Debian Policy Manual defines a system user as one whose UID is outside of the range 1000‒29999.

Create /etc/exim4/conf.d/router/405_system_user:

# Redirect messages for users with a uid outside of the range 1000 - 29999 to
# root. The range is defined in Policy section 9.2.2:
# <http://www.debian.org/doc/debian-policy/ch-opersys.html#s9.2.2>
system_user:
        debug_print = "R: system_user for $local_part@$domain [$local_user_uid]"
        driver = redirect

        domains = +local_domains
        check_local_user
        data = root
        condition = ${if or {{< {$local_user_uid} {1000}} {> {$local_user_uid} {29999}}}}

Testing

Fake SMTP session from specified IP address: exim4 -bh 1.2.3.4

Address testing ("what will Exim do with this address?"): exim4 -bt

Configuration options testing ("what is the effective value of an option?: exim4 -bP

robots.org.uk mail system

Design

TODO

Database setup

PostgreSQL configuration

Mail database creation

Initial Exim configuration

Incoming mail—"virtual" domains

TLS (aka SSL) encryption

Outgoing mail—SMTP authentication

Mailing lists

Mailman configuration

Exim configuration

Content scanning—antispam/malware

Exim configuration

Vacation

Subaddressing

Mail for system users

Testing

Sources