Go multilingual with Hugo

Language icon.

I have made my small company site multilingual, in Swedish and in English. It’s a static site built with Hugo and my own Zen theme for Hugo.

Hugo has good multilingual support. What took the most time was building a language selector that works both when content is translated and when it is not.

Configure the languages

In the “config.yml” file I have Swedish set as the default language and configure the two languages I want, “sv” and “en”. The “LanguageName” parameter is used in the language selector.

defaultContentLanguage: "sv"

languages:
  sv:
    weight: 1
    params:
      LanguageName: "Svenska"
  en:
    weight: 2
    params:
      LanguageName: "English"

Translate the content

Duplicate each page you want to translate and add the language code to the file name. The “about.md” page gets a English version with “about.en.md”. All the content with “.en.” in the name will get generated in a “en” subdirectory inside “public”.

Simple and straight forward solution by Hugo I think.

Theme i18n files

My zen theme already has i18n files for Swedish and English, if you need another language add it to the sites “i18n” directory. These files has translations for terms in the themes templates.

Language selector

I wanted a language selector that would switch to the translated version of the current page if it existed and to the front page of the language if not.

I solve this by checking if “.IsTranslated” is set. If it is, the code iterate over all the available translations and link to them. If not, it iterate over all the available site languages and link to the front page for that language.

<h2 class="visually-hidden">{{ i18n "lang_select_title" }}</h2>
<nav class="language-selector layout__language-selector" role="navigation">
<ul class="navbar">
{{ if .IsTranslated -}}
{{ range .Translations }}
<li><a rel="alternate" href="{{ .RelPermalink }}" hreflang="{{ .Lang }}" lang="{{ .Lang }}">{{ .Site.Language.Params.LanguageName }}</a></li>
{{ end -}}
{{ else -}}
{{ range .Site.Languages -}}
{{ if ne $.Site.Language.Lang .Lang }}
<li><a rel="alternate" href="/{{ .Lang }}" hreflang="{{ .Lang }}" lang="{{ .Lang }}">{{ .Params.LanguageName }}</a></li>
{{ end -}}
{{ end -}}
{{ end -}}
</ul>
</nav>

It’s common to use country flags for a language selector but language is not the same thing as country so I went looking for a better icon.

I found the Language Icon and it works well I think, looks nice and I believe a lot of people can guess what it’s for. I also output the language names in their own language. Do not demand that everyone know English just to switch to their language of choice.

This language selector is part of the Zen theme for Hugo.

The result

Take a look at my company site for the result.

That site only has two languages but it works equally well with as many languages as you need.

Run your own mail server with Postfix and Dovecot

Ansible role with commentary for setting up your own mail server with Postfix and Dovecot.

This could be considered a part two of Mail relay, MX backup and spam filtering with Postfix. Many postfix configurations are identical between these setups.

Updated 2018-05-18: Added more configuration files and instructions how to use the Ansible role.

Why

Anyone can set up their own mail server and start exchanging e-mail with every other Internet user in the world. This is quite amazing I think.

So many things on the Internet today is controlled by a handful of tech giants. E-mail is something you can and should control yourself. It’s a bit complex to setup but done right it’s stable and low maintenance.

I have run my own mail servers for well over a decade. The setup I describe below has with minor changes been running in production since 2013. I recently upgraded them to Debian 9 Stretch and in early 2016 I started using Letsencrypt certs.

I host domains for my company and my family so mail between us are reasonably privat since all traffic uses TLS. E-mail is not a secure way to communicate but with your own server your mail is at least not used to target you for ads and what not.

To get started

Don’t be a cargo cult sysadmin, read the documentation.

Ansible mailserver role

Complete configurations can be found in my Ansible mailserver role at frjo/ansible-roles on Github. This is what I use to set up my own servers.

The common role that set up a firewall and other essentials on all my servers, the letsencrypt role for free certs and the dbserver role are also on GitHub. At the moment you will need to setup a web server with PHP support yourself, or take a look at Running Drupal on Debian 9 with Apache 2.4, HTTP/2, event MPM and PHP-FPM (via socks and proxy).

More on how I use Ansible can be found in my post My first 2 minutes on a server - letting Ansible do the work.

What you will get

  • Mail server with (almost) only standard Debian 9 packages so easy to keep updated via apt.
  • Virtual domains, mailboxes and aliases stored in MariaDB (MySQL but better).
  • Postfix for SMTP with opportunistic TLS, SPF and Postscreen configured.
  • Dovecot for IMAP/POP with required TLS.
  • PostfixAdmin - web based administration interface for Postfix mail servers. (the only non Debian package)
  • Spam filtering with DNSBL Spamhaus ZEN and BarracudaCentral.
  • Support for address extensions, user+whatever@example.com addresses.
  • Striping of outgoing mail headers that reveal unneeded information like users IP address, mail client etc.

What you will not get

  • Webmail, see no use for that now that most people have a smartphone with a e-mail client built in.
  • Bayesian filtering or other text filtering systems. I believe this belongs on the client side.
  • DKIM/DMARK, I find them cumbersome and of no benefit.

How to use the Ansible role

If you are not already using Ansible take a look at Getting Started with Ansible.

Step 1: Set up the directory structure and files needed.

ansible-playbooks/:
    mail-server-playbook.yml
    host_vars/
        your.host.yml
    roles/
        common/
        dbserver/
        letsencrypt/
        mailserver/
    vars/
        passwords.yml

Step 2: Set up a playbook, above I named it “mail-server-playbook.yml”.

---
- name: Apply configuration to mail server
  hosts:
    - your.host
  remote_user: root
  port: 2222
  roles:
    - common
    - dbserver
    - mailserver
  vars_files:
    - vars/passwords.yml

Step 3: In “host_vars/your.host.yml” you add all the variables needed. The roles have sensible defaults (see “defaults/main.yml” in each role) but some things need to be set for each host.

---
acme_certs_dir: "/etc/ssl/letsencrypt"
acme_domains:
  - your.host
  - www.your.host
  - other.your.host
acme_services:
  - apache2
  - dovecot
  - postfix
openports_list:
  - 25
  - 80
  - 443
  - 587
  - 993
  - 995
  - 2222
openports_udp_list:
  - "60000:60100"

Step 4: The “vars/passwords.yml” file is a good place to keep all the passwords needed. I recommend using the “ansible-vault” command to have it encrypted.

---
ssl_key_passwd: a-good-password
db_root_passwd: a-good-password
db_backup_passwd: a-good-password
postfix_db_passwd: a-good-password
postfix_admin_db_passwd: a-good-password
acme_account_key_passwd: a-good-password

Step 5: Run the playbook.

$ ansible-playbook mail-server-playbook.yml

DNS - get this right and good things will follow

Make sure the servers IP address is not blacklisted. It need to be a static address in good standing or your mail will get marked as junk.

The DNS record should look something like this. Please do not forget to set a valid PTF (pointer) record. In best case it should be the reverse of the A record but it must exist and be a valid address for the server.

mail.example.com.	3600	IN	A	123.4.5.6
6.5.4.123.in-addr.arpa.	3600	IN	PTR	mail.example.com.

With the MX record you tell all other mail servers what server handle the mail for your domain.

example.com.		3600	IN	MX	10 mail.example.com.

If you set up some mail relay servers as I have done your MX record might look like this.

example.com.		3600	IN	MX	10 mx1.example.com.
example.com.		3600	IN	MX	20 mx2.example.com.

The SPF record tells other mail servers what servers are allowed to send mail for your domain.

The following is what I often use. It says that servers with a A or MX record for the domain is valid but none other. If you are using other external services to send e.g. news letters you need to add them as well.

example.com.		3600	IN	TXT	"v=spf1 mx a -all"

If you run your web server on a separate host and have the MX records pointed at mail relays you can explicitly add the mail servers A record like this.

example.com.		3600	IN	TXT	"v=spf1 a a:mail.example.com mx -all"

If you like some of my customers use Mailgun (or other services) to send out mail remember to add them to your SPF record. For Mailgun it looks like this.

example.com.		3600	IN	TXT	"v=spf1 a mx include:mailgun.org -all"

Dovecot

Postfix handles the sending and receiving of mail. Dovecot is what users, or rather their mail client of choice, connect to when they want to read the mail.

Dovecot is the most standard compliant IMAP server and it just works. I used Courier for the first few years but never looked back after switching to Dovecot.

It supports the IDLE command so mail will arrive almost instantaneously on desktop mail clients that support it. Push support for iOS would be really nice to have and it’s possible to get working but it’s not straight forward.

Dovecot configuration consist mostly of making it talk with Postfix and MariaDB for user authentication. Postfix will handle all authentication via Dovecot.

auth_cache_size = 1M
auth_cache_ttl = 1 hour
auth_cache_negative_ttl = 1 hour

disable_plaintext_auth = yes
auth_mechanisms = plain login

mail_uid = vmail
mail_gid = vmail

mail_location = maildir:/var/spool/vmail/%d/%n

default_process_limit = 150

ssl = required

ssl_protocols = !SSLv3

{% if acme_certs_dir_check.stat.isdir is defined %}
ssl_cert = <{{ acme_certs_dir }}/{{ acme_domains[0] }}/fullchain.pem
ssl_key = <{{ acme_certs_dir }}/domain.key
{% endif %}

protocol imap {
  mail_max_userip_connections = 30
}

namespace inbox {
  separator = .
  inbox = yes

  mailbox Drafts {
    special_use = \Drafts
  }
  mailbox Junk {
    special_use = \Junk
  }
  mailbox Sent {
    special_use = \Sent
  }
  mailbox "Sent Messages" {
    special_use = \Sent
  }
  mailbox Trash {
    special_use = \Trash
  }
}

passdb {
  driver = sql
  args = /etc/dovecot/local-sql.conf
}

userdb {
  driver = static
  args = uid=vmail gid=vmail home=/var/spool/vmail/%d/%n
}

service imap-login {
  # Number of processes to always keep waiting for more connections.
  process_min_avail = 2
}

service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
    mode = 0666
    group = postfix
    user = postfix
  }

  user = vmail
}

service auth {
  unix_listener auth-userdb {
    mode = 0666
    user = vmail
    group = vmail
  }

  # Postfix smtp-auth
  unix_listener /var/spool/postfix/private/auth {
    mode = 0666
    user = postfix
    group = postfix
  }

  # Auth process is run as this user.
  user = $default_internal_user
}

service auth-worker {
  user = $default_internal_user
}
driver = mysql
connect = host=localhost dbname=postfix user=postfix password={{ postfix_db_passwd }}
default_pass_scheme = SHA512-CRYPT
password_query = SELECT username AS user, password FROM mailbox WHERE username = '%u' AND active = '1'

Postfix

A lot of the Postfix configuration is identical to the Mail Relay setup, see article link above. What follows is a quick overview of some Postfix configurations specific for the mail server.

The Ansible role will set up all this but it’s good to understand what it does and why.

Let Postfix in chroot jail access MariaDB sock

On Debian Postfix is by default set up to run in a chroot environment. This provides a significant barrier against intrusion. It also creates a stumbling block since it stops Postfix from accessing files outside the chroot jail.

By mounting /var/run/mysqld in /var/spool/postfix/var/run/mysqld we allow the postfix processes to access the MariaDB sock and all is well.

Authenticate via Dovecot

This will make Postfix authenticate users via Dovecot. Users mail clients will connect directly to Postfix for sending mail and we need them to authenticate.

# SASL settings
smtpd_sasl_auth_enable = yes
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_authenticated_header = yes

Virtual domains, mailboxes and aliases stored in MariaDB

This makes it possible to host multiple domains on one single mail server. Postfix will look up mailboxes and aliases in the database and deliver valid mail to dovecot or send onward if an alias point to an external address.

All local mail will be stored in the /var/spool/vmail directory belonging to the vmail user.

virtual_minimum_uid = 5000
virtual_gid_maps = static:5000
virtual_uid_maps = static:5000
virtual_mailbox_base = /var/spool/vmail
virtual_alias_domains =
virtual_alias_maps = proxy:mysql:/etc/postfix/mysql_virtual_alias_maps.cf
virtual_mailbox_domains = proxy:mysql:/etc/postfix/mysql_virtual_domains_maps.cf
virtual_mailbox_maps = proxy:mysql:/etc/postfix/mysql_virtual_mailbox_maps.cf
virtual_transport = dovecot
dovecot_destination_recipient_limit = 1
default_destination_concurrency_limit = 5
relay_destination_concurrency_limit = 1

Administer with PostfixAdmin

PostfixAdmin is a web interface written in PHP. It’s simple but works well for administering mailboxes and aliases for multiple domains.

The playbook is not using the latest version since I had trouble getting that to work. Since the older version works without issues I have not put a lot of time investigating the problem.

To be on the safe side I put it behind an Apache basic auth protection.

<?php

$CONF['configured'] = true;
$CONF['database_type'] = 'mysqli';
$CONF['database_host'] = 'localhost';
$CONF['database_user'] = $_SERVER['DB_USER'];
$CONF['database_password'] = $_SERVER['DB_PASS'];
$CONF['database_name'] = 'postfix';
$CONF['admin_email'] = 'postmaster@{{ ansible_domain }}';
$CONF['encrypt'] = 'dovecot:SHA512-CRYPT';
$CONF['dovecotpw'] = "/usr/bin/doveadm pw";
$CONF['password_validation'] = array(
#    '/regular expression/' => '$PALANG key (optional: + parameter)',
    '/.{14}/'               => 'password_too_short 15',     # minimum length 14 characters
    '/([a-zA-Z].*){4}/'     => 'password_no_characters 4',  # must contain at least 4 characters
);
$CONF['generate_password'] = 'YES';
$CONF['page_size'] = '50';
$CONF['default_aliases'] = array (
    'abuse' => 'abuse@{{ ansible_domain }}',
    'hostmaster' => 'hostmaster@{{ ansible_domain }}',
    'postmaster' => 'postmaster@{{ ansible_domain }}',
    'webmaster' => 'webmaster@{{ ansible_domain }}'
);
$CONF['domain_path'] = 'YES';
$CONF['domain_in_mailbox'] = 'NO';
$CONF['aliases'] = '100';
$CONF['mailboxes'] = '100';
$CONF['maxquota'] = '1024';
$CONF['domain_quota_default'] = '10240';
$CONF['alias_domain'] = 'NO';
$CONF['fetchmail'] = 'NO';
$CONF['show_footer_text'] = 'NO';
$CONF['show_undeliverable']='NO';
$CONF['show_status']='NO';
$CONF['welcome_text'] = <<<EOM
Hi,

Welcome to your new account at {{ ansible_domain }}.
EOM;

Removing headers on outgoing mail for security reasons

With smtp_header_checks you can manipulate the headers of mail sent out from the server by users. I use it to remove some headers for security reasons.

I remove X-Originating-IP, information about the mail client and the received header. This way the e-mail will appear to originate from the mail server itself and not reveal unnecessary information about the sending device/user.

# /etc/postfix/smtp_header_checks
#
# /^HEADER:.*content_to_act_on/ ACTION [MESSAGE]

/^Received:/                 IGNORE
/^User-Agent:/               IGNORE
/^X-Mailer:/                 IGNORE
/^X-Originating-IP:/         IGNORE

Further reading

Ars Technica has a four part article series “Taking e-mail back” that has a lot of good information.

Other good articles.

Mail relay, MX backup and spam filtering with Postfix

If you run your own mail server it is a good idea to have a MX backup in place. When your mail server goes down or you need to upgrade it, the MX backup will step in and store all mail until the mail server is back up.

A pure MX backup server is bit of a waste so I set mine up to act as a mail relay with spam filtering. Having two of them in separate locations seems like a good idea.

The setup I describe below has with minor changes been running in production since 2014. I recently upgraded them to Debian 9 Stretch and in early 2016 I started using Letsencrypt certs.

The servers handle mail for my own domains, for our company and for a number of customers. In total they deliver a bit over a thousand mail and discarded tens of thousands of spam per day. They do this with very little server resources.

Ansible mailrelay role

Complete configurations can be found in my mailrelay role at frjo/ansible-roles on Github. This is what I use to set up my own servers.

The common role that set up a firewall and other essentials on all my servers and the letsencrypt role for free certs are also on GitHub. If you use them plus the mailrelay role on a Debian 9 Stretch host you will get servers identical to mine.

Postfix

I have been using Postfix for more than ten years. It might not be the easiest software to set up but mail in general is a bit complex. Postfix has proven to be extremely stable and reliable, actively maintained, well documented and with a good set of features.

The only separate software used in this setup is pypolicyd-spf, a Postfix policy engine for SPF. I have used it for years and it is as stable and good as Postfix itself.

Two external DNSBL services are used as well.

This makes this setup plain and simple, and very stable. Besides upgrading packages when needed they require almost no maintenance.

Relaying Mail and MX backup

First make sure the server is not an open relay, it would allow anyone sending mail through the server.

smtpd_relay_restrictions =
  permit_mynetworks,
  reject_unauth_destination

The “reject_unauth_destination” is the vital part.

The following tells postfix what mail to relay and where.

relay_domains = hash:/etc/postfix/transport
transport_maps = hash:/etc/postfix/transport
relay_recipient_maps = hash:/etc/postfix/relay_recipients

In the “transport” file set up each domain and where it should be relayed. This file is also used for the “relay_domains” parameter that will only read the first column.

# /etc/postfix/transport
# run  "postmap  /etc/postfix/transport"  after each edit

example.com    smtp:mail.example.com:25

List all recipients that should be relayed in the “relay_recipients” file. Easiest is to simply list a domain and accept all addresses for it.

By instead specifying each real address the mail relay server can discard mail to non existing users directly. Then the list will however need to be updated when you add/remove addresses/mailboxes on your mail server.

# /etc/postfix/relay_recipients
# run  "postmap  /etc/postfix/relay_recipients"  after each edit

@example.com        OK

info@example.com    OK
joe@example.com     OK

Some other good settings include the maximum message size that I set to 25 MB, same as Gmail. The queue lifetime decides how long the server will keep trying to send mail. I set this to 10 days, this gives ample time to get a mail server up and running again.

message_size_limit = 25600000
maximal_queue_lifetime = 10d

See Postfix Configuration Parameters for a detailed explanation of all the parameters.

Spam filtering

With some basic spam filtering techniques around 98 percent of incoming mail are discarded with near zero false positives.

My biggest problem is with Swedish spam. For some inexplicable reason it is legal for a business to spam another business in Sweden. Mail services sending out what I consider spam is also sending out legit news letters etc. so one can not outright block them. It is a mess.

By just checking good DNSBL (DNS-based Blackhole List) 95 percent of incoming traffic can be discarded. I use Spamhaus ZEN and BarracudaCentral, they work really well and are solid services. Good DNSBL are by far the most important part of any spam filtering system.

Since I have paying customers on my servers I use the paid version of Spamhaus ZEN from Spamhaus Technology. Minimum cost is 250 USD for up to 355 mail users, a small cost for all the spam it stops.

Postscreen

Postscreen is a fast and light weight process included in Postfix 2.8 and later. It can filter out spam with the help of DNSBL before handing the rest over to the smtp server process. I highly recommend using it, it will keep server loads to a minimal.

# Postscreen
postscreen_greet_action = enforce
postscreen_dnsbl_action = enforce
postscreen_blacklist_action = enforce
postscreen_access_list = permit_mynetworks, cidr:/etc/postfix/client_access.cidr
postscreen_dnsbl_sites = zen.spamhaus.org, b.barracudacentral.org
postscreen_cache_map = proxy:btree:$data_directory/postscreen_cache

The client_access.cidr file can be used to whitelist (or blacklist) IP addresses. On my mail server I e.g. whitelist the IP addresses of the mail relay servers.

Filter bad attachments

With Postfix mime_header_checks you can easily reject mail with known bad file types. I e.g. block mails with bat|com|exe|jar|pif|scr|swf extensions, this is mostly to protect users with older versions of Windows.

# Filter on content in mime headers
mime_header_checks = pcre:/etc/postfix/mime_header_checks

Smtpd recipient restrictions

Most of the remaining spam gets discarded here. The two important parts are the SPF check and reject_unknown_reverse_client_hostname. Some guides suggest reject_unknown_client_hostname but I get to many false positive with that. There are a surprising number of mail operators that miss to set up proper PTR records. Do not make the same mistake for your own servers!

# Requirement for the recipient address.
smtpd_recipient_restrictions =
  reject_unauth_pipelining,
  reject_non_fqdn_recipient,
  reject_unknown_recipient_domain,
  permit_mynetworks,
  check_client_access cidr:/etc/postfix/client_access.cidr,
  reject_unknown_reverse_client_hostname,
  reject_rbl_client zen.spamhaus.org=127.0.0.10,
  reject_rbl_client zen.spamhaus.org=127.0.0.11,
  reject_rbl_client zen.spamhaus.org,
  reject_rbl_client b.barracudacentral.org,
  reject_unlisted_recipient,
  check_policy_service unix:private/policyd-spf,
  permit

See Postfix Configuration Parameters for a detailed explanation of all the parameters.

Bayesian filtering and other text filtering systems

I do not do this on any of my servers.

It complicates things and make false positives more likely. Better to do this on the client side and most mail clients today have good built in support for this. If you are on macOS I highly recommend SpamSieve.

MX records

With the servers up and running you need to reconfigure the DNS and make MX records for the new servers.

With this MX setup in the DNS:

example.com.  MX  10  mx1.example.com.
example.com.  MX  20  mx2.example.com.

Incoming mail flows like this:

                    mx1.example.com ->
Incoming mail ->                        mail.example.com
                    mx2.example.com ->

If you are in Sweden and do not want to run your own mail relay and mx backup servers please have a look at my service Mail relay, MX backup och skräppostfilter. All the servers are in Sweden, one of the reasons I set them up to begin with.

A big commercial service that seems to have good reputation is Mail route. I have not used them myself. They have more advanced filtering and a web interface to handle settings, false positives etc.

Blog archive