MTA-STS tweaks, add status check using postfix-mta-sts-resolver, change to enforce

Author: JoshData

CHANGELOG.md

@@ -1,6 +1,14 @@
 CHANGELOG
 =========
 
+In Development
+--------------
+
+Mail:
+
+* An MTA-STS policy for incoming mail is now published (in DNS and over HTTPS) when the primary hostname and email address domain both have a signed TLS certificate installed.
+* MTA-STS reporting is enabled with reports sent to administrator@ the primary hostname.
+
 v0.45 (May 16, 2020)
 --------------------
 
@@ -24,12 +32,6 @@ Web:
 
 * Add a new hidden feature to set nginx alias in www/custom.yaml.
 
-MTA-STS:
-
-* Added support for client side MTA-STS when there is a valid SSL Certificate on the primary domain
-* Automatically adds reporting when alias "tlsrpt@<primary-domain>" is added.
-* Starts default on 'testing', but changes will be kept between MiaB Upgrades.
-
 Setup:
 
 * Improved error handling.

README.md

@@ -28,15 +28,20 @@ It is a one-click email appliance. There are no user-configurable setup options.
 
 The components installed are:
 
-* SMTP ([postfix](http://www.postfix.org/)), IMAP ([dovecot](http://dovecot.org/)), CardDAV/CalDAV ([Nextcloud](https://nextcloud.com/)), Exchange ActiveSync ([z-push](http://z-push.org/))
-* Webmail ([Roundcube](http://roundcube.net/)), static website hosting ([nginx](http://nginx.org/))
-* Spam filtering ([spamassassin](https://spamassassin.apache.org/)), greylisting ([postgrey](http://postgrey.schweikert.ch/))
-* DNS ([nsd4](https://www.nlnetlabs.nl/projects/nsd/)) with [SPF](https://en.wikipedia.org/wiki/Sender_Policy_Framework), DKIM ([OpenDKIM](http://www.opendkim.org/)), [DMARC](https://en.wikipedia.org/wiki/DMARC), [DNSSEC](https://en.wikipedia.org/wiki/DNSSEC), [DANE TLSA](https://en.wikipedia.org/wiki/DNS-based_Authentication_of_Named_Entities), and [SSHFP](https://tools.ietf.org/html/rfc4255) records automatically set
-* Backups ([duplicity](http://duplicity.nongnu.org/)), firewall ([ufw](https://launchpad.net/ufw)), intrusion protection ([fail2ban](http://www.fail2ban.org/wiki/index.php/Main_Page)), system monitoring ([munin](http://munin-monitoring.org/))
+* SMTP ([postfix](http://www.postfix.org/)), IMAP ([dovecot](http://dovecot.org/)), CardDAV/CalDAV ([Nextcloud](https://nextcloud.com/)), and Exchange ActiveSync ([z-push](http://z-push.org/)) servers
+* Webmail ([Roundcube](http://roundcube.net/)), mail filter rules (also using dovecot), email client autoconfig settings (served by [nginx](http://nginx.org/))
+* Spam filtering ([spamassassin](https://spamassassin.apache.org/)) and greylisting ([postgrey](http://postgrey.schweikert.ch/))
+* DNS ([nsd4](https://www.nlnetlabs.nl/projects/nsd/)) with [SPF](https://en.wikipedia.org/wiki/Sender_Policy_Framework), DKIM ([OpenDKIM](http://www.opendkim.org/)), [DMARC](https://en.wikipedia.org/wiki/DMARC), [DNSSEC](https://en.wikipedia.org/wiki/DNSSEC), [DANE TLSA](https://en.wikipedia.org/wiki/DNS-based_Authentication_of_Named_Entities), [MTA-STS](https://tools.ietf.org/html/rfc8461), and [SSHFP](https://tools.ietf.org/html/rfc4255) policy records automatically set
+* TLS certificates automatically provisioned [Let's Encrypt](https://letsencrypt.org/)
+* Backups ([duplicity](http://duplicity.nongnu.org/)), firewall ([ufw](https://launchpad.net/ufw)), intrusion protection ([fail2ban](http://www.fail2ban.org/wiki/index.php/Main_Page)), and basic system monitoring ([munin](http://munin-monitoring.org/))
 
-It also includes:
+It also includes system management tools:
 
-* A control panel and API for adding/removing mail users, aliases, custom DNS records, etc. and detailed system monitoring.
+* Comprehensive health monitoring that checks each day that services are running, ports are open, TLS certificates are valid, and DNS records are correct
+* A control panel for adding/removing mail users, aliases, custom DNS records, configuring backups, etc.
+* An API for all of the actions on the control panel
+
+It also supports static website hosting since the box is serving HTTPS anyway.
 
 For more information on how Mail-in-a-Box handles your privacy, see the [security details page](security.md).
 

management/dns_update.py

@@ -304,37 +304,47 @@ def has_rec(qname, rtype, prefix=None):
 		if not has_rec(qname, rtype):
 			records.append((qname, rtype, value, explanation))
 
-	# If this is a domain name that there are email addresses configured for, i.e. "something@" 
-	# this domain name, then the domain name is a MTA-STS (https://tools.ietf.org/html/rfc8461) 
-	# Policy Domain. 
+	# If this is a domain name that there are email addresses configured for, i.e. "something@"
+	# this domain name, then the domain name is a MTA-STS (https://tools.ietf.org/html/rfc8461)
+	# Policy Domain.
 	#
-	# A "_mta-sts" TXT record signals the presence of a MTA-STS policy, and an effectively random policy 
-	# ID is used to signal that a new policy may (or may not) be deployed any time the DNS is 
-	# updated. 
-	# 
-	# The policy itself is served at the "mta-sts" (no underscore) subdomain over HTTPS. The 
-	# TLS certificate used by Postfix for STARTTLS must be a valid certificate for the MX 
-	# name (PRIMARY_HOSTNAME), so we do not set an MTA-STS policy if the certificate is not 
-	# valid (e.g. because it is self-signed and a valid certificate has not yet been provisioned).
-	get_prim_cert = get_ssl_certificates(env)[env['PRIMARY_HOSTNAME']]
-	response = check_certificate(env['PRIMARY_HOSTNAME'], get_prim_cert['certificate'],get_prim_cert['private-key'])
-	
-	if response[0] == 'OK' and domain in get_mail_domains(env):
+	# A "_mta-sts" TXT record signals the presence of a MTA-STS policy, and an effectively random policy
+	# ID is used to signal that a new policy may (or may not) be deployed any time the DNS is
+	# updated.
+	#
+	# The policy itself is served at the "mta-sts" (no underscore) subdomain over HTTPS. Therefore
+	# the TLS certificate used by Postfix for STARTTLS must be a valid certificate for the MX
+	# domain name (PRIMARY_HOSTNAME) *and* the TLS certificate used by nginx for HTTPS on the mta-sts
+	# subdomain must be valid certificate for that domain. Do not set an MTA-STS policy if either
+	# certificate in use is not valid (e.g. because it is self-signed and a valid certificate has not
+	# yet been provisioned).
+	mta_sts_enabled = False
+	if domain in get_mail_domains(env):
+		# Check that PRIMARY_HOSTNAME and the mta_sts domain both have valid certificates.
+		for d in (env['PRIMARY_HOSTNAME'], "mta-sts." + domain):
+			cert = get_ssl_certificates(env).get(d)
+			if not cert:
+				break # no certificate provisioned for this domain
+			cert_status = check_certificate(d, cert['certificate'], cert['private-key'])
+			if cert_status[0] != 'OK':
+				break # certificate is not valid
+		else:
+			# 'break' was not encountered above, so both domains are good
+			mta_sts_enabled = True
+	if mta_sts_enabled:
 		mta_sts_records = [
-			("mta-sts", "A", env["PUBLIC_IP"], "Provides MTA-STS support"),
-			("mta-sts", "AAAA", env.get('PUBLIC_IPV6'), "Provides MTA-STS support"),
-			("_mta-sts", "TXT", "v=STSv1; id=%sZ" % datetime.datetime.now().strftime("%Y%m%d%H%M%S"), "Enables MTA-STS support")
+			("mta-sts", "A", env["PUBLIC_IP"], "Optional. MTA-STS Policy Host serving /.well-known/mta-sts.txt."),
+			("mta-sts", "AAAA", env.get('PUBLIC_IPV6'), "Optional. MTA-STS Policy Host serving /.well-known/mta-sts.txt."),
+			("_mta-sts", "TXT", "v=STSv1; id=%sZ" % datetime.datetime.now().strftime("%Y%m%d%H%M%S"), "Optional. Part of the MTA-STS policy for incoming mail. If set, a MTA-STS policy must also be published.")
 		]
 		# Rules can be custom configured accoring to https://tools.ietf.org/html/rfc8460.
 		# Skip if the rules below if the user has set a custom _smtp._tls record.
 		if not has_rec("_smtp._tls", "TXT", prefix="v=TLSRPTv1;"):
-			# if the alias 'tlsrpt@PRIMARY_HOSTNAME' is configured, automaticly, reporting will be enabled to this email address
-			tls_rpt_email = "tlsrpt@%s" % env['PRIMARY_HOSTNAME'] 
 			tls_rpt_string = ""
-			for alias in get_mail_aliases(env):
-				if alias[0] == tls_rpt_email: 
-					tls_rpt_string = " rua=mailto:%s" % tls_rpt_email
-			mta_sts_records.append(("_smtp._tls", "TXT", "v=TLSRPTv1;%s" % tls_rpt_string, "For reporting, add an email alias: 'tlsrpt@%s' or add a custom TXT record like 'v=TLSRPTv1; rua=mailto:[youremail]@%s' for reporting" % (env["PRIMARY_HOSTNAME"], env["PRIMARY_HOSTNAME"])))
+			tls_rpt_email = env.get("MTA_STS_TLSRPT_EMAIL", "postmaster@%s" % env['PRIMARY_HOSTNAME'])
+			if tls_rpt_email: # if a reporting address is not cleared
+				tls_rpt_string = " rua=mailto:%s" % tls_rpt_email
+			mta_sts_records.append(("_smtp._tls", "TXT", "v=TLSRPTv1;%s" % tls_rpt_string, "Optional. Enables MTA-STS reporting."))
 		for qname, rtype, value, explanation in mta_sts_records:
 			if value is None or value.strip() == "": continue # skip IPV6 if not set
 			if not has_rec(qname, rtype):

management/status_checks.py

@@ -5,11 +5,13 @@
 # what to do next.
 
 import sys, os, os.path, re, subprocess, datetime, multiprocessing.pool
+import asyncio
 
 import dns.reversename, dns.resolver
 import dateutil.parser, dateutil.tz
 import idna
 import psutil
+import postfix_mta_sts_resolver.resolver
 
 from dns_update import get_dns_zones, build_tlsa_record, get_custom_dns_config, get_secondary_dns, get_custom_dns_records
 from web_update import get_web_domains, get_domains_with_a_records
@@ -327,6 +329,11 @@ def run_domain_checks(rounded_time, env, output, pool):
 def run_domain_checks_on_domain(domain, rounded_time, env, dns_domains, dns_zonefiles, mail_domains, web_domains, domains_with_a_records):
 	output = BufferedOutput()
 
+	# When running inside Flask, the worker threads don't get a thread pool automatically.
+	# Also this method is called in a forked worker pool, so creating a new loop is probably
+	# a good idea.
+	asyncio.set_event_loop(asyncio.new_event_loop())
+
 	# we'd move this up, but this returns non-pickleable values
 	ssl_certificates = get_ssl_certificates(env)
 
@@ -611,6 +618,19 @@ def check_mail_domain(domain, env, output):
 		if mx != recommended_mx:
 			good_news += "  This configuration is non-standard.  The recommended configuration is '%s'." % (recommended_mx,)
 		output.print_ok(good_news)
+
+		# Check MTA-STS policy.
+		loop = asyncio.get_event_loop()
+		sts_resolver = postfix_mta_sts_resolver.resolver.STSResolver(loop=loop)
+		valid, policy = loop.run_until_complete(sts_resolver.resolve(domain))
+		if valid == postfix_mta_sts_resolver.resolver.STSFetchResult.VALID:
+			if policy[1].get("mx") == [env['PRIMARY_HOSTNAME']] and policy[1].get("mode") == "enforce": # policy[0] is the policyid
+				output.print_ok("MTA-STS policy is present.")
+			else:
+				output.print_error("MTA-STS policy is present but has unexpected settings. [{}]".format(policy[1]))
+		else:
+			output.print_error("MTA-STS policy is missing: {}".format(valid))
+
 	else:
 		output.print_error("""This domain's DNS MX record is incorrect. It is currently set to '%s' but should be '%s'. Mail will not
 			be delivered to this box. It may take several hours for public DNS to update after a change. This problem may result from

security.md

@@ -101,19 +101,17 @@ The box restricts the envelope sender address (also called the return path or MA
 Incoming Mail
 -------------
 
-### Encryption
-
-As discussed above, there is no way to require on-the-wire encryption of mail. When the box receives an incoming email (SMTP on port 25), it offers encryption (STARTTLS) but cannot require that senders use it because some senders may not support STARTTLS at all and other senders may support STARTTLS but not with the latest protocols/ciphers. To give senders the best chance at making use of encryption, the box offers protocols back to TLSv1 and ciphers with key lengths as low as 112 bits. Modern clients (senders) will make use of the 256-bit ciphers and Diffie-Hellman ciphers with a 2048-bit key for perfect forward secrecy, however. ([source](setup/mail-postfix.sh))
-
-### DANE
+### Encryption Settings
 
-When DNSSEC is enabled at the box's domain name's registrar, [DANE TLSA](https://en.wikipedia.org/wiki/DNS-based_Authentication_of_Named_Entities) records are automatically published in DNS. Senders supporting DANE will enforce encryption on-the-wire between them and the box --- see the section on DANE for outgoing mail above. ([source](management/dns_update.py))
+As with outbound email, there is no way to require on-the-wire encryption of incoming mail from all senders. When the box receives an incoming email (SMTP on port 25), it offers encryption (STARTTLS) but cannot require that senders use it because some senders may not support STARTTLS at all and other senders may support STARTTLS but not with the latest protocols/ciphers. To give senders the best chance at making use of encryption, the box offers protocols back to TLSv1 and ciphers with key lengths as low as 112 bits. Modern clients (senders) will make use of the 256-bit ciphers and Diffie-Hellman ciphers with a 2048-bit key for perfect forward secrecy, however. ([source](setup/mail-postfix.sh))
 
 ### MTA-STS
 
-SMTP MTA Strict Transport Security ([SMTP MTA-STS for short](https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol#SMTP_MTA_Strict_Transport_Security)). 
+The box publishes a SMTP MTA Strict Transport Security ([SMTP MTA-STS](https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol#SMTP_MTA_Strict_Transport_Security)) policy (via DNS and HTTPS) in "enforce" mode. Senders that support MTA-STS will use a secure SMTP connection. (MTA-STS tells senders to connect and expect a signed TLS certificate for the "MX" domain without permitting a fallback to an unencrypted connection.)
+
+### DANE
 
-MTA-STS is a mechanism that instructs an SMTP server that the communication with the other SMTP server MUST be encrypted and that the domain name on the certificate should match the domain in the policy. It uses a combination of DNS and HTTPS to publish a policy that tells the sending party what to do when an encrypted channel can not be negotiated.
+When DNSSEC is enabled at the box's domain name's registrar, [DANE TLSA](https://en.wikipedia.org/wiki/DNS-based_Authentication_of_Named_Entities) records are automatically published in DNS. Senders supporting DANE will enforce encryption on-the-wire between them and the box --- see the section on DANE for outgoing mail above. ([source](management/dns_update.py))
 
 ### Filters
 

setup/management.sh

@@ -50,7 +50,7 @@ hide_output $venv/bin/pip install --upgrade pip
 hide_output $venv/bin/pip install --upgrade \
 	rtyaml "email_validator>=1.0.0" "exclusiveprocess" \
 	flask dnspython python-dateutil \
-	"idna>=2.0.0" "cryptography==2.2.2" boto psutil
+	"idna>=2.0.0" "cryptography==2.2.2" boto psutil postfix-mta-sts-resolver
 
 # CONFIGURATION
 

setup/start.sh

@@ -82,14 +82,10 @@ if [ ! -f $STORAGE_ROOT/mailinabox.version ]; then
 	chown $STORAGE_USER.$STORAGE_USER $STORAGE_ROOT/mailinabox.version
 fi
 
-# Default policy (initial) for MTA_STS = testing in the current state of inclusion.
-# it can be changed to "none", "testing" or "enforce". With this extention, this is preserved by
-# future upgrades
-
-MTA_STS="${DEFAULT_MTA_STS:-testing}"
-
 # Save the global options in /etc/mailinabox.conf so that standalone
-# tools know where to look for data.
+# tools know where to look for data. The default MTA_STS_MODE setting
+# is blank unless set by an environment variable, but see web.sh for
+# how that is interpreted.
 cat > /etc/mailinabox.conf << EOF;
 STORAGE_USER=$STORAGE_USER
 STORAGE_ROOT=$STORAGE_ROOT
@@ -98,7 +94,7 @@ PUBLIC_IP=$PUBLIC_IP
 PUBLIC_IPV6=$PUBLIC_IPV6
 PRIVATE_IP=$PRIVATE_IP
 PRIVATE_IPV6=$PRIVATE_IPV6
-MTA_STS=$MTA_STS
+MTA_STS_MODE=${MTA_STS_MODE-}
 EOF
 
 # Start service configuration.

setup/web.sh

@@ -126,12 +126,13 @@ chmod a+r /var/lib/mailinabox/mozilla-autoconfig.xml
 # nginx configuration at /.well-known/mta-sts.txt
 # more documentation is available on: 
 # https://www.uriports.com/blog/mta-sts-explained/
-# default mode is "testing", which means: "Messages will be delivered as 
-# though there was no failure but a report will be sent if TLS-RPT is configured"
-# other valid modes are: "enforce" and "none".
+# default mode is "enforce". Change to "testing" which means
+# "Messages will be delivered as though there was no failure
+# but a report will be sent if TLS-RPT is configured" if you
+# are not sure you want this yet. Or "none".
 PUNY_PRIMARY_HOSTNAME=$(echo "$PRIMARY_HOSTNAME" | idn2)
 cat conf/mta-sts.txt \
-        | sed "s/MODE/$MTA_STS/" \
+        | sed "s/MODE/${MTA_STS_MODE:-enforce}/" \
         | sed "s/PRIMARY_HOSTNAME/$PUNY_PRIMARY_HOSTNAME/" \
          > /var/lib/mailinabox/mta-sts.txt
 chmod a+r /var/lib/mailinabox/mta-sts.txt