What is MTA-STS and how will it improve email security?

The SMTP MTA Strict Transport Security specification is an internet standard that improves the security of connections between SMTP servers. When the Simple Mail Transfer Protocol was first specified in 1982, it did not provide any security at the transport layer to protect communication between mail transfer agents, but the STARTTLS command was added to SMTP in 1999 to support opportunistic encryption of email between servers. It provided the ability to upgrade an insecure connection to a secure one that is encrypted using the Transport Layer Security protocol.

But STARTTLS has two problems: The first one is that it's optional. A man-in-the-middle attacker can modify a connection and prevent the encryption update from happening. The second problem is that, even if STARTTLS is being used, there is no way to authenticate the sending server's identity because SMTP mail servers will not validate certificates.

The DNS-Based Authentication of Named Entities (DANE) protocol was an early attempt to fix this problem. DANE relies on Domain Name System Security Extensions (DNSSec), which provide a method to sign DNS records. However, DNSSEC is controversial in the security community and was never widely adopted. Therefore, DANE, which relies on DNSSEC, only secures a small fraction of mail server connections. In particular, none of the big players, like Google, Yahoo or Microsoft, ever adopted DNSSEC or DANE.

The MTA-STS protocol is implemented by having a DNS record that specifies that a mail server can fetch a policy file from a defined subdomain. This policy will then be fetched via HTTPS, authenticated and list the names of the recipients' mail servers. These names are also authenticated with certificates. Implementing MTA-STS is relatively simple on the recipient side, but on the sending side, it requires support in the mail server software. Some mail servers already include this support. For example, the widely used Postfix mail server can be retrofitted with an external module.

The MTA-STS standard has widespread support among major mail service providers. The authors include members from Microsoft, Oath and Google. Google's Gmail already sets and validates MTA-STS policies.

MTA-STS fixes a long-standing gap in email connection security. While connections from the user to the mail server are usually encrypted with Transport Layer Security these days, the connections between servers remained a weak spot. With support from major mail providers, it is likely that the majority of mail connections will be encrypted soon.

That still does not mean the mail content is encrypted all the time. An attacker with access to the mail server itself can still read the content -- that can only be prevented with end-to-end encryption methods, like Pretty Good Privacy. But MTA-STS is effective at preventing attackers on the networks from reading email content, which is already a huge improvement.