5 email security appliance comparison criteria to consider

Criterion No. 1: How advanced are basic security functions?

The basic security functions that every email security gateway performs are fundamentally the same: antivirus, antimalware, antiphishing and antispam. Buyers must be aware that this does not imply all gateways are equally effective when it comes to detecting and stopping threats.

Traditional antivirus, antimalware, antiphishing and antispam features are all less effective than they used to be because attack techniques have evolved to evade detection. Email security gateway vendors have compensated for this by integrating more advanced detection techniques into their products.

Sandboxing is one of these techniques. Sandboxing uses an isolated environment to test a file to see how it behaves when opened, executed or otherwise accessed. Using a sandbox provides a safe way to identify malware by monitoring its behavior.

Some products promote sandboxing capabilities, including Symantec Email Security.cloud and Forcepoint Email Security. The Trend Micro products also support sandboxing through an optional add-on called the Deep Discovery Analyzer, and the Sophos Email Appliance supports sandboxing with the addition of Sophos Sandstorm. Fortinet's FortiMail supports sandboxing via its FortiSandbox product.

Another advanced detection technique involves the use of threat intelligence. Threat intelligence is information about current threats -- such as IP addresses of hosts that have been attacking other hosts -- an organization can use to make better decisions about which activities to allow and which to block.

Most of the products covered in this article use threat intelligence to improve detection capabilities, or they offer add-ons that provide threat intelligence. Products that did not explicitly mention their utilization of threat intelligence include Clearswift Secure Email Gateway and Microsoft Exchange Online Protection.

For those products that offer threat intelligence, it is important to learn not only about the overall quality of the threat intelligence feature, such as what sources it comes from, but also how often the vendor updates it and how often those updates are transferred to the gateway itself. Ideally, updates should be made in near real time, such as every few minutes.

Some vendors boast support for other advanced detection techniques in addition to threat intelligence and sandboxing, but omit details of the techniques. Examples of these advanced techniques include deep content analysis, file retrospection and advanced content filtering. When performing a product evaluation, an organization should identify any such techniques that products offer and ask for more information about the details behind them.

Criterion No. 2: What additional security features are offered?

As mentioned in the first criterion, the best email security gateway products offer antivirus, antimalware, antiphishing and antispam features. Most gateways provide additional security functionality, typically in the form of data loss prevention (DLP) or email encryption.

Organizations that already have DLP and email encryption deployments may disregard this criterion in their evaluations because those deployments are likely to be more comprehensive than those provided by an email security gateway.

DLP technologies scan outbound email for sensitive information that should not be transferred via email. Examples include the organization's financial records, healthcare records and intellectual property.

All of the email security gateway products provide built-in or optional support for DLP. Optional support for Trend Micro InterScan Messaging Security is provided by its Data Privacy and Encryption Module.

Email encryption technologies are most commonly available for protecting the contents of an organization's outbound email, although some also allow email encryption between its own email accounts.

Products with built-in email encryption technologies include Cisco Email Security Appliance, Clearswift Secure Email Gateway, Sophos Email Appliance and Symantec Email Security.cloud. Optional add-ons are available for Symantec Messaging Gateway -- Content Encryption or Gateway Email Encryption -- and Trend Micro InterScan Messaging Security -- Data Privacy and Encryption Module.

Buyers should thoroughly evaluate every DLP feature or email encryption feature on its own merits before product selection. Some products offer comprehensive implementations of these features, similar to what enterprise DLP and email encryption products offer, while others provide only limited implementations that lack many features of their enterprise counterparts.

Criterion No. 3: How usable and customizable are the management features?

Usability and customizability are email security gateway characteristics that are hard to quantify. Yet they are incredibly important to consider. These two characteristics often work in opposition to each other. More usable products are often less customizable and vice versa.

Small and medium-sized organizations usually have a strong preference for usability over customization so for these businesses, customization may be largely irrelevant. For large organizations, customization may be more important to make the product as effective as possible in detecting and stopping threats.

The most important gateway feature in terms of usability, however, is having a single console to manage all the gateway instances. Ideally, this should be true even when companies deploy gateways in different computing environments, such as hybrid cloud models.

Customizability is most often noted in terms of dashboards, security policies and reporting. An organization should separately consider its needs for customization in each of these areas. For example, a particular organization might want to heavily customize a particular product's dashboard, but have no need to customize that same product's reporting capabilities.

Because organizations have their own usability and customization needs, they should view demos and perform their own testing of candidate products as part of the product evaluation process to find the best fit.

Criterion No. 4: What are typical false positive and negative rates for each detection technique?

In an ideal world, every email security gateway vendor would publish detailed statistics about its product's typical false positive and false negative rates for each type of email-borne threat. In reality however, vendors publish one or two statistics about their detection rates, and this can make it difficult to compare products based on these rates.

Most published statistics involve spam detection. Products such as Cisco Email Security Appliance, Microsoft Exchange Online Protection, Proofpoint Email Protection (cloud implementation), and Symantec Messaging Gateway and Email Security.cloud claim spam detection rates of at least 99%. Clearswift Secure Email Gateway claims to provide 99.9% spam detection. Based on these numbers, it is reasonable to expect an email security gateway to be able to achieve at least 99% spam detection.

A few products report detection rates for known viruses, typically at 100%. An organization should expect any email security gateway to have a 100% detection rate for known viruses because it is easy for the vendor to write antivirus signatures to detect them.

A final category where a few vendors provide statistics is False positive rates. Unfortunately, it's rarely clear whether false positives relate to spam only or to other categories of email, as well. But the accuracy claimed by some products is impressive. For example, the Cisco Email Security Appliance and the Symantec Messaging Gateway both claim a false positive rate of less than one in a million.

Criterion No. 5: Are email messages or attachments processed or stored externally?

This criterion may not be a concern for many organizations because their email messages and attachments may already be processed by cloud-based email services. Still, it is important for any organization to know where a third party may process or store its email, especially if it is subject to privacy regulations like the European Union's GDPR.

This isn't referring so much to cloud-based email security gateway products -- because it's obvious that they are, at a minimum, processing email in the cloud -- but, rather, products that are on premises.

Such products may, for particular situations, transfer email messages or attachments to a cloud-based service that provides more in-depth analysis to determine if their contents are malicious. This could involve a cloud-based sandbox to evaluate execution of a suspicious file.

Other products may transfer metadata only, not the contents of an email. For example, the McAfee Security for Email Servers product first conducts a local analysis of email and, if it detects a suspicious file, sends a fingerprint of the file -- not the file itself -- to McAfee Labs for additional analysis. This helps improve the detection accuracy of the product, both for the targeted organization and for other McAfee customers, without revealing the contents of email messages and attachments to McAfee.

Other email security gateway vendors do not provide information publicly on where their customers' email may be processed or stored. Any organization that is concerned about complying with the GDPR or otherwise inadvertently revealing email contents to a third party should carefully consider this criterion during their evaluation.

Finding the best email security gateway for you

Determining the best email security gateway product can be harder than most other types of security products because vendors tend to provide relatively few details about the characteristics of their products.

For example, vendors state whether their products offer DLP or email encryption capabilities, but typically few, if any, provide details on the comprehensiveness of these capabilities. False positive and false negative rates are reported incompletely, and there's no guarantee the numbers are truly comparable. At best, they are based on some measures of typical rates, which may be quite different from the rates an individual organization experiences.

Organizations should look for products that offer sandboxing and threat intelligence capabilities and a published spam detection rate of at least 99%. The Symantec Email Security.cloud, for example, claims to meet these requirements.

This does not mean other products do not have these capabilities or should not be evaluated, but rather that it may be more challenging to get the necessary information about these products to make an educated decision. The Cisco and Proofpoint products are the only two evaluated products that support all of the public cloud, hybrid cloud, local appliance and virtual appliance deployment models.

Server-based deployment is only available for McAfee Security for Email Servers (Microsoft Exchange and IBM Domino) and Trend Micro ScanMail Suite for IBM Domino and ScanMail Suite for Microsoft Exchange.

Similarly, the cloud-based Microsoft Exchange Online Protection product only supports Microsoft Exchange use. Organizations with existing Microsoft Exchange or IBM Domino implementations may want to consider these products, while other organizations can automatically exclude them from consideration because of their limited platform support.

The remaining products all support DLP and email encryption. While Clearswift Secure Email Gateway and Microsoft Exchange Online Protection did not explicitly mention their use of threat intelligence, most of the products support threat intelligence, and some support sandboxing. These include Sophos Email Appliance, Trend Micro InterScan Messaging Security and Forcepoint Email Security.

To summarize, Proofpoint Email Protection, Symantec Email Security.cloud and Trend Micro InterScan Messaging Security products support all four security capabilities -- DLP, email encryption, threat intelligence and sandboxing. The other products are missing one or more of these capabilities.

Linda Rosencrance contributed to this report.