IPsec vs. SSL VPN: Comparing speed, security risks and technology

How does IPsec work?

IPsec, also known as Internet Protocol Security, defines the official architecture for securing IP network traffic. IPsec specifies ways in which IP hosts can encrypt and authenticate data being sent at the IP network layer. IPsec is used to create a secure tunnel between entities that are identified by their IP addresses. IPsec VPNs typically are used to connect a remote host with a network VPN server; the traffic sent over the public internet is encrypted between the VPN server and the remote host. IPsec enables the communicating hosts to negotiate which cryptographic algorithms are to be used to encrypt or authenticate data.

This type of tunneling enables the remote user's system -- the VPN client -- to communicate with any systems behind the VPN server. The remote host negotiates the initial connection with the VPN server, after which all traffic between the remote host and any systems inside the protected network is encrypted. Authentication of network data may be negotiated between the remote host and the VPN server in addition to, or instead of, encryption. However, VPNs use encryption to obscure all data sent between the VPN client and server.

An attacker monitoring encrypted network traffic between the VPN client and VPN server would be able to see that the two hosts are communicating and would be able to identify the traffic as being encrypted with IPsec -- but that is all.

IPsec VPNs usually require each remote endpoint to use specific software to create and manage the IPsec circuits, which means they can be more complicated to set up, configure and administer than SSL VPNs.

How does SSL work?

Modern SSL VPNs actually use TLS to encrypt streams of network data being sent between processes. The TLS protocol enables encryption and authentication of connections between programs. These connections are usually defined by the IP addresses of the endpoints, as well as the port numbers of the programs running on those endpoints. TLS enables the communicating hosts to negotiate which cryptographic algorithms are to be used to encrypt or authenticate data. While some configurations permit the use of deprecated SSL versions, security best practices recommend using only the latest versions of TLS.

SSL VPNs can be used to protect network interactions between a web browser and a web server or between an email client and an email server. SSL VPN clients access specific services by connecting to the SSL VPN server, which itself acts as a proxy for the protected services inside the enterprise network.

SSL VPNs enable more granular control over connections. Whereas an IPsec VPN enables connections between an authorized remote host and any system inside the enterprise perimeter, an SSL VPN can be configured to enable connections only between authorized remote hosts and specific services offered inside the enterprise perimeter.

Operating at the transport layer also means that an external attacker monitoring network traffic may be able to identify the application protocols being used by remote users. Instead of a single tunnel circuit carrying all network interactions as with IPsec, there will be a separate network circuit for each different connection, so the attacker might be able to gain some information about the applications and services being used by remote users. SSL VPNs can be implemented without the need to install client software on remote hosts because modern TLS-enabled browser clients can be used to create secured tunnels.

Comparing IPsec vs. SSL VPNs

The choice between an IPsec and SSL VPN should be based on the conditions and requirements of the organization. While there may be philosophical or theoretical preferences for one model or the other, the actual decision should be based on fact-based comparisons of the advantages and disadvantages as they apply to the actual deployment.

The first step in comparing IPsec vs. SSL VPNs is to determine the requirements for the organization and its users and deciding the most important features and functions of the VPN. Some of the differences between IPsec and SSL VPNs include the following:

• Performance: With modern hardware, the type of encryption used by IPsec and SSL VPNs does not usually cause performance issues, but organizations should use benchmarks to test VPN candidates. IPsec VPNs configure a tunnel between client and server using a piece of software on the client, which may require a relatively lengthy setup process; SSL VPNs that operate through web browsers will usually be capable of setting up connections much faster.
• Security: One type of VPN is not necessarily more secure in all circumstances. The most important factor in determining which type of VPN will be more secure is the threat model the organization is basing its VPN requirements on. Each VPN type should be evaluated in the context of the type of attacks the organization is defending against. Security of the encryption algorithms being used is important but so is the security of the other components of the implementation.
• Data authentication: VPNs can encrypt all transmitted data, but they can also add data authentication to defend against tampering by using strong cryptographic authentication algorithms to verify that data has not been modified in transit between VPN clients and servers. However, they do require a secure key exchange mechanism to enable authentication. While the SSL/TLS protocol incorporates negotiation of key exchange algorithms, IPsec relies on an external protocol, Internet Key Exchange, for this purpose.
• Attack defense: Attacks on IPsec VPNs and SSL VPNs -- and defense against those attacks -- will differ based on the underlying VPN protocol, implementation and added features. The key difference between IPsec and SSL VPNs lies in the difference in endpoints for each protocol. An IPsec VPN typically enables remote access to an entire network and all the devices and services offered on that network. If attackers gain access to the secured tunnel, they may be able to access anything on the private network. SSL enables connections among a device, specific systems and applications so the attack surface is more limited.
• Client security: Although the IPsec protocol is a part of the TCP/IP suite, it is not always implemented as a default component of OSes that support TCP/IP. In contrast, SSL VPNs rely on TLS, which is incorporated by default in web browsers, as well as many other application layer protocols. As a result, comparing IPsec and SSL VPNs should include consideration of how clients connect to and use the VPN, as well as how secure those options are. Implementers should consider how clients connect to the VPN, the attack surface of VPN-enabled clients and the VPN user profiles.
• VPN gateway: An SSL VPN gateway is likely to enable far more granular configuration options as far as limiting access to specific systems or services on the protected network. Gateways for IPsec VPN products are likely to have far less configurability. While they may have added packet filtering features that enable policies or configurations to limit access to specific IP addresses or subsets of the protected network, care should be taken to avoid adding unnecessary complexity and extra security risks that come with software add-ons. In either case, consider deploying a VPN alongside a network access control system that can enhance overall security by restricting access to network resources based on explicitly defined policies.
• End-to-end networking: TLS is used at the transport layer, meaning the network layer where communication is conducted between processes. In contrast, IPsec operates at the network layer where communication is conducted between network nodes with IP addresses. This makes securing end-to-end encryption more difficult when either end of the secured VPN circuit is on a network that uses Network Address Translation (NAT) to virtualize IP addresses. With an IPsec VPN, enabling secure communication across NAT gateways requires additional configuration and management.

While many of the differences between IPsec and SSL VPNs are attributable to the differences between the underlying protocols being implemented, the specific implementations should also be considered. Does the IPsec VPN implementation you prefer add features that make it functionally comparable to SSL VPN implementations? How does the implementation of either protocol VPN compare to other products from other vendors?

How to test your VPN implementations

VPN implementations should be tested with the same degree of thoroughness as any security product. Proper testing should be preceded by research about the VPN implementations being considered. Also like other security systems and services, VPN system testing should never initially be done on production systems or networks.

VPN testing should address all security aspects, especially as they pertain to the organization's threat models and attack surfaces. VPN testing should address the following:

• VPN infrastructure: This includes any VPN hardware, software and cloud applications and the way they are integrated with systems and applications to be protected. Even the best VPN can't protect against attacks on services or applications that are not secure themselves, so they should be tested as well.
• VPN cryptographic algorithms and protocols: Do the VPN components implement strong encryption protocols? Do VPN systems use up-to-date algorithms? Implementations of IPsec and TLS are sometimes slow to deprecate unsafe algorithms, which can enable some types of attack, like the Heartbleed vulnerability that made some TLS implementations vulnerable.
• VPN users: The human element is always a critical aspect of any security system. Do the people who use the VPN understand how it works? Are they able to use it securely? Do they understand the type of threats that they may face from attackers? Can the chosen VPN system withstand attacks from malicious insiders?

Ideally, enterprises would deploy both IPsec and SSL VPNs, as each one solves slightly different security issues. However, in practice, the need for complete coverage may be overbalanced by the expense of purchasing, testing, installing, administering and managing two VPN systems.