VPN Protocols – Explained

In a world of confusing internet and technology jargon, new or beginner users just starting their journey into the world of VPNs and online security can be forgiven for not knowing each of the VPN protocols that are currently available.  

We’ll cover most of the commonly-used protocols as well as a number of others that are in use today as well as provide a clear but detailed breakdown of what each offers and also what potential drawbacks they have.  

Let’s dive in.  

VPN vs. Protocol

Before we get started on the different types of protocols available, it’s important to understand what a VPN protocol is and how it differs from a VPN service.  

Put simply, a VPN service, such as ExpressVPN or NordVPN, allows a user to choose which VPN protocol they require from a list of available options depending on things like speeds and security and then facilitates the transfer of data using the chosen protocol through an encrypted VPN tunnel.  

The protocol itself is a system of programs and processes that establishes a secure connection, determines how the tunnel is formed, what level of encryption, and how quickly the data is transferred through the tunnel. This is done by verifying the authenticity of the user’s device and the VPN server, which then generates an encryption key that can be used by both. 

VPN protocols mainly operate in one of two ways.  

1) The first being two protocols are used with one being used in the transfer of data through the tunnel and the other to secure that traffic.  

2) The other being only one protocol is used for the transfer of data as well as security.  

Each protocol presents different strengths and weaknesses and should be paired with the task you wish to accomplish.  

For example, if you want the added security of VPN when you’re using public WIFI (you absolutely should) then you wouldn’t need to choose a protocol such as OpenVPN UDP with a low-latency transmission of data. Similarly, if you’re an investigative journalist dealing with sensitive and potentially damaging information then you would opt for the highest levels of security available.  

Now we’ve covered the basics let’s take a look at the most common VPN protocols.  

PPTP

Point-to-Point tunneling protocol first came into existence in the mid-1990s and was developed by Microsoft and a number of others as an upgrade to its predecessor PPP. Initially designed for dial-up connections, it quickly encountered a number of issues, mostly related to the underlying PPP authentication protocols used.  

While PPTPs popularity has certainly dwindled over the last 2 decades in part due to the susceptibility to the ASLEAP dictionary attack tool, and other known vulnerabilities including the MS-CHAP protocol, it is still found around the web in various applications and operating systems.  

You might ask why PPTP is still around when there is clear data, from the NSA no less, that vulnerabilities exist and are there to be exploited?  

This is in part due to the integration of PPTP into operating systems such as Linux, Windows, and macOS in order to enable encrypted tunnels between the PC and VPN server using TCP port 1723 and General Routing Encapsulation, also known as GRE.  

PPTP does provide fast data speeds and has a lot of support but is to be avoided at all costs where possible due to the known vulnerabilities associated with it.

L2TP / IPSec

L2TP or Layer 2 Tunnel Protocol was initially launched in 1999 by Cisco as a direct upgrade to both the PPTP VPN protocol and L2F (Layer 2 Forwarding Protocol). L2TP alone does not provide strong encryption levels and is most often paired with another VPN protocol, IPSec (Internet Protocol Security).  

Whereas PPTP uses 128-bit encryption, L2TP/IPSec uses 265-bit encryption levels and is one of the most fundamentally secure algorithms used today with modern applications in financial, military and technology fields.  

Available support includes mobile operating systems, Windows and macOS 10.3  

L2TP/IPSec is not without flaws. More suited to anonymization over security with increased overheads due to the complex encryption levels and an overreliance on port 500, other protocols are more viable alternatives in certain applications.  

SSTP

Secure Socket Tunneling Protocol, also known as Microsoft Secure Socket Tunneling Protocol, was developed and is still owned and controlled by Microsoft with full integration on all Windows systems since the launch of Vista SP1.  

SSTP utilizes 2048-bit SSL/TLS certificates for authentication and 256-bit SSL keys for encryption and is generally regarded as a secure protocol with certain security comparisons to OpenVPN due to its use of SSL and data packets over HTTPS encapsulation.  

Designed for remote client access and without developer access to its underlying code mean SSTPs viability is somewhat limited to Microsoft-only devices.  

OpenVPN

OpenVPN is the most popular open-source VPN protocol meaning developers and members of the community can access the source code and action any security flaws rather than allow and backdoors or potential flaws to exist in the code. Other open-source protocols include SoftEther VPN, Libreswan VPN, and Freelan VPN, and each comes with its own benefits and drawbacks which we’ll discuss in another article.  

OpenVPN benefits from its customisability with changes to encryption procedures, ciphers, and network configurations all available to its users as well as its ability to support Perfect Forward Secrecy – an encryption method that makes the decoding of data extremely difficult for hackers.  

Using incredibly powerful AES-256-bit key encryption with 2048-bit RSA authentication and a 160-bit SHA1 hash algorithm and providing protection from man-in-the-middle attacks.  

Support is available on all operating systems including Linux or Windows, macOS, iOS, Android, and FreeBSD 

Again, OpenVPN is not without flaws. A limited number of available servers, complex manual setup, and the additional software client requirements mean OpenVPN is best suited to new or intermediate users through a VPN service that can handle all the heavy lifting.  

IKEv2

IKEv2 or Internet Key Exchange Version 2 is another popular VPN protocol and shares similarities with L2TP in that it is mostly paired with IPSec for encryption and authentication purposes.  

Security includes the AES-256-GCM cipher for encryption, as well as the SHA2-384 for integrity. IKEv2 also supports Perfect Forward Secrecy, using 3072-bit Diffie Hellman keys. 

IKEv2 excels at re-establishing connections after temporary loss and has great support across all operating systems, smartphones, routers, and connected homeware. It also supports the MOBIKE protocol which makes the VPN connection more adaptable to changing networks such as changing from wireless to cellular connection. 

As well as offering adaptability and connection-loss support, IKEv2 provides fast data-transfer which translates well into everyday use, gaming, and streaming.  

Limited to UDP port 500, IKEv2 is limited to firewall and network administration blocks.  

WireGuard

A relative newcomer to VPN protocols, WireGuard is still in a relatively nascent stage of development but has shown potential to be the best of all world. With faster speeds and fewer overheads due to the smaller and simpler codebase and comparative security in comparison to the other options available, WireGuard looks to be a protocol for the future.  

Several VPNs including Mullvad and NordVPN have been early adopters with the latter including it as part of their NordLynx technology. However, some providers are still somewhat hesitant to incorporate it into their service with concerns regarding stability and limited available testing.  

Lightway

Although not nearly as popular or as well-established as the other entrants to this list, we thought we should include the ExpressVPN developed protocol Lightway

With similarities to WireGuard in terms of speeds, lines of code, and overheads, Lightway looks to be an excellent addition to ExpressVPNs arsenal.  Still, in a beta-stage, Lightways core library will soon be open source, adding to the fundamental security of the protocol, and will definitely be interesting to see how it progresses.