IPsec VPN: the basics

What is IPsec?

IPsec stands for Internet Protocol Security. The IP part tells the data where to go, and the sec encrypts and authenticates it. In other words, IPsec is a group of protocols that set up a secure and encrypted connection between devices over the public internet.

IPsec protocols are usually grouped by their tasks:

• Authentication Header (AH);
• Encapsulating Security Payload (ESP);
• Security Association (SA).

What is IPsec made of?

Asking what it is made of is similar to asking how it works. IPsec’s singular goal is to encrypt and establish a secure connection; that singular goal can be dissected into (and achieved by) three smaller groups of protocols. Each of those three separate groups takes care of separate unique tasks. 

Security Authentication Header (AH) — it ensures that all the data comes from the same origin and that hackers aren’t trying to pass off their own bits of data as legitimate.

Imagine you get an envelope with a seal. If the seal isn’t broken, nobody has tampered with the letter, right? Authentication Headers do the same for all the data transmitted over the IPsec VPN. However, this is but one of two ways IPsec can operate. The other is ESP.

Encapsulating Security Payload (ESP) — it’s an encryption protocol, meaning that the data package is transformed into an unreadable mess. Aside from encryption, ESP is similar to Authentication Headers — it can authenticate the data and check its integrity.

To get back to the letter and seal, if someone were to intercept the letter and open it, they’d find just a bunch of gibberish no human could read. On your end, the encryption happens on the VPN client, while the VPN server takes care of it on the other. 

Security Association (SA) is a set of specifications that are agreed upon between two devices that establish an IPsec connection.

The Internet Key Exchange (IKE) or the key management protocol is part of those specifications. IKE negotiates with security associations of other devices and authenticates them. IKE provides secure communication channels by exchanging a cryptographic key with the SA of another device.

To explain in layperson’s terms, SAs are like the cipher between two spies — it tells them how to interpret the coded message and to whom they are supposed to relay it. It’s the agreement between two devices about how to protect information during communication.

How does IPsec transmit information?

After IPsec is set up to use AH or ESP, it chooses the transmission mode: transport or tunnel.

IPsec Transport Mode: this mode encrypts the data you’re sending but not the info on where it’s going. So while malicious actors couldn’t read your intercepted communications, they could tell when and where they were sent. 

IPsec Tunnel Mode: tunneling creates a secure, enclosed connection between two devices by using the same old internet. Therefore, the connection is made safe and private. IPsec VPN works in this mode, as it creates the IPsec VPN tunnel.

A VPN protocol is a set of rules that describes how a VPN works and the steps it takes to secure your connection. Using IPsec is one of them. A VPN using an IPsec protocol suite is called an IPsec VPN.

What is IPsec VPN, and how does it work?

A VPN application encrypts your connection to the internet (or another device). A VPN uses protocols to encrypt the connection, and there is more than one way to do so. Using IPsec is one of them. A VPN using an IPsec protocol suite is called an IPsec VPN.

Let’s say you have an IPsec VPN client running. How does it all work?

1. You click Connect; 
2. An IPsec connection starts using ESP and Tunnel Mode; 
3. The SA establishes the security parameters, like the kind of encryption that’ll be used; 
4. Data is ready to be sent and received while encrypted.

These are the basics of IPsec. It may sound simple, but it goes a little deeper than this.

How does IPsec work step by step?

Let’s say two devices are trying to communicate using IPsec. One device is trying to send a message, and here’s what happens:

1. Key exchange —  IPsec sets up cryptographic keys (strings of randomized characters) to encrypt and decrypt each device’s messages;
2. Headers and trailers — all data transmitted via the internet gets chopped into smaller bits called packets. The IPsec protocol puts authentication and encryption info at the front (header) of the packet and the end of it (trailer). These headers and trailers tell the device what should be done with the data packet;
3. Authentication — IPsec authenticates each data packet by giving it a unique “name” or ID;
4. Encryption — IPsec encrypts the data packets’ contents;
5. Transmission — the packets are shipped off using the transport or tunneling protocols; 
6. Decryption — the device on the other end receives the message and uses the decryption key to unscramble the message.

How does IPsec impact MSS and MTU?

All data travels in data packets, which are measured in bytes. 

MSS, or maximum segment size, refers to a value of the maximum size a data packet can be (which is 1460 bytes).

MTU, the maximum transmission unit, on the other hand, is the value of the maximum size any device connected to the internet can accept (which is 1500 bytes).

These packets contain the information being sent, the IP header (20 bytes), and the TCP header (20 bytes), and all have a size limit.

IPsec doesn’t impact the value of a maximum transmission unit but will always lower the value of the maximum segment size. And this is the math proving it:

Experience IPsec in action

So now you know what IPsec VPN is within the VPN realm. For Surfshark VPN users running the app on smartphones, you are most likely using it under the IKEv2 moniker, which is a part of the IPsec family. And if you’re not a Surfshark user, why not become one? We have more than just IPsec to offer you!

FAQ

What is IKEv2?

IKEv2 (Internet Key Exchange version 2) is a protocol used in the Security Association part of the IPsec protocol suite.

It authenticates users — confirms that the devices at the ends of the connection are who they say they are — and then sets up an encrypted connection using Diffie–Hellman key exchange. It’s a widely used method of sending encryption ciphers over a public network without making them into the key for unlocking encrypted data.

An IPsec running the IKEv2 protocol is called IKEv2/IPsec, but the industry-standard name is IKEv2. It’s a development launched in 2005 that updated and fixed some of the issues the original IPsec with IKEv1 (launched in 1995) had.

Is IPsec safe?

When paired with IKEv2, IPsec is considered safe enough by major VPN providers worldwide. However, around 2015, allegations came out that the USA’s National Security Agency (NSA) was able to exploit it. The agency had either worked backdoors into IPsec or found ways to mess with the Diffie–Hellman key exchange. However, some experts in the field have disputed this claim.

Nevertheless, if you don’t feel safe, most VPN suppliers have alternatives to IPsec VPN protocols.

Is IPsec better than SSL?

Comparing IPsec and SSL (Secure Sockets Layer) is like comparing apples and oranges. Both protocols work on different levels of the OSI (Open Systems Interconnection) model. In short, this is how the two look side by side:

Both IPsec and SSL are safe and can be used to build a VPN. However, it ultimately boils down to the configurations, goals, and preferences of the VPN and its developers.