Operation

  Central to both ESP and AH are an abstraction called security association, or SA. In each SA there is information (algorithm IDs, keys, etc.) stored describing how the wanted protection should be setup. For two peers to be able to communicate they need matching SAs at each end. When deciding what SA should be used for outbound traffic, some kind of security policy database needs to be consulted. In OpenBSD, this is currently implemented as an extension to the routing table, where source/destination addresses, protocol, and ports serve as selectors.

Looking at the wire format, IPsec works by inserting an extra header between the IP header and the payload. This header holds IPsec-specific data, such as an anti-replay sequence number, cryptographic synchronization data, and integrity check values. If the security protocol in use is ESP, a cryptographic transform is applied to the payload in-place, effectively hiding the data. As an example, an UDP datagram protected by ESP is shown in figure 1.

  

Figure 1: IPsec Transport Mode

This mode of operation is called transport mode, as opposed to tunnel mode which is typically used when a security gateway is protecting datagrams for other hosts. Tunnel mode differs from transport mode by the addition of a new, outer, IP header consisting of the security gateways' addresses instead of the actual source and destination, as shown in figure 2.

  

Figure 2: IPsec Tunnel Mode

As was mentioned earlier, this mode is ideal for implementing VPNs.

The last, but not least, part of the picture is a key management infrastructure. IPsec can only work if the keys in the SAs are synchronized and updated in a secure fashion. To automate this task, different protocols have been devised that allow two peers to compute identical keys without actually sending all the data needed for it over the wire [7,8]. The Internet Key Exchange, IKE, is one such, and Photuris is another. The main difference between these two lies in the complexity level. IKE is a very complex protocol which, however, offers considerable flexibility in negotiating and establishing SAs. IKE is the official IETF standard. Both protocols work in a similar vein, by first building an encrypted application-level ``tunnel'' where further key exchanges take place. The Diffie-Hellman algorithm [7] is used to make it computationally hard to crack the key computation. Every SA is assigned a lifetime, either in wall-clock time or in volume, and when such a lifetime expires, the key management daemon renegotiates with the peer, creating new SAs with fresh keys.