The mbuf_tags were originally developed for use in conjunction with the OpenBSD IPsec stack [6]. Their primary purpose, described in more detail in Section 3, was to record how ``securely'' each packet was received, i.e., under what IPsec security association(s) was a packet received. Practically immediately, however, a second use was established: to detect loops in outgoing IPsec packets. For this, we needed to record the same information as in the incoming packet case. However, we (correctly, as it turns out) foresaw the need for adding tags with different types of information in the future. Thus, we chose an approach similar in some respects to the BSD struct sockaddr for recording network address information.
mbuf_tags consist of two parts: a fixed-size header, which contains the length and type of the tag as well as a pointer to other tags attached to the same packet, followed by type-dependent data. These tags can be combined together in a chain, and attached to the first mbuf of a packet. An mbuf is a data structured used by the BSD networking stack to contain packets and other information. A chain of mbufs can be used to store large packets, with the first mbuf containing additional information about the packet as a whole.
These tags can serve multiple roles, as we shall see in Section 3. They can indicate processing that has already occurred to the packet (e.g., IPsec SA under which a packet was received), or it may represent a ``reminder'' for processing that must be applied to the packet in the future (e.g., cryptographic processing that must be done to the packet by a combined network+cryptographic accelerator card).
A similar approach was taken by NetBSD, in the form of aux mbufs. These are mbufs that are attached to an mbuf header in a way similar to mbuf_tags chains. Because they use unmodified mbufs, the former enable the use of all the mbuf-manipulating routines and, perhaps more importantly, allow space to be allocated in chunks without resorting to the kernel memory allocator with every allocation, as is the case with the use of malloc(9) in the mbuf_tag approach. Thus, the processing cost of adding new tags to a packet is a step function with aux mbufs, whereas it increases linearly with the number of tags attached to a packet in our scheme. However, the mbuf_tag approach does not place more pressure on the mbuf allocator, which can run out of space on a busy router or firewall, since mbufs are allocated from a reserved area of memory, which is typically fixed to a certain percentage of kernel memory at kernel-configuration time. Furthermore, we intend to use memory pool, as we discuss in Section 4, to reduce to overhead of the kernel memory allocator. Finally, mbuf_tags are better-integrated with the mbuf subsystem, allowing for seamless replication and de-allocation when the respective mbufs are duplicated or released.
Linux uses a 48-byte array in the skbuff structure, which the ``owner'' of a packet (the protocol or socket that queued the packet) can use to store private information. Apart from its limited size, the ownership semantics of this array make it unsuitable for use in certain scenarios (e.g., when the producer and the consumer of a tag are separated by code that performs its own processing that requires use of the array).
FreeBSD recently adopted mbuf_tags, adding a cookie field in the tag header. This allows for private, module-specific definition of new tags without requiring coordinated allocation among different modules/developers. We intend to include this change in the OpenBSD tag implementation.
