We demonstrate how operating system protection can be used to control fine-grained programs flexibly and efficiently. Operating systems use hardware-based protection to isolate processes from one another. However, the way that current operating systems implement this protection has caused researchers to deem them too slow and inflexible for controlling fine-grained programs. Fine-grained programs have different protection domains and may interact often in the course of a computation. The effect of a large number of protection domain crossings must be handled securely (i.e., correctly with respect to the security requirements) to prevent attacks and efficiently to minimize performance degradation. In this paper, we show that a security model implemented on a fast and flexible IPC mechanism can enforce security requirements that language-based systems cannot with little performance impact.
Several operating systems use hardware-based protection to prevent processes from inadvertently and/or maliciously modifying one another. Each process has an address space that defines a set of memory segments and the process's access rights to those segments. A process can only access memory in its own address space. In addition, the operating system has a security model that associates processes with their access rights to system resources. Using address spaces of suitable granularity and process access rights to controlled resources, an operating system can control a process's operations as desired.
However, operating system security models have been deemed to lack the
performance and flexibility necessary to control fine-grained
programs. While some systems have been built that efficiently control
processes in dynamically-defined protection
domains [3, 8, 14], these systems have been applied only to
more traditional applications (e.g., PostScript interpreters). In an
application composed of fine-grained programs, programs with different
protection domains interact often (perhaps as much as on each method
invocation). In a recent paper, Wallach et
al. [25] discard address space-based protection
from consideration for applications with fine-grained programs by
noting that IPC between two COM objects on Windows NT takes 1000 times
longer than a procedure call (230 
s to 0.2 s). We claim
that this discrepancy can be virtually eliminated while gaining
security and maintaining flexibility.
We have developed a prototype implementation of a flexible security
model for controlling downloaded content. This model is implemented
on the Lava Nucleus. The Lava Nucleus provides address spaces,
threads, fast IPC, flexible paging, and IPC interception that enable
efficient and flexible control of processes. In this paper, we
primarily concentrate on the effectiveness of the Lava nucleus for
implementing a flexible security model and its resultant performance.
We show that fast IPC and IPC interception enable the implementation
of dynamically authorized IPC that can be performed in as little
as 9.5 s. We believe further room for optimziation is
possible, given an ideal estimate for dynamically authorized IPC is
about 4 s. Also, flexible page mapping in the Lava Nucleus
enables objects of size greater than the hardware page size to be
shared among processes, so coarse-grained sharing of memory between
processes is possible.
The structure of this paper is as follows. In Section 2, we compare language-based and operating system-based security models. In Section 3, we describe an operating system security model for downloaded executable content. In Section 4, we describe the implementation of this model on the Lava Nucleus. In Section 5, we examine the performance of the prototype implementation and compare its performance to language-based models for fine-grained programs of the sort discussed by Wallach et al. In Section 6, we conclude and present future work.