What is the overhead of running pH?

To determine the performance impact of our kernel modifications, we ran the HBench-OS 1.0 [11] low-level benchmark suite on an HP Pavilion 8260 (266 MHz Pentium II, 160M SDRAM, Maxtor 91020 10G Ultra-DMA IDE hard disk) running a pre-release version of Debian/GNU Linux 2.2. Tests were run for ten iterations on a system running in single user mode. In Tables 1 and 2, ``Standard'' refers to a stock Linux 2.2.14 kernel. ``pH'' refers to a 2.2.14 kernel with pH extensions, with monitoring enabled for all processes and with status messages and automated response turned off. All times are in microseconds.

Tables 1 and 2 show that our modifications add significantly to system call overhead. Table 1 indicates that pH adds approximately 4.7 $\mu s$ to the execution time of simple system calls that normally would take less that 2 $\mu s$ to execute. Table 2 shows that pH causes process creation to be almost twice as slow for a dynamically-linked shell. Although these tables show a significant performance hit, they are not indicative of the impact on overall system performance.

Table 3 shows how overall performance is affected for a set of tasks. Here we report the output of time for three different kinds of operations: kernel builds, find / -print > /dev/null (a basic traversal of the file system), and Quake 2 frame rates. All of these tests were run in single-user mode. The most dramatic effect is seen in the system time of the kernel build, which almost doubles due to monitoring overhead. This difference, however, only causes a 4% slowdown in the clock time. The find test shows almost a 10% slowdown, and this is for a program that is almost entirely bound by the speed of filesystem-access system calls. Interestingly, the Quake 2 frame rate tests shows virtually no slowdown. These tests illustrate what we have observed informally by using the system ourselves: If delays are turned off, a user can use the modified workstation without noticing any differences in system behavior, even if she decides to run a compute and I/O intensive application such as Quake 2.

Table 1: System call latency results. All times are in microseconds. Standard deviations are listed in parentheses.

System Call	Standard ($\mu s$)	pH ($\mu s$)
getpid	1.1577 (0.00000)	5.8898 (0.00025)
getrusage	1.9145 (0.00000)	6.6137 (0.00138)
gettimeofday	1.6703 (0.00184)	6.3779 (0.00112)
sigaction	2.5609 (0.00010)	7.2928 (0.01029)
write	1.4135 (0.00187)	6.1637 (0.00075)

Table 2: Dynamic process creation latency results. Null refers to a fork of the current process. Simple is a fork of the current process plus an exec() of a hello-world program written in C. /bin/sh refers to the execution of hello-world through the libc system() interface, which uses /bin/sh to invoke hello-world. All times are in microseconds. Standard deviations are listed in parentheses.

Operation	Standard ($\mu s$)	pH ($\mu s$)
null	408.80 (00.618)	2497.90 (40.923)
simple	2396.24 (11.124)	8206.62 (11.795)
/bin/sh	9385.66 (26.761)	18223.96 (26.777)

Table 3: Overall system performance. All units are seconds (s), except for the Quake 2 test, which is in frames per second (fps). Ten trials were run for each example, except 100 trials were run for find. Each test was run once before beginning the measurements in order to eliminate initial I/O transients. Standard deviations are listed in parentheses.

Benchmark	Standard	pH
kernel build (s)
real	702.47 (0.07)	727.44 (0.29)
user	669.35 (0.60)	673.67 (0.55)
sys	33.00 (0.61)	53.60 (0.70)
find / -print (s)
real	5.68 (0.58)	6.24 (0.54)
user	1.61 (0.09)	1.59 (0.09)
sys	3.27 (0.09)	3.90 (0.17)
Quake 2 (fps)
demo1	22.89 (0.03)	22.87 (0.05)
demo2	23.30 (0.00)	23.30 (0.00)