The OO7 Benchmark

OO7 is the standard benchmark for object-oriented databases [oo7]. The database consists of objects of various types connected in a tree structure (Figure 7).

We used the small, medium-3, and medium-9 configurations described in [oo7] (numbers in the medium configurations show the density of internal connections). The total database sizes are about 10MB for the small, 60MB for the medium-3, and 100MB for the medium-9.

We ran three types of traversals, T1, T2A, and T2C. They all traverse the object hierarchy and visit one element within each intermediate node. T1 visits all the elements in read only mode. T2a updates one element for each intermediate node. T2c updates each element four times.

Two types of numbers, cold and warm, are shown for the small configuration. To obtain the cold numbers, we started the benchmark with an empty buffer cache. An ObjectStore file open call (objectstore::open) pre-fetches some of the database contents into memory. The time needed to execute this procedure is also included in the cold numbers. Hot numbers are obtained by running four consecutive transactions after the cold run and computing the mean of the first three. For ObjectStore, the option to retain persistent pointers (objectstore::retain_persistent_pointers) was enabled. Thus, hot runs do not include pointer-swizzling overhead.

We do not report the setrange performance, because the setrange algorithm could not run OO7 for larger databases: as described in Section 4.3.4, setrange must retain all the range information in memory until a commit, and it uses up the in-kernel heap.

  

Figure: OO7 database structure. The database consists of composite parts, each of which is a web of atomic parts. Composite parts are indexed by a tree.

  

Figure: Small configuration results. Buffer cache was warm.

  

Figure: Small configuration results. Buffer cache was cold.

  

Figure: Medium configuration (fanout=3) results. Buffer cache was cold.

  

Figure: Medium configuration (fanout=9) results. Buffer cache was cold.

Figure 8 shows ``warm'' results from the small database. First, for T1 traversal, page grain logging and page diffing exhibit the same performance, because they perform the same tasks in read-only transactions. For other traversals, page diffing performs better than page grain logging because of smaller log activity. The UNIX page diffing version is consistently twice as slow as SPIN's page diffing version, for the same reason described in the previous section. ObjectStore consistently performs worse than any of the Rhino versions. However, the performance discrepancy is smaller than in the RVM benchmark, because OO7 touches more bytes per page, and thus IPC overhead is amortized.

Figure 9 shows ``cold'' results for the small database. In the cold runs all Rhino versions perform about the same, because disk I/O dominates the time.

Figures 10 and 11 show the ``cold'' results on a medium database of fanout 3 and fanout 9, respectively. Page diffing does not perform well in these benchmarks, because of the memory pressure caused by shadow pages. Page grain logging is clearly the best choice in medium-3 and medium-9. ObjectStore performs better here, because the IPC cost is amortized over a large amount of updates.