From the SPLASH-2 suite, we chose 4 SVM applications, radix, barnes, fft, and water-nsquare. They all use the SVM protocol developed by our colleagues at Princeton [26]. The SVM protocol communication is built directly on top of VMMC, using mostly send and remote fetch mechanism. In addition, we selected 3 applications from Princeton Display Wall Project [11] glaze, iso-surface, and vdd. These applications use VMMC Winsock as their underlying communication mechanism.
Vis is a visualization program, implemented by our colleagues to use a cluster of 13 PCs to visualize isosurfaces of scientific data. The program has three components: client control, isosurface extraction, and rendering. The client control runs on a single PC to implement the user interface for users to steer the entire visualization process. The isosurface extraction uses 4 PCs in the cluster to extract isosurfaces in parallel, and sends them to the rendering program which runs on 8 PCs. Each renderer PC simply receives isosurfaces, renders and display the rendered images on a tiled, scalable display wall. VMMC Winsock is used for all interprocess communication among the PCs running the client control, isosurface extraction, and rendering programs.
Glaze is a commercial OpenGL evaluation program from Evans & Sutherlands. An OpenGL framework, developed by our colleague, allows us to use one PC to drive OpenGL animation of any commercial software on a tiled display. We ran the glaze program on a PC; a custom wrapper DLL (opengl32.dll) intercepts the OpenGL calls made by the program and distributes the commands to 8 separate PCs which together drive a 2x4 tiled display. The wrapper DLL transmits rendering commands over the VMMC Winsock layter to the renderers. The renders behave just like those in vis: they simply receive the commands and render them for its portion of the tiled display.
Vdd is a virtual display driver that implements a large desktop that has the same resolution (3850x1500) as the Display Wall. It is installed on a 450 MHz Pentium II PC running NT 5B2 OS. A user process is responsible for packetizing the updates made to the vdd's memory-resident framebuffer and distributing them to the 8 PCs that drive the wall. We used NT VMMC Sockets with the pointer-based extension to distribute framebuffer updates efficiently.
Table 3 presents OS-dependent components of UTLB overhead, without any memory constraint. The miss rates and overheads for the NT platform are measured directly from running the 7 applications on our NT VMMC cluster. We use these numbers, as well as the micro-benchmark results, to predict the overheads on Linux.
Under tight memory constraints, a host UTLB lookup miss may also trigger unpinning pages. UTLB can deal with low-memory situations by unpinning virtual pages that are not currently involved in data transfer. This is a critical feature for UTLB-based VMMC to be useful in a multi-programming environment. Table 4 presents the the same experiments with additional memory constraints. Due to varying nature of the applications, the memory constraint is set differently on a per-application basis. Also, it turned out that our non-SVM applications require most of its working set to be pinned as receive buffers. Therefore, we only present the memory-constrained result for the 4 SVM applications.
In the tables, host pin rate is the ratio between the number of pin operations and the number of UTLB lookups; similarly for host unpin rate. Lower pin rate (or lookup miss rate) translates into lower average UTLB overhead. And faster page-pin and page-unpin can further reduce the average UTLB overhead. Our NT vs. Linux comparison indicates that improving system calls such as page-pin and -unpin may further improve user-level communication.