The primary goal of the SHRIMP project is to investigate how to design high-performance servers by leveraging commodity PC hardware and commodity PC software. A key research component is to design and implement a communication mechanism whose performance is competitive with or better than that of custom-designed multicomputers. The challenge is to achieve good communication performance without requiring special operating system support. This paper describes our design decisions, experience, and performance results of porting our virtual memory-mapped communication (VMMC) mechanism and stream sockets to Windows NT.
Our previous work on fast communication for PC clusters was done for
the Linux operating system. During the initial phase of the project,
we studied how to design a network interface to support VMMC and
successfully implemented a 16-node prototype PC cluster with our
custom-designed network interfaces and an Intel Paragon routing
network [5, 7, 6]. The VMMC mechanism,
our low-level communication layer, performs direct data transfers
between virtual memory address spaces, bypassing the operating system.
Its main ideas are separating control from data transfer and using
virtual memory mechanism to provide full protection in a
multi-programming environment. We demonstrated that VMMC achieves
3.75 
s end-to-end latency using a custom-designed network
interface.
During the second phase, we designed and implemented a VMMC mechanism
with extended features for PC clusters connected via Myrinet, a
commercial system area network and the operating system was again
Linux [15, 9]. With a programmable network
interface (Myrinet), we showed that VMMC achieves about 14 s
end-to-end latency and delivers bandwidth close to the hardware limit.
We also designed and implemented several compatibility communication
layers such as message-passing [1], RPC [3],
and Unix stream sockets [14], and showed that
they deliver good performance.
We recently ported VMMC and several compatibility software libraries to PCs running Windows NT 4.0 and 5.0 Beta. Several factors motivated our entry into the NT world. First, we want to leverage more PC hardware devices and software systems that are available for the NT platform. We particularly want to use high performance graphics cards that have only NT drivers. Second, we needed good kernel support for SMPs. Linux did not have such solid support for multiprocessors. Ever changing nature of Linux kernel sources also poses a daunting task for maintaining our VMMC software up-to-date. It also makes it really difficult difficult to distribute our software to other research institutes. Porting VMMC to Windows NT eliminates this concern and allows us to take advantage of the huge user base.
This paper describes our design decisions, porting experience, and performance results. We also address the questions, whether the promise of requiring no special OS support still holds when porting the communication mechanisms to NT, and whether the porting retains good communication performance. Lastly we would like to share our experience with the readers about the pros and cons of porting user-level communication mechanisms to Windows NT.