Check out the new USENIX Web site. next up previous
Next: Inside the Anypoint Switch Up: Anypoint Services Previous: ALRMs

    
Example: NFS Storage Router

To illustrate the structure of Anypoint services, we outline use of Anypoint for an L7 storage router for the NFSv3 file system protocol. We refer to this system as ``Slice-lite'' or Slite because it is a simplified form of the Slice storage architecture [5,4]. Slite presents an ensemble of standard NFS servers as a single, unified, virtual NFS server. As in Slice, the directory tree is partitioned into subtree buckets and spread across the servers as it grows. This partitioning uses a mkdir switching policy, with a parameter for subtree granularity. A soft-state map tracks the assignment of subtree buckets to nodes.

The Slite ALRM uses content-aware routing to direct NFS requests to the assigned servers. Slice has previously shown that NFS is amenable to virtualization using this technique because most NFS operations apply to a single content item--a directory, name entry, or file--evident from the request type and arguments. The ALRM extracts a key identifying the subtree bucket from each request's NFS file handle at a known offset, and uses it to index the map to identify the assigned server. This indirection allows dynamic rebalancing of the buckets across the ensemble; this technique is common to cluster-based Internet services [31,21,36,23]. With Slite, as in Slice and these other services, fine-grained content-aware request routing improves locality of the server caches as well as balancing load. A switch-based implementation can deliver the best latency and bandwidth, both of which are critical for file services.

Slite differs from Slice in two key respects. First, Anypoint enables Slite to run the NFSv3 protocol over a reliable, rate-controlled transport; Slice is limited to NFS/UDP, which sends one request or response per packet and relies on the RPC layer for primitive rate control and retransmission. Second, Slite is easier to deploy and it can use standard NFS servers. Although both are NFS-compliant, Slite is not Posix-compliant because it does not support readdirplus, link, and rename operations that cross server boundaries. While still compatible with Anypoint, this coordination requires the more comprehensive Slice approach. We use Slite for simplicity.

The Slite ALRM redirects some complex operations to a designated back-end server derived from the coordinator in the Slice architecture. In particular, the coordinator executes namespace operations (mkdir, rmdir, lookup) on name entries that cross servers (switched directories). The coordinator uses a write-ahead intention logging protocol for failure atomicity [4]. To simplify the ALRM further, our prototype indirects lookup through the coordinator, which maintains an index of switched directories. These redirect cases are easy to encode in the ALRM.


next up previous
Next: Inside the Anypoint Switch Up: Anypoint Services Previous: ALRMs
Kenneth G. Yocum
2003-01-20