As the dispatcher component is centralized, its performance determines the scalability of the cluster. We demonstrate in Section 6.1 that the bulk of the overhead imposed on the dispatcher node is associated with communication with other cluster nodes. The key to achieving greater cluster scalability, therefore, is to reduce this communication overhead.
The communication overhead can be reduced in two ways, (1) by reducing the size of the messages sent to the dispatcher, and (2) by limiting the number of network packets sent to the dispatcher. For (1), the distributor compacts the request URL by hashing it into a 32-bit integer before sending it to the dispatcher. (2) is achieved by batching together several messages before they are sent to the dispatcher. This enables several messages to be put in the same TCP packet and reduces interrupt and protocol processing overheads at the dispatcher node.
An interesting design choice is the degree of batching, i.e., the number of messages collected together before sending them to the dispatcher. Although the overhead on the dispatcher node is lower with a higher degree of batching, the response time is adversely affected because messages might be held for a long time before a batch of the requisite size is formed.
Rather than fixing the degree of batching, our implementation batches messages only as long as the replies for messages in the previous batch are outstanding. This has the beneficial effect of dynamically adjusting the degree of batching based on the load on the dispatcher node. If the dispatcher is heavily loaded, the time taken by it to respond to messages in an earlier batch is long, which in turn causes high degree of batching of the next set of messages. Similarly, a lightly loaded dispatcher would result in sets of messages where the degree of batching is low.