We have currently modified the Bro Intrusion Detection System [2] to operate on a cluster of end host systems [5]. The goal is to take advantage of the significant potential parallelism present in the IDS problem [3] which observes that much of the work involved is per-flow independent, such as stream reassembly and application parsing. Additional work is per-host-pair independent, such as session structure-based analysis. Thus if we can take advantage of the many network streams present in actual traffic, we should be able to scale our rich IDS to very high data rates.
Our approach decomposes the analysis into multiple, independent processing elements running on commodity PC hardware. We use a single manager node which serves as a central point for alerts and responses (such as issuing ACL blocks to routers), one or two proxy systems which act to coordinate shared state between the IDSs, and a number of sensor nodes which interpret the actual traffic.
Additionally, a load balancer [4] is used to spread the traffic to the sensors based on a hash of the IP address pair. The load balancer ensures that all traffic associated with a pair of hosts always goes to the same sensor node, while distributing the entire stream across the group of sensors.
We are currently operating this IDS on two clusters of approximately 10 systems, one at a major university and one at a national laboratory. But we need to understand how the system will scale to larger clusters and how different traffic may stress a cluster differently from single systems. To further this, we need to build a framework for further testing.
Our framework for DETER consists of a Click module [1] to amplify a small test trace into a large volume of traffic, another Click module which implements our load balancer when read from a trace, and a series of scripts used to control experiments. Our initial testing used a highly stressful case composed entirely of HTTP traffic, which can cause a single-processor Bro to drop packets at rates barely exceeding 3600 packets per second.
In this particular case, we see indications of a super linear speedup when processing the trace on a cluster, as a cluster setup removes significant inter-flow state explosion effects. This speedup evaporates when the traffic volume is also scaled with cluster side.