3. EtE Monitor Architecture

EtE monitor consists of four program modules shown in Figure 1:

  

Figure 1: EtE Monitor Architecture.

1.

The Network Packet Collector module collects the network packets using tcpdump[24] and records them to a Network Trace, enabling offline analysis.

2.

In the Request-Response Reconstruction module, EtE monitor reconstructs all TCP connections from the Network Trace and extracts HTTP transactions (a request with the corresponding response) from the payload. EtE monitor does not consider encrypted connections whose content cannot be analyzed. After obtaining the HTTP transactions, the monitor stores some HTTP header lines and other related information in the Transaction log for future processing (excluding the HTTP payload). To rebuild HTTP transactions from TCP-level traces, we use a methodology proposed by Feldmann [7] and described in more detail and extended to work with persistent HTTP connections by Krishnamurthy and Rexford [14].

3.

The Web Page Reconstruction module is responsible for grouping underlying physical object retrievals together into logical web pages and stores them in the Web Page Session Log.

4.

Finally, the Performance Analysis and Statistics module summarizes a variety of performance characteristics integrated across all client accesses.

EtE monitor can be deployed in several different ways. First, it can be installed on a web server as a software component to monitor web transactions on a particular server. However, our software would then compete with the web server for CPU cycles and I/O bandwidth (as quantified in Section 7). Another solution is to place EtE monitor as an independent network appliance at a point on the network where it can capture all HTTP transactions for a web server. If a web site consists of multiple web servers, EtE monitor should be placed at the common entrance and exit of all web servers. If a web site is supported by geographically distributed web servers, such a common point may not exist. Nevertheless, distributed web servers typically use ``sticky connections'', i.e., once the client has established a connection with a web server, the subsequent client requests are sent to the same server. In this case, EtE monitor can still be used to capture a flow of transactions to a particular geographic site.