Local Proxy Server

The local proxy server, localCache, implements all the different parts of the local proxy server shown in Figure 1. Specifically, the HTTP request filter is implemented as a method call that does substring matching on HTTP requests and invokes scripts to create HTTP responses based on the rules database. The local profile engine is implemented as a class (a directed graph with weighted edges) which trims itself when the number of nodes grows too large. Currently, it trims itself when the number of nodes reaches 1024. It does so by removing half of the nodes (512) that are lightest according to the weighting method we described earlier. The local pre-fetch engine is implemented using a queue. The local profile engine feeds the queue with URLs. When a pre-fetch request is issued, the URL is removed from the queue and placed into a session set. The same URL will then no longer be pre-fetched, even if it was placed in the queue again. The local cache manager is implemented as a class that interfaces a URL to a disk file via a hashing function. Currently, HTTP response headers are also stored. In addition to disk store, we also keep a copy of pre-fetched items in virtual memory. This is more for ease of implementation than for performance. An optimized version of the program can use just the disk store and thus leave a small memory map.

Of particular interest with the local cache manager is the determination of an item's expiry time. Let t_expire, t_date, and t_last_modified be times (in seconds) extracted from the respective HTTP response headers. If a header is not available, the variable defaults to 0. Further, let HTML_expire and IMAGE_expire be the times (in seconds) the user provides. These times specify how long the user wants an HTML file or an image file to stay fresh in the absence of some/all of the above HTTP response headers. Let t_now be the current time in seconds at the local machine and expire be the time when the document expires. The algorithm for determining the freshness of an item in the local cache manager is as follows:

if (t_expire>0 && t_date>0) 
  expire = t_expire - t_date + t_now; 
else if (t_last_modified>0 && t_date>0)
  expire = (t_date - t_last-modified)/2 + t_now;
else if (t_last_modified>0) 
  expire = (t_now - t_last_modified)/2 + t_now;
else if (HTMLDocument(url))
  expire = HTML_expire + t_now;
else
  expire = IMAGE_expire + t_now;