This section describes experiments we performed to evaluate the effectiveness of bundling compared to other network class loading mechanisms.
There are two fundamental ways to apply bundling. First, it may be applied to the class and resource files used by a single application, based on profiles collected from that application. This approach is reasonable if the intent is to optimize the transfer of files used by a single application in isolation. The second way it can be applied is to a library used by many applications. Applying bundling to a library used by multiple applications is a more difficult problem than applying it to a single application, because the class loading behavior of different applications is much less consistent than the class loading behavior of multiple runs of a single application. (Note that there is no absolutely rigid distinction between these approaches. For example, an office application might have a common library of classes as well as several `sub-applications', such as word processor, spreadsheet, etc. The loading behavior for the classes and resources within the sub-applications would likely be quite consistent from run to run. However, each sub-application might use the common library in different ways, resulting in inconsistent loading behavior within the classes and resources of the library from run to run.)
In order to test both extremes (multiple applications vs. single application), we generated bundlings from a subset of JDK 1.2.2's rt.jar, which contains the standard Java libraries used by all applications. The subset includes AWT, Swing, and Java2D, but does not include the `core' packages (java.lang.*, java.util.*, etc.) Since almost all interactive Java applications use AWT, Swing, or Java2D, we felt the rt.jar subset was a good example of a substantial library used by many applications in nontrivial ways.
The first experiment uses profiles from several applications and applets to generate bundlings for the subset of rt.jar. Since the applications load different class files and resources in significantly different orders, this experiment represents a `stress test' of the bundling algorithm. We would like to emphasize that applying bundling to a large, multi-purpose library is not an ideal application of bundling, since bundling is designed to take advantage of regularities in class loading behavior.
The second experiment uses profiles from a single application to generate bundlings for the rt.jar subset. Since there is more regularity in the class and resource loading behavior of a single application than in several different applications, this experiment represents an ideal scenario for bundling, in which we would expect its performance to come close to that of cumulative compression in terms of the compression ratio achieved, while avoiding the high latency delays associated with on-demand loading.
The third experiment measures the effectiveness of the bundlings generated in the first experiment on two applications not represented in the input profiles used to generate those bundlings. This experiment is an even less ideal application for bundling than the first experiment; however, it does offer some insight into how the performance of bundling may degrade when it encounters class loading behavior not represented in the input profiles.
The fourth experiment measures the startup time for one of the test applications using simulated network bandwidth and latency. The experiment is designed to determine how well bundling performs in a real JVM, as opposed to off-line simulations.