Conclusion and Future Work

We have designed, implemented and evaluated an automatic application-controlled file prefetching system called AASFP that is particularly useful for multimedia applications with irregular disk access patterns. It is automatic in the sense that the system exploits application-specific disk access patterns for file prefetching without any manual programming. The idea of extracting I/O related code from the original code is very general and we believe it is applicable to other languages such as C++ or Java as well; the required effort to support other languages should also be comparable to this work. The Linux-based AASFP prototype implementation is fully operational and provides up to 54% overall application improvement for a real-world volume visualization application. Currently we are continuing the development of AASFP. We are extending the current prototype to allow multiple I/O-intensive applications to run on an AASFP-based system simultaneously. The key design issue here is to allocate disk resource among multiple processes, depending on their urgency on disk access requirements. We are also building on the AASFP technology to develop a high-performance I/O subsystem for large-scale parallel computing clusters.

Finally we are extending the AASFP prototype to the context of Network File System (NFS), and generalize the application-specific prefetching to a more general concept called active file server architecture. Here, in addition to standard file access service, we allow an application to deposit an arbitrary program either to manipulate accessed file data on the fly such as compression or encryption (data plane), or to exercise different control policies such as prefetching, replacement, and garbage collection (control plane). The active file server architecture significantly enhances the customizability and flexibility of file access, and thus improves both the performance of individual applications and the overall efficiency of the file system. A major research challenge for active file server is the design of a procedural interface for application program segments that is both general and efficient enough to accommodate various control plane or data plane processing requirements, and sufficiently rigid to ensure data security and safety. We plan to use AASFP with NFS as a case study to gain some concrete experiences with the architectural design issues associated with active file server. The code of this project will be available for download at the URL: https://www.ecsl.cs.sunysb.edu/archive.html.