|
USENIX 2005 Annual Technical Conference, FREENIX Track — Abstract
Pp. 175–188 of the Proceedings
Auto-pilot: A Platform for System Software Benchmarking
Charles P. Wright, Nikolai Joukov, Devaki Kulkarni, Yevgeniy Miretskiy, and Erez Zadok, Stony Brook University
Abstract
When developing software, it is essential to evaluate its performance
and stability, making benchmarking an essential and significant
part of the software development cycle.
Benchmarking is also used to show that a system is useful or provide
insight into how systems behave.
However, benchmarking is a tedious task that few enjoy, but every
programmer or systems researcher must do.
Developers need an easy-to-use system for collecting and analyzing
benchmark results.
We introduce Auto-pilot, a tool for producing accurate and
informative benchmark results.
Auto-pilot provides an infrastructure for running tests, sample test
scripts, and analysis tools. Auto-pilot is not just another metric or
benchmark: it is a system for automating the repetitive tasks of
running, measuring, and analyzing the results of arbitrary programs.
Auto-pilot can run a given test until results stabilize, automatically
highlight outlying results, and automatically detect memory leaks.
We have used Auto-pilot for over three years on eighteen distinct
projects and have found it to be an invaluable tool that saved us
significant effort.
- View the full text of this paper in HTML and PDF.
Until April 2006, you will need your USENIX membership identification in order to access the full papers. The Proceedings are published as a collective work, © 2005 by the USENIX Association. All Rights Reserved. Rights to individual papers remain with the author or the author's employer. Permission is granted for the noncommercial reproduction of the complete work for educational or research purposes. USENIX acknowledges all trademarks within this paper.
- If you need the latest Adobe Acrobat Reader, you can download it from Adobe's site.
|
Need help? Use our Contacts page.
