|
USENIX Technical Program - Abstract - WinsSys - August 2000
An Empirical Study of the Robustness of Windows NT Applications Using
Random Testing
Justin E. Forrester and Barton P. Miller, University of Wisconsin
Abstract
We report on the third in a series of studies on the reliability of
application programs in the face of random input.
In 1990 and 1995, we studied the reliability of UNIX application programs,
both command line and X-Window based (GUI).
In this study, we apply our testing techniques to applications running on the
Windows NT operating system.
Our testing is simple black-box random input testing; by any measure, it is a
crude technique, but it seems to be effective at locating bugs in real
programs.
We tested over 30 GUI-based applications by subjecting them to two kinds of
random input:
(1) streams of valid keyboard and mouse events and
(2) streams of random Win32 messages.
We have built a tool that helps automate the testing of Windows NT applications.
With a few simple parameters, any application can be tested.
Using our random testing techniques, our previous UNIX-based studies showed
that we could crash a wide variety of command-line and X-window based
applications on several UNIX platforms.
The test results are similar for NT-based applications.
When subjected to random valid input that could be produced by using the
mouse and keyboard, we crashed 21% of applications that we tested and hung an
additional 24% of applications.
When subjected to raw random Win32 messages, we crashed or hung all the
applications that we tested.
We report which applications failed under which tests,
and provide some analysis of the failures.
- View the full text of this paper in
HTML form and PDF form.
- If you need the latest Adobe Acrobat Reader, you can download it
from Adobe's
site.
- To become a USENIX Member, please see our Membership Information.
- Current USENIX Members may change their password.
|
Need help? Use our Contacts page.
