Estimating Software Reliability Using Inverse Sampling

Résumé du livre

Abstract:"This paper addresses one of the perpetual questions faced by software developers, 'How much testing is enough testing?'. Software testing accounts for a substantial portion of software development costs, but releasing software with unacceptable reliability is also very costly. We begin with a discussion of how classical notions of reliability can be applied to deterministic software and explain that the reliability of a software is as much a function of the way that it is used as of quality of the software. We then illustrate how simple operational profiles can be used to characterize usage patterns. Finally, we show how the reliability requirements can be used to determine how much testing is necessary and whether or not the software is acceptable. Using the method of inverse sampling, also called negative binomial sampling, we develop an efficient statistical procedure for quantifying the reliability of a piece of software. The procedure allows substantial reductions in the average number of executions run over the traditional binomial testing. Other issues such as the calculation of upper confidence bounds for software failure rate under both binomial and negative binomial sampling are also addressed. The results obtained are illustrated numerically and graphically on several cases arising in practice. Some issues for futher work, namely the use of sequential testing, the computer implementation of the methods developed and the testing of continuously-run software, are also discussed."