Program Reliability Estimation Tool


Dr. S.M. Reza Nejat


Abstract

Testing is a very demanding procedure in software production, that takes a lot of effort, time and resources during both development and maintenance. Moreover, statistical testing is a very costly procedure, especially if high reliability requirements are placed on the software as in safety-critical, or safety-related software cases. The main question is when to stop testing, or how many tests are needed?

Singh et al. [49], using the method of the negative binomial, developed a procedure for quantifying the reliability of a module. According to their approach, the number of tests can be computed based on hypothesis testing. We implemented this method for a reliability estimation of a program.

In this work, a prototype black-box automated testing tool, called Program Reliability Estimation Tool (PRET) was developed as a statistical test generator and reliability estimation tool based on an operational profile (a proposed testing model) and negative binomial sampling.

The tool has a command line user interface. The inputs to the PRET are: an integer (0 or 1) to choose the usage (0: only generate test cases, 1: does the testing process), the test specification context file name, the data file name, the program under test name, and the oracle name.

PRET computes the number of test cases, generates test cases, runs the generated test cases, evaluates the result of each test run by using an oracle, and estimates the reliability of the program based on test results.