Generating a Test Oracle from Program Documentation

Mr. Dennis K. Peters
(peters@mcmaster.ca)

CRL, McMaster University,
Hamilton, Ontario, Canada L8S 4K1


Abstract

Software testing involves execution of a program under test using some fault revealing input data and examination of the output to determine success or failure. A fundamental assumption of this testing is that there is some mechanism, an oracle, that will determine whether or not the results of a test execution are correct. In practice, this is often done by comparing the output, either automatically or manually, to some pre-calculated, presumably correct, output [39]. However, if the program is formally documented it is possible to use the specification to determine the success or failure of a test execution, as in [1], for example. This thesis discusses the development of a prototype tool that automatically generates a test oracle from formal program documentation.

In [25], [27] and [28] Parnas et al. advocate the use of a relational model for documenting the intended behaviour of programs. In this method, tabular expressions are used to improve readability so that formal documentation can replace conventional documentation. Relations are described by giving their characteristic predicate in terms of the values of concrete program variables. This documentation method has the advantage that the characteristic predicate can be used as a test oracle-it must be evaluated for each test execution (input and output) to assign pass or fail. This form of documentation is used for generating an oracle.

The design of a test oracle and a tool that can be used to generate an oracle are discussed in this thesis.