Beyond Code Coverage – an Approach for Test Suite Assessment and
Dávid Tengeri, Árpád Beszédes, Tamás Gergely, László Vidács, Dávid
Havas and Tibor Gyimóthy
Code coverage is successfully used to guide white box
test design and evaluate the respective test completeness.
However, simple overall coverage ratios are often not precise
enough to effectively help when a (regression) test suite needs to
be reassessed and evolved after software change. We present an
approach for test suite assessment and improvement that utilizes
code coverage information, but on a more detailed level and adds
further evaluation aspects derived from the coverage. The main use
of the method is to aid various test suite evolution situations
such as removal, refactoring and extension of test cases as a
result of code change or test suite efficiency enhancement. We
define various metrics to express different properties of test
suites beyond simple code coverage ratios, and present the
assessment and improvement process as an iterative application of
different improvement goals and more specific sub-activities. The
method is demonstrated by applying it to improve the tests of one
of our experimental systems.
Keywords: code coverage,
regression testing, test suite quality, test suite refactoring,
test suite evolution, white box testing metrics.