An Empirical Evaluation of Evolutionary Algorithms for Test Suite Generation

Evolutionary algorithms have been shown to be effective
at generating unit test suites optimised for code coverage. While many
aspects of these algorithms have been evaluated in detail (e.g., test length
and different kinds of techniques aimed at improving performance, like
seeding), the influence of the specific algorithms has to date seen less
attention in the literature. As it is theoretically impossible to design
an algorithm that is best on all possible problems, a common approach
in software engineering problems is to first try a Genetic Algorithm,
and only afterwards try to refine it or compare it with other algorithms
to see if any of them is more suited for the addressed problem. This
is particularly important in test generation, since recent work suggests
that random search may in practice be equally effective, whereas the
reformulation as a many-objective problem seems to be more effective.
To shed light on the influence of the search algorithms, we empirically
evaluate six different algorithms on a selection of non-trivial open source
classes. Our study shows that the use of a test archive makes evolutionary
algorithms clearly better than random testing, and it confirms that the
many-objective search is the most effective.