Several tools are published in the literature which are able to mine design pattern usage from source code. Because a common test database – a benchmark – is not available, the accuracy of the tools is difficult to check and measuring any kind of improvements on the tools is also problematic. As an all-in-one solution we have developed a benchmark for evaluating and comparing design pattern miner tools and for ensuring a test database for them. In this paper we present some experiments performed with the benchmark. Two design pattern miner tools – Columbus and Maisa – are evaluated and compared. The tools are evaluated on C++ reference implementations of design patterns, on a real software system called NotePad++ and on FormulaManager, which is a software implemented by us to have a test case where the usage of design patterns is well defined and documented. Design pattern instances from NotePad++ recovered by professional software developers are also added to the benchmark.
BibTeX:
@Article{FIV09,
author = {F{\"u}l{\"o}p, Lajos Jen\H{o} and Ilia, {\'A}rp{\'a}d and V{\'e}gh, {\'A}d{\'a}m Zolt{\'a}n and Heged\H{u}s, P{\'e}ter and Ferenc, Rudolf},
title = {Comparing and Evaluating Design Pattern Miner Tools},
journal = {Annales Universitatis Scientiarum Budapestinensis de Rolando E{\"o}tv{\"o}s Nominatae Sectio Computatorica},
year = {2009},
volume = {31},
pages = {167--184},
issn = {0138-9491},
url = {http://ac.inf.elte.hu/Vol_031_2009/167.pdf},
}