Several tools are published in the literature which are able to mine design patterns from source code. Because a common test database - a benchmark - is not available, the accuracy of the tools is difficult to check and measuring any kind of improvements on the tools is also problematic. As an all-in-one solution we have developed a benchmark for evaluating and comparing design pattern miner tools, and for ensuring a test database for them. In this paper we present some experiments performed with the benchmark. Two design pattern miner tools - Columbus and Maisa - are evaluated and compared. The tools are evaluated on C++ reference implementations of design patterns and on a real software system called NotePad++. Design pattern instances from NotePad++ recovered by professional software developers are also added to the benchmark.
BibTeX:
@InProceedings{FAV07,
author = {F{\"u}l{\"o}p, Lajos Jen\H{o} and {\'A}rp\'ad, Ilia and V{\'e}gh, {\'A}d\'am Zolt\'an and Ferenc, Rudolf},
title = {Comparing and Evaluating Design Pattern Miner Tools},
booktitle = {Proceedings of the 10th Symposium on Programming Languages and Software Tools (SPLST 2007)},
year = {2007},
pages = {372--386},
address = {Dobog\'ok\H{o}, Hungary},
month = jun,
publisher = {E\"otv\"os Lor\'and University},
}