Comparing and Evaluating Design Pattern Miner Tools


Several tools are published in the literature which are able to mine design pattern usage from source code. Because a common test database – a benchmark – is not available, the accuracy of the tools is difficult to check and measuring any kind of improvements on the tools is also problematic. As an all-in-one solution we have developed a benchmark for evaluating and comparing design pattern miner tools and for ensuring a test database for them. In this paper we present some experiments performed with the benchmark. Two design pattern miner tools – Columbus and Maisa – are evaluated and compared. The tools are evaluated on C++ reference implementations of design patterns, on a real software system called NotePad++ and on FormulaManager, which is a software implemented by us to have a test case where the usage of design patterns is well defined and documented. Design pattern instances from NotePad++ recovered by professional software developers are also added to the benchmark.

Annales Universitatis Scientiarum Budapestinensis de Rolando Eötvös Nominatae Sectio Computatorica, 31:167–184


    author   = {F{\"u}l{\"o}p, Lajos Jen\H{o} and Ilia, {\'A}rp{\'a}d and V{\'e}gh, {\'A}d{\'a}m Zolt{\'a}n and Heged\H{u}s, P{\'e}ter and Ferenc, Rudolf},
    title    = {Comparing and Evaluating Design Pattern Miner Tools},
    journal  = {Annales Universitatis Scientiarum Budapestinensis de Rolando E{\"o}tv{\"o}s Nominatae Sectio Computatorica},
    year     = {2009},
    volume   = {31},
    pages    = {167--184},
    issn     = {0138-9491},
    url      = {},