Introducing a Benchmark for Evaluating Reverse Engineering Tools

Abstract

Useful information can be discovered from source code with the examination of software systems. First step of the examination is to analyze the source code and making an abstract representation. In the second step different structures and problems can be discovered from the abstract representation, e.g. design patterns, code duplications and coding rule violations. These structures and problems are discovered by different kind of reverse engineering tools. Reverse engineering tools present their results in different formats, which makes them very difficult to compare. In our previous works [5][6] we presented a tool called DEEBEE(DEsign pattern Evaluation BEnchmark Environment) which supports the evaluation and comparison of design pattern miner tools only. We realized that DEEBEE can be extended to support the evaluation and comparison of every kind of reverse engineering tools as well. In this paper we introduce a benchmark - called BEFRIEND (BEnchmark For Reverse engInEering tools workiNg on source coDe) - with which the outputs of reverse engineering tools can be evaluated and compared easily and efficiently. The extension of DEEBEE into a general benchmark BEFRIEND is motivated by other works which try to evaluate and compare code duplication finder tools [3][4] and rule violation checker tools[1][7] as well. BEFRIEND supports different kinds of tool families, programming languages and software systems, and it enables the users to define their own evaluation criteria. Furthermore, it is a web-application open to the community and freely available [2]. We hope it will be accepted and used by the community in the future to evaluate and compare their tools with others.

@InProceedings{FHF08c,
    author    = {F\"{u}l\"{o}p, Lajos Jen\H{o} and Heged{\H{u}}s, P\'eter and Ferenc, Rudolf},
    booktitle = {Proceedings of the 6th Conference of PhD Students in Computer Science (CSCS 2008)},
    title     = {Introducing a Benchmark for Evaluating Reverse Engineering Tools},
    year      = {2008},
    address   = {Szeged, Hungary},
    month     = jul,
    pages     = {25},
    publisher = {University of Szeged},
    url       = {http://www.inf.u-szeged.hu/~cscs/cscs2008/pdf/CSCS2008-proceedings.pdf},
}