A Public Bug Database of GitHub Projects and Its Application in Bug Prediction

Abstract

Detecting defects in software systems is an evergreen topic, since there is no real world software without bugs. Many different bug locating algorithms have been presented recently that can help to detect hidden and newly occurred bugs in software. Papers trying to predict the faulty source code elements or code segments in the system always use experience from the past. In most of the cases these studies construct a database for their own purposes and do not make the gathered data publicly available. Public datasets are rare; however, a well constructed dataset could serve as a benchmark test input. Furthermore, open-source software development is rapidly increasing that also gives an opportunity to work with public data.

In this study we selected 15 Java projects from GitHub to construct a public bug database from. We matched the already known and fixed bugs with the corresponding source code elements (classes and files) and calculated a wide set of product metrics on these elements. After creating the desired bug database, we investigated whether the built database is usable for bug prediction. We used 13 machine learning algorithms to address this research question and finally we achieved F-measure values between 0.7 and 0.8. Beside the F-measure values we calculated the bug coverage ratio on every project for every machine learning algorithm. We obtained very high and promising bug coverage values (up to 100).

Publication
Proceedings of the 16th International Conference on Computational Science and Its Applications (ICCSA 2016), Beijing, China, Pages 625–638

BibTeX:

@InProceedings{TGF16,
    author    = {T{\'o}th, Zolt{\'a}n and Gyimesi, P{\'e}ter and Ferenc, Rudolf},
    booktitle = {Proceedings of the 16th International Conference on Computational Science and Its Applications (ICCSA 2016)},
    title     = {A Public Bug Database of {GitHub} Projects and Its Application in Bug Prediction},
    year      = {2016},
    address   = {Beijing, China},
    month     = jul,
    pages     = {625--638},
    publisher = {Springer International Publishing},
    doi       = {10.1007/978-3-319-42089-9_44},
    keywords  = {Bug prediction, Bug database},
    url       = {https://link.springer.com/chapter/10.1007%2F978-3-319-42089-9_44},
}