A public unified bug dataset for java and its assessment regarding metrics and bug prediction

Abstract

Bug datasets have been created and used by many researchers to build and validate novel bug prediction models. In this work, our aim is to collect existing public source code metric-based bug datasets and unify their contents. Furthermore, we wish to assess the plethora of collected metrics and the capabilities of the unified bug dataset in bug prediction. We considered 5 public datasets and we downloaded the corresponding source code for each system in the datasets and performed source code analysis to obtain a common set of source code metrics. This way, we produced a unified bug dataset at class and file level as well. We investigated the diversion of metric definitions and values of the different bug datasets. Finally, we used a decision tree algorithm to show the capabilities of the dataset in bug prediction. We found that there are statistically significant differences in the values of the original and the newly calculated metrics; furthermore, notations and definitions can severely differ. We compared the bug prediction capabilities of the original and the extended metric suites (within-project learning). Afterwards, we merged all classes (and files) into one large dataset which consists of 47,618 elements (43,744 for files) and we evaluated the bug prediction model build on this large dataset as well. Finally, we also investigated cross-project capabilities of the bug prediction models and datasets. We made the unified dataset publicly available for everyone. By using a public unified dataset as an input for different bug prediction related investigations, researchers can make their studies reproducible, thus able to be validated and verified.

@Article{FTL20,
    author   = {Ferenc, Rudolf and T{\'o}th, Zolt{\'a}n and Lad{\'a}nyi, Gergely and Siket, Istv{\'a}n and Gyim{\'o}thy, Tibor},
    journal  = {Software Quality Journal},
    title    = {A public unified bug dataset for java and its assessment regarding metrics and bug prediction},
    year     = {2020},
    note     = {Open Access},
    pages    = {1447–1506},
    volume   = {28},
    doi      = {10.1007/s11219-020-09515-0},
    keywords = {Bug dataset, Code metrics, Static code analysis, Bug prediction},
    url      = {https://doi.org/10.1007/s11219-020-09515-0},
}