Replication and Extension of Schnappinger’s Study on Human-level Ordinal Maintainability Prediction Based on Static Code MetricsShort Paper
As a part of a research project concerning software maintainability assessment in collaboration with the development team, we replicated a study from Schnappinger et al. on human-level ordinal maintainability prediction. Our goal was to validate that we could obtain the same results with the open-source dataset and the open-source tool Javanalyser. Moreover, we extended the setup to predict continuous maintainability and evaluated the overall influence of the size of the class over the predictions. Our approach consisted of nearly $20,000$ experimental shots to replicate and extend the original study. All our datasets, code, and results are released publicly available to allow for further analysis by the community. In the end, we successfully replicated the original study. Moreover, we showed that continuous maintainability leverage better prediction than an ordinal scale. Finally, we have shown that metrics other than size contain information that is essential for a fine-grained maintainability prediction. This study shows that it is necessary to explore the nature of what is measured by code metrics, and is also the first step in the construction of a maintainability model.