Binary and multi-class classification of Self-Admitted Technical Debt: How far can we go?
Aiming for a trade-off between short-term efficiency and long-term stability, software teams often resort to suboptimal solutions, neglecting best software development practices. Such solutions may introduce technical debt (TD), which can lead to maintenance issues. To facilitate future fixing, developers mark code with any issues using textual comments, resulting in Self-Admitted Technical Debt (SATD). Detecting SATD in source code is crucial, as it helps programmers locate potentially erroneous snippets, allowing for suitable interventions and thereby improving code quality. There are two main types of SATD detection, i.e., binary classification and multi-class classification, grouping TD comments into SATD/Non-SATD categories, and multiple categories, respectively.
We attempt to understand the extent to which state-of-the-art research has addressed the issue of detecting SATD, both binary and multi-class classification. Based on this investigation, we also propose a practical approach for detecting SATD using Large Language Models (LLMs).
First, we conducted a literature review to understand the extent to which the two types of classification have been tackled by existing research. Second, we developed SALA, a dual-purpose tool that leverages Natural Language Processing (NLP) techniques and neural networks to handle both types of classification. An empirical evaluation has been performed to compare SALA with state-of-the-art baselines.
The literature review reveals that while binary classification has been well studied, multi-class classification has not received adequate attention. The empirical evaluation shows that SALA obtains a promising performance, and outperforms the baselines with respect to various quality metrics.
We conclude that more effort is needed to tackle multi-class classification of SATD. To this end, LLMs hold potential, albeit with further investigation into possible fine-tuning and prompt engineering strategies.