Background: Numerous methodologies have been used to study technical debt. Among different data sources, questions and answer sites provide an opportunity to study how users reference and request support on technical debt. To date only a handful of studies, focusing on narrow aspects of technical debt, investigate the phenomenon through the lens of Stack Overflow.

Aims: We aim at gaining an in-depth understanding on the characteristics of technical debt questions of one of the most popular programming question and answer sites, namely Stack Overflow. In addition, we assess if identification strategies based on natural language processing and machine learning models, rather than the keyword-based ones used so far, can be leveraged to automatically identify and classify questions regarding technical debt on the platform.

Method: We use combination of automated and manual processes to identify technical debt questions on Stack Overflow. The final set of 415 questions is analyzed both quantitatively and qualitatively to study (i) technical debt types, (ii) question length, (iii) perceived urgency, (iv) sentiment, and (v) emerging themes. Natural language processing and machine learning techniques are used to evaluate if technical debt questions can be identified and classified automatically.

Results: Architecture debt is the most recurring debt type, followed by code and design debt. Most questions display mild urgency, with the frequency of higher urgency steadily declining as urgency rises. Question length varies across debt types. Sentiment of questions is mostly neutral. 29 themes emerge, with technical debt resolution and management being most recurrent. Machine learning models can be used to automatically identify technical debt questions and binary urgency accurately, but not debt types.

Conclusions: Different patterns emerge from the analysis of technical debt questions on Stack Overflow. The results provide further insights on the phenomenon, and support the adoption of a more comprehensive strategy to identify technical debt questions.