TCSE logo 
 Sigsoft logo
Sustainability badge
Sat 3 May 2025 11:33 - 11:51 at 203 - ML4ESE Chair(s): Andreas Jedlitschka

Context. The use of large language models for qualitative analysis is gaining attention in various fields, including software engineering, where qualitative methods are essential to understanding human and social factors. Goal. This study aimed to investigate how LLMs are currently used in qualitative analysis and how they can be used in software engineering research, focusing on identifying the benefits, limitations, and practices associated with their application. Method. We conducted a systematic mapping study and analyzed 21 relevant studies to explore reports of using LLM for qualitative analysis reported in the literature. Findings. Our findings indicate that LLMs are primarily used for tasks such as coding, thematic analysis, and data categorization, with benefits including increased efficiency and support for new researchers. However, limitations such as output variability, challenges capturing nuanced perspectives, and ethical concerns regarding privacy and transparency were also evident. Discussions. The study highlights the need for structured strategies and guidelines to optimize LLM use in qualitative research within software engineering. Such strategies could enhance the effectiveness of LLMs while addressing ethical considerations. Conclusion. While LLMs show promise for supporting qualitative analysis, human expertise remains essential for data interpretation, and continued exploration of best practices will be crucial for their effective integration into empirical software engineering research.

Sat 3 May

Displayed time zone: Eastern Time (US & Canada) change

11:00 - 12:30
ML4ESEWSESE at 203
Chair(s): Andreas Jedlitschka Fraunhofer IESE
11:00
15m
Talk
A Framework for Using LLMs for Repository Mining Studies in Empirical Software Engineering
WSESE
Vincenzo De Martino University of Salerno, Joel Castaño Fernández Universitat Politècnica de Catalunya, Fabio Palomba University of Salerno, Xavier Franch Universitat Politècnica de Catalunya, Silverio Martínez-Fernández UPC-BarcelonaTech
Pre-print
11:15
18m
Talk
Can Machine Learning Support the Selection of Studies for Systematic Literature Review Updates?
WSESE
Marcelo Costalonga Pontifical Catholic University of Rio de Janeiro (PUC-Rio), Bianca Minetto Napoleão Université du Québec à Chicoutimi, Maria Teresa Baldassarre Department of Computer Science, University of Bari , Katia Felizardo Federal Technological University of Paraná, Igor Steinmacher NAU RESHAPE LAB, Marcos Kalinowski Pontifical Catholic University of Rio de Janeiro (PUC-Rio)
11:33
18m
Talk
Applications and Implications of Large Language Models in Qualitative Analysis: A New Frontier for Empirical Software Engineering
WSESE
Matheus de Morais Leça University of Calgary, Lucas Valença University of Calgary, Reydne Bruno dos Santos UFPE, Ronnie de Souza Santos University of Calgary
11:51
18m
Talk
Large Language Model for Qualitative Research - A Systematic Mapping Study
WSESE
Cauã Ferreira Barros Federal University of Goiás, Bruna Borges Azevedo Federal University of Goiás, Valdemar Graciano Neto Federal University of Goiás, Mohamad Kassab Boston University, USA, Marcos Kalinowski Pontifical Catholic University of Rio de Janeiro (PUC-Rio), Hugo Alexandre D. do Nascimento Federal University of Goiás, Michelle C.G.S.P. Bandeira Federal University of Goiás
12:09
21m
Live Q&A
ML4ESE: Discussion
WSESE

:
:
:
: