ESEIW 2024
Sun 20 - Fri 25 October 2024 Barcelona, Spain

Context: In software engineering (SE), the systematic mapping study (SMS) is one of the methods adopted for evidence-based decision-making, selecting and synthesizing relevant literature on a specific research topic. Tool support is beneficial due to the time-intensive nature of the SMS process and its activities. Gap: Large language models (LLMs) such as ChatGPT-4.o can potentially accelerate repetitive activities, such as the data extraction in the SMS process. Therefore, having a tool to assist this activity could save time and effort. This proof-of-concept study evaluates how ChatGPT-4.o can support SMS activities in SE, particularly data extraction. Method: We assessed the accuracy of utilizing ChatGPT-4.o for extracting data in one SMS, in contrast to the manual extraction. Results: The accuracy of ChatGPT-4.o was 87.83%. Conclusions: Our preliminary findings suggest that entirely replacing the human extraction process with ChatGPT-4.o is not recommended. However, it is promise employing ChatGPT for semi-automated data extraction for evidence syntheses in SMSs in SE.

Thu 24 Oct

Displayed time zone: Brussels, Copenhagen, Madrid, Paris change

11:00 - 12:30
11:00
20m
Full-paper
ChatGPT application in Systematic Literature Reviews in Software Engineering: an evaluation of its accuracy to support the selection activity
ESEM Technical Papers
Katia Romero Felizardo UTFPR-CP, Marcia Sampaio Lima Universidade do Estado do Amazonas - UEA, Anderson Deizepe UTFPR-CP, Tayana Conte Universidade Federal do Amazonas, Igor Steinmacher Northern Arizona University
11:20
20m
Full-paper
Is generalisation hindering the adoption of your findings?
ESEM Technical Papers
Rogardt Heldal Western Norway University of Applied Science
11:40
20m
Full-paper
Threats to Validity in Software Engineering -- hypocritical paper section or essential analysis?
ESEM Technical Papers
Patricia Lago Vrije Universiteit Amsterdam, Per Runeson Lund University, Qunying Song Lund University, Roberto Verdecchia University of Florence
Pre-print
12:00
15m
Vision and Emerging Results
Data extraction for systematic mapping study using a large language model - a proof-of-concept study in software engineering
ESEM Emerging Results, Vision and Reflection Papers Track
Katia Romero Felizardo UTFPR-CP, Igor Steinmacher Northern Arizona University, Marcia Sampaio Lima Universidade do Estado do Amazonas - UEA, Anderson Deizepe UTFPR-CP, Tayana Conte Universidade Federal do Amazonas, Monalessa P. Barcellos Federal University of Espírito Santo
12:15
15m
Vision and Emerging Results
Crossover Designs in Software Engineering Experiments: Review of the State of Analysis
ESEM Emerging Results, Vision and Reflection Papers Track
Julian Frattini Blekinge Institute of Technology, Davide Fucci Blekinge Institute of Technology, Sira Vegas Universidad Politecnica de Madrid
Link to publication DOI Pre-print Media Attached