ESEIW 2025
Sun 28 September - Fri 3 October 2025

This program is tentative and subject to change.

Recent innovations in artificial intelligence (AI), primarily powered by large language models (LLMs), have transformed how programmers develop and maintain software—leading to new frontiers in software engineering (SE). The advanced capabilities of LLM-based programming assistants to support software development tasks have led to a rise in the adoption of LLMs in SE. However, little is known about the evidenced-based practices, tools and processes verified by research findings, supported and adopted by AI programming assistants. To this end, our work conducts a preliminary evaluation exploring the textit{beliefs} of LLM used to support software development tasks. We investigate 17 evidence-based claims posited by empirical SE research across five LLM-based programming assistants. Our findings show that LLM-based programming assistants have ambiguous beliefs regarding research claims and lack credible evidence to support responses. Based on our results, we provide implications for practitioners adopting LLM-based programming assistants in development contexts and shed light on future research directions to enhance the reliability and trustworthiness of LLMs—aiming to increase awareness and adoption of evidence-based SE research findings in practice.

This program is tentative and subject to change.

Thu 2 Oct

Displayed time zone: Hawaii change

10:10 - 11:10
Evidence and Research Quality in Software EngineeringESEM - Technical Track / ESEM - Emerging Results and Vision Track / ESEM - Journal First Track / at Kaiulani II
Chair(s): Mika Mäntylä University of Helsinki and University of Oulu
10:10
15m
Talk
Cognitive Biases in Software Engineering: Debiasing through Reconception
ESEM - Emerging Results and Vision Track
Heidi Hietala M3S, University of Oulu, Burak Turhan University of Oulu
10:25
15m
Talk
Exploring the Evidence-Based Beliefs of LLM-Based Programming Assistants
ESEM - Emerging Results and Vision Track
Chris Brown Virginia Tech, Jason Cusati Virginia Tech
10:40
15m
Talk
Research artifacts for human-oriented experiments in software engineering: An ACM badges-driven structure proposal
ESEM - Journal First Track
Cathy Guevara-Vega Universidad Técnica del Norte, Beatriz Bernárdez University of Seville, Margarita Cruz Risco University of Seville, Amador Durán University of Seville, Antonio Ruiz-Cortés University of Seville, Martín Solari Universidad ORT Uruguay
10:55
15m
Talk
Aggregating empirical evidence from data strategies studies: a case on model quantization
ESEM - Technical Track
Santiago del Rey Universitat Politècnica De Catalunya - Barcelona Tech, Paulo Sérgio Medeiros Federal University of the State of Rio de Janeiro (UNIRIO), Guilherme Horta Travassos Federal University of Rio de Janeiro, Xavier Franch Universitat Politècnica de Catalunya, Silverio Martínez-Fernández UPC-BarcelonaTech
Pre-print
Hide past events