ICSE 2026 (series) / New Ideas and Emerging Results (NIER) /
Empirical and Sustainability Aspects of Software Engineering Research in the Era of Large Language Models: A Reflection
This program is tentative and subject to change.
Research involving the use of Large Language Models (LLMs) may face several challenges related to rigour in benchmarking, contamination, replicability, and sustainability. With this paper, we invite the research community to understand and reflect on how these challenges are addressed in Software Engineering (SE). Our results provide a structured overview of current LLM-based SE research at ICSE, highlighting both encouraging practices and persistent shortcomings. We conclude with recommendations to strengthen benchmarking rigour, improve replicability, and address the financial and environmental costs of LLM-based SE.
This program is tentative and subject to change.
Fri 17 AprDisplayed time zone: Brasilia, Distrito Federal, Brazil change
Fri 17 Apr
Displayed time zone: Brasilia, Distrito Federal, Brazil change
11:00 - 12:30 | AI for Software Engineering 20New Ideas and Emerging Results (NIER) / Research Track / Journal-first Papers at Asia I Chair(s): Mike Papadakis University of Luxembourg | ||
11:00 15mTalk | Is Hyper-Parameter Optimization Different for Software Analytics? Journal-first Papers Link to publication Pre-print | ||
11:15 15mTalk | On the Effectiveness of LLM-as-a-judge for Code Generation and Summarization Journal-first Papers Giuseppe Crupi Università della Svizzera italiana, Rosalia Tufano Università della Svizzera Italiana, Alejandro Velasco William & Mary, Antonio Mastropaolo William and Mary, USA, Denys Poshyvanyk William & Mary, Gabriele Bavota Software Institute @ Università della Svizzera Italiana | ||
11:30 15mTalk | A Catalog of Data Smells for Coding Tasks Journal-first Papers Antonio Vitale Politecnico di Torino, University of Molise, Rocco Oliveto University of Molise, Simone Scalabrino University of Molise Link to publication | ||
11:45 15mTalk | Towards Automating Domain-Specific Data Generation for Text-to-SQL: A Comprehensive Approach Journal-first Papers Salmane Chafik UM6P College of Computing, Saad Ezzini King Fahad University of Petroleum and Minerals, Ismail Berrada UM6P College of Computing | ||
12:00 15mTalk | Empirical and Sustainability Aspects of Software Engineering Research in the Era of Large Language Models: A Reflection New Ideas and Emerging Results (NIER) David Williams University College London, Maria Kechagia National and Kapodistrian University of Athens, Max Hort Simula Research Laboratory, Aldeida Aleti Monash University, Justyna Petke University College London, Federica Sarro University College London | ||
12:15 15mTalk | FORGE: An LLM-driven Framework for Large-Scale Smart Contract Vulnerability Dataset Construction Research Track Jiachi Chen Sun Yat-sen University, Yiming Shen Sun Yat-sen University, Jiashuo Zhang Peking University, China, Zihao Li Hong Kong Polytechnic University, John Grundy Monash University, Zhenzhe Shao Sun Yat-sen University, Yanlin Wang Sun Yat-sen University, Jiashui Wang Zhejiang University, Ting Chen University of Electronic Science and Technology of China, Zibin Zheng Sun Yat-sen University Pre-print | ||