Understanding Specification-Driven Code Generation with LLMs: An Empirical Study Design
This program is tentative and subject to change.
Large Language Models (LLMs) are increasingly integrated into software development workflows, yet their behavior in structured, specification-driven processes remains poorly understood. This paper presents an empirical study design using \textit{CURRANTE}, a Visual Studio Code extension that enables a human-in-the-loop workflow for LLM-assisted code generation. The tool guides developers through three sequential stages—Specification, Tests, and Function—allowing them to define requirements, generate and refine test suites, and produce functions that satisfy those tests. Participants will solve medium-difficulty problems from the \emph{LiveCodeBench} dataset, while the tool records fine-grained interaction logs, effectiveness metrics (e.g., pass rate, all-pass completion), efficiency indicators (e.g., time-to-pass), and iteration behaviors. The study aims to analyze how human intervention in specification and test refinement influences the quality and dynamics of LLM-generated code. The results will provide empirical insights into the design of next-generation development environments that align human reasoning with model-driven code generation.
This program is tentative and subject to change.
Thu 19 MarDisplayed time zone: Athens change
14:00 - 15:30 | Session 5C - Specification-Driven Code and Model DevelopmentIndustrial Track / Early Research Achievement (ERA) Track / Short Papers and Posters Track / Research Track / Registered Report Track | ||
14:00 11mTalk | Requirement Formalization using Large Language Models Research Track Zhiyuan Hu National University of Defense Technology, Wei Ma Singapore Management University, Qiang Wang Academy of Military Sciences, Lingxiao Jiang Singapore Management University, Dongsheng Li National University of Defense Technology | ||
14:11 11mTalk | Understanding Specification-Driven Code Generation with LLMs: An Empirical Study Design Registered Report Track Giovanni Rosa Universidad Rey Juan Carlos, David Moreno-Lumbreras Universidad Rey Juan Carlos, Gregorio Robles Universidad Rey Juan Carlos, Jesus M. Gonzalez-Barahona Universidad Rey Juan Carlos | ||
14:22 11mTalk | AI-Assisted Requirements Traceability for Large-Scale Optical Network Systems: An Industrial Experience Report Industrial Track | ||
14:33 11mTalk | From Textual Descriptions to Code: A Filtering Approach for Locating Business Rules Industrial Track Nour Ayachi Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189 CRIStAL F-59000 Lille, France, Benoit Verhaeghe Berger-Levrault, Christopher Fuhrman École de technologie supérieure, Nicolas Anquetil University of Lille, Lille, France | ||
14:45 11mTalk | Generating User Clones from Questionnaires: A Lightweight Approach to Requirements Elicitation Short Papers and Posters Track Mai Hirabayashi Waseda University, Hironori Washizaki Waseda University, Naoyasu Ubayashi Waseda University, Juichi Takahashi AGEST, Inc, Yohei Takagi AGEST Inc. | ||
14:56 11mTalk | How Well Does Knowledge Injection Enhance LLM-aided Formal Protocol Modeling? Early Research Achievement (ERA) Track Yajia Lin Xidian University, Jie Su Xidian University, Cheng Wen Xidian University, rong wang , Cong Tian Xidian University, Zhenhua Duan Xidian University, Shengchao Qin Xidian University | ||
15:07 11mTalk | LLM Driven Business Rule Extraction from Enterprise Applications Early Research Achievement (ERA) Track Shrishti Pradhan TCS Research, Aishwarya Malvade TCS Research, Raveendra Kumar Medicherla TCS Research, Tata Consultancy Services, Manasi Patwardhan TCS Research | ||
15:18 11mTalk | SQL3M: Token Efficient Text-to-SQL Generation Short Papers and Posters Track Ibrahim Ücelehan Johannes Gutenberg University Mainz, Alina Geiger Johannes Gutenberg University Mainz, Dominik Sobania University of Duisburg-Essen, Germany | ||