ReqGenX: An Empirical Study of Atomic Decomposition, Artifact Regeneration, and Reconstruction for Legacy SRS Documents
This program is tentative and subject to change.
\textbf{Background:} Evaluating automated Software Requirements Specification (SRS) generation is challenging because few datasets provide fine-grained traceability between source requirements, intermediate elicitation artifacts, and generated specifications. \textbf{Aims:} We aim to study whether legacy SRS documents can be transformed into traceable synthetic pre-SRS artifacts that support fine-grained evaluation of LLM-based SRS generation. \textbf{Method:} We conduct an empirical study using ReqGenX, a controlled pipeline that decomposes SRS sections into source-grounded atomic statements, routes atoms to standards-inspired artifact types through multi-LLM plurality voting, and generates artifacts using constrained prompts with iterative judge-guided refinement. We evaluate ReqGenX on seven PURE SRS documents using grounding, quality, information retention, and downstream reconstruction analyses. \textbf{Results:} ReqGenX produces faithful and usable atoms, with median AlignScore values typically between 0.96 and 0.99 and Prometheus scores ranging from 4.34 to 4.85. Generated artifacts remain strongly grounded in their source atoms, with AlignScore values typically between 0.80–0.94 and judge pass rates near 100%; stricter Prometheus evaluation yields pass rates from 54.8% to 97.1%. In a downstream SRS reconstruction case study, artifact-backed atoms remain recoverable from generated SRSs, with SBERT means between 0.69 and 0.75 and AlignScore medians between 0.76 and 0.84. \textbf{Conclusions:} Traceable synthetic pre-SRS artifacts can support more fine-grained evaluation of LLM-based SRS generation, while exposing tradeoffs among faithfulness, information retention, and artifact completeness.
This program is tentative and subject to change.
Thu 8 OctDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
11:00 - 12:30 | Requirements, Regulation and Compliance for AI SystemsESEM - Software Engineering in Practice Track / ESEM - Emerging Results, Vision, and Reflection Papers Track / ESEM - Technical Track at Mars | ||
11:00 15mTalk | ReqGenX: An Empirical Study of Atomic Decomposition, Artifact Regeneration, and Reconstruction for Legacy SRS Documents ESEM - Technical Track Ragib Shahariar Ayon Texas State University, Rayed Fahmi Texas State University, Sumon Biswas Case Western Reserve University, Shibbir Ahmed Texas State University | ||
11:15 15mTalk | Can LLMs identify missing elements in requirements elicitation? An empirical evaluation using production derived user stories in an industry setting ESEM - Software Engineering in Practice Track Vinay Kabadi The University of Melbourne, Naresh Gutha , Abhiram Wuntakal , Lahiri Bellarykar , Xuan-Bach D. Le University of Melbourne, Patanamon Thongtanunam University of Melbourne, Christoph Treude Singapore Management University, Matthias Galster University of Canterbury | ||
11:30 15mTalk | Blurred Compliance Boundaries: Analyzing Open Source License Conflicts in MCP-Based Agent Workflows ESEM - Technical Track Xing Cui Institute of Software, Chinese Academy of Sciences; University of Chinese Academy, Jingzheng Wu Institute of Software, The Chinese Academy of Sciences, Tianyue Luo Institute of Software, Chinese Academy of Sciences, Xiang Ling Institute of Software, Chinese Academy of Sciences | ||
11:45 15mTalk | Operationalizing the EU AI Act in Agile Software Development: A Guideline-Based Approach ESEM - Software Engineering in Practice Track Dennis Schrader University of Applied Sciences and Arts Hannover, Eva-Maria Schön University of Applied Sciences Emden/Leer, Henning Fritzemeier Volkswagen AG, Michael Neumann University of Applied Sciences & Arts Hannover Pre-print | ||
12:00 15mTalk | Assessing the Substitution of LLM-Based Decision Components ESEM - Technical Track Sergio Cobos Universitat Oberta de Catalunya, Robert Clarisó Universitat Oberta de Catalunya, Javier Luis Cánovas Izquierdo Universitat Oberta de Catalunya | ||
12:15 10mTalk | Toward Ethical Requirements for Healthcare AI Software Systems: Emerging Results from a Multi-Stakeholder Interview Study ESEM - Emerging Results, Vision, and Reflection Papers Track Yutan Huang Monash University, Chetan Arora Monash University, Tanjila Kanij , Anuradha Madugalla Deakin University, School of IT, Australia, John Grundy Monash University | ||