Debugging Requirements Interview Scripts: A Framework for Quality Assessment
Requirements elicitation interviews are a fundamental technique for identifying stakeholders’ needs in software projects. Interview scripts, which are lists of questions prepared by the analyst to guide discussions with stakeholders, often exhibit issues or “smells”, such as unclear phrasing, use of technical jargon, omission of quality requirements, or failure to address prioritisation. Providing structured, high-quality feedback on such scripts remains costly and difficult to scale, particularly when expert review is required. This paper introduces a frame- work for systematically assessing and “debugging” requirements interview scripts, based on a coding scheme capturing interview smells and missing topics. We further investigate the extent to which such assessment can be supported by automated feedback generated by Large Language Models (LLMs). We compare automatic feedback with that of human reviewers using a ground truth defined by requirements engineering experts. Our results reveal a clear trade-off: LLMs provide broad coverage when identifying potential interview issues but tend to generate overly inclusive feedback, while human reviewers produce more precise, context-sensitive assessments. These findings show that interview-script quality can be systematically operationalised and partially automated. LLMs enable scalable first-pass screening, while human reviewers provide validation and contextual judgment, together supporting hybrid human–AI review workflows for requirements elicitation in both professional and training contexts.
Thu 20 AugDisplayed time zone: Eastern Time (US & Canada) change
13:45 - 15:15 | Requirements Elicitation: Methods & ValidationResearch Papers at A-1600 Chair(s): Andrea Wohlgemuth Utrecht University & FH Dortmund & GEBHARDT Intralogistics Group | ||
13:45 30mTalk | Debugging Requirements Interview Scripts: A Framework for Quality Assessment Research Papers Aref El-Maarawi Tefur Hasso Plattner Institute, University of Potsdam, Farnaz Fotrousi Chalmers University of Technology and University of Gothenburg, Alessio Ferrari University College Dublin (UCD) and Consiglio Nazionale delle Ricerche (CNR), Paola Spoletini Kennesaw State University, Walid Maalej University of Hamburg | ||
14:15 30mTalk | Under Pressure: Navigating Human Factors in AI-Supported Requirements Validation Research Papers Fabian Romano Rusch Helmut Schmidt University Hamburg, Lisa Stickel Ipsos GmbH, Frank Mantwill Helmut Schmidt University Hamburg | ||
14:45 30mTalk | From Chat to Interview: Agentic Requirements Elicitation with an Experience Typology Research Papers Dongming Jin Peking University, China, Zhi Jin Peking University, Wuhan University, Yaotian Yang Beijing Forestry University, Linyu Li Key Lab of High-Confidence of Software Technologies (PKU), Ministry of Education, Fang Zheng Key Lab of High-Confidence of Software Technologies (PKU), Ministry of Education, Yuanpeng He Key Lab of High-Confidence of Software Technologies (PKU), Ministry of Education, Wenchun Jing Institute for Artificial Intelligence, Peking University, Xiaohong Chen East China Normal University | ||
Main Auditorium