In agile software development, user stories (US) and acceptance criteria (AC) are popular ways of recording requirements. While guidelines have been proposed in the literature to assess the quality of US and AC, their correct application remains a manual task. In this work, we designed both a machine learning (ML) and a natural language processing (NLP) classifier for automatically assessing agile software requirements following quality indicators found in the literature. We evaluated their performance to improve the quality of requirements in a user study, as well as the users’ perceptions on the usage of such tools as writing aids. While improvements were notable in the quality of requirements written by participants, the improvements were more marginal when using the NLP classifier compared to the ML one. However, participants reported more satisfaction towards the NLP classifier for its ``explainability'' compared to the ML one.
Yi Peng University of Gothenburg and Chalmers University of Technology, Hans-Martin Heyn University of Gothenburg & Chalmers University of Technology, Jennifer Horkoff Chalmers and the University of Gothenburg
Anne Hess Technical University of Applied Sciences Würzburg-Schweinfurt, Gerald Heller Consultant and Trainer, Hartmut Schmitt HK Business Solutions GmbH, Cornelia Seraphin msg systems AG, Ismaning, Oliver Karras TIB - Leibniz Information Centre for Science and Technology
