ECSA 2024
Tue 3 - Fri 6 September 2024 Luxembourg, Luxembourg

User stories serve as a fundamental tool in agile software development methodologies, articulating the functional requirements of a system from an end-user perspective. However, while user stories excel in delineating the desired features and functionalities, they frequently overlook the non-functional aspects critical to the system’s success. These non-functional aspects encompass a spectrum of quality concerns, including but not limited to performance, security, reliability, usability, and compatibility. Despite their paramount importance, these quality concerns often remain implicit or underrepresented in user stories, necessitating a deliberate effort to extract and elucidate them during the requirements elicitation process. Failure to address these quality concerns upfront can lead to architectural decisions that overlook critical performance bottlenecks, security vulnerabilities, reliability issues, and usability shortcomings. Consequently, this oversight may result in suboptimal system designs, increased development costs, delayed time-to-market, diminished user satisfaction, and heightened operational risks. This paper presents an ISO-25010 compliant Transfer Learning approach for automated quality concerns extraction from user stories and corresponding acceptance criteria. The proposed solution is constructed upon the Transformer-based RoBERTa-Large model, leveraging and extending its pre-trained capabilities. This approach proficiently classifies user stories and acceptance criteria into 5 most critical user quality concerns including Usability, Performance, Reliability, Security, and Compatibility. This process involves cleaning and preprocessing the dataset followed by fine-tuning the pre-trained models on the refined data set. A comparative analysis of Three mainstream BERT variants including RoBERTa-base, DistilBERT and XLNET is also provided. Considering non-availability of public data sets in this scope, a dataset of approximately 1000 user stories with acceptance criteria was compiled by mining 30 projects, collected from different sources. This dataset was subsequently labeled through an extensive labeling activity. The findings suggest that the RoBERTa-Large fine-tuned variant achieves an impressive level performance in terms of accuracy, precision, recall and Avg F1 score.

Wed 4 Sep

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

11:00 - 12:35
Technical Session 1: Architecture decision makingResearch Papers at Hollenfels
Chair(s): Jasmin Jahic University of Cambridge, UK
11:00
15m
Short-paper
Towards Teamwise Informed Decisions On Microservice Security SmellsShort Paper
Research Papers
Francisco Ponce , Jacopo Soldani University of Pisa, Italy, Hernan Astudillo Universidad Andrés Bello, Chile, Antonio Brogi Università di Pisa
11:15
15m
Short-paper
Automated Quality Concerns Extraction from User Stories and Acceptance Criteria for Early Architectural DecisionsShort Paper
Research Papers
Khubaib Amjad Alam National University of Computer and Emerging Sciences, Hira Asif National University of Computer & emerging Sciences (FAST-NUCES), Irum Inayat Chalmers | University of Gothenburg, Saif-Ur-Rehman Khan Department of Computing, Shifa Tameer-e-Millat University (STMU)
11:30
25m
Full-paper
Exploring Architectural Design Decisions in Mailing Lists and their Traceability to Issue TrackersBest Paper Award CandidateArtifact Award CandidateResearch Paper
Research Papers
Mohamed Soliman Paderborn University
11:55
25m
Full-paper
Introducing Architecture Decision Records in Practice: An Action Research StudyExperience Report
Research Papers
Bardha Ahmeti Chalmers | University of Gothenburg, Maja Linder Chalmers | University of Gothenburg, Raffaela Groner Chalmers | University of Gothenburg, Rebekka Wohlrab Chalmers University of Technology
12:20
15m
Short-paper
Helping architects to make quality design decisions using LLM-based assistantsShort Paper
Research Papers
Andres Diaz Pace UNICEN University, Antonela Tommasel ISISTAN Research Institute, CONICET-UNCPBA, Rafael Capilla Universidad Rey Juan Carlos