Leveraging LLMs for User Stories in AI Systems: UStAI Dataset
AI systems are gaining widespread adoption across various sectors and domains. Creating high-quality AI system requirements is crucial for aligning the AI system with business goals and consumer values and for social responsibility. However, with the uncertain nature of AI systems and the heavy reliance on sensitive data, more research is needed to address the elicitation and analysis of AI systems requirements. With the proprietary nature of many AI systems, there is a lack of open-source requirements artifacts and technical requirements documents for AI systems, limiting broader research and investigation. With Large Language Models (LLMs) emerging as a promising alternative to human-generated text, this paper investigates the potential use of LLMs to generate user stories for AI systems based on abstracts from scholarly papers. We conducted an empirical evaluation using three LLMs and generated $1260$ user stories from $42$ abstracts from $26$ domains. We assess their quality using the Quality User Story (QUS) framework. Moreover, we identify relevant non-functional requirements and ethical principles. Our analysis demonstrates that the investigated LLMs can generate user stories inspired by the needs of various stakeholders, offering a promising approach for generating user stories for research purposes and for aiding in the early requirements elicitation phase of AI systems. We have compiled and curated a collection of stories generated by various LLMs into a dataset (UStAI), which is now publicly available for use.
Thu 26 JunDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
11:00 - 13:00 | |||
11:00 5mDay opening | Opening PROMISE 2025 | ||
11:06 59mKeynote | Keynote 1 (Dr. Jacques Klein) PROMISE 2025 Jacques Klein University of Luxembourg | ||
12:06 14mTalk | LO2: Microservice API Anomaly Dataset of Logs and Metrics PROMISE 2025 Alexander Bakhtin University of Oulu, Jesse Nyyssölä University of Helsinki, Yuqing Wang University of Helsinki, Finland, Noman Ahmad University of Oulu, Ke Ping University of Helsinki, Matteo Esposito University of Oulu, Mika Mäntylä University of Helsinki and University of Oulu, Davide Taibi University of Oulu | ||
12:21 14mTalk | LogLSHD: Fast Log Parsing with Locality-Sensitive Hashing and Dynamic Time Warping PROMISE 2025 Shu-Wei Huang Polytechnique Montréal, Xingfang Wu Polytechnique Montréal, Heng Li Polytechnique Montréal | ||
12:36 14mTalk | Leveraging LLMs for User Stories in AI Systems: UStAI Dataset PROMISE 2025 Asma Yamani King Fahd University of Petroleum and Minerals, Malak Baslyman King Fahd University of Petroleum & Minerals, Moataz Ahmed King Fahd University of Petroleum and Minerals |
Vega is close to the registration desk.
Facing the registration desk, its entrance is on the left, close to the hotel side entrance.