Inconsistency Detection in Natural Language Requirements using ChatGPT: a Preliminary Evaluation}
With the rapid advancement of tools based on Artificial Intelligence, it is interesting to assess their usefulness in requirements engineering. In early experiments, we have seen that chatGPT can detect inconsistency defects in natural language (NL) requirements, that traditional NLP tools cannot identify or can identify with difficulties even after domain-focused training. This study is devoted to specifically measuring the performance of chatGPT in finding inconsistency in requirements. Positive results in this respect could lead to the use of chatGPT to complement existing requirements analysis tools to automatically detect this important quality criterion. For this purpose, we consider GPT-3.5, the Generative Pretrained Transformer language model developed by OpenAI. We evaluate its ability to detect inconsistency by comparing its predictions with those obtained from expert judgments on a few example requirements documents.
| Slides of the presentation (FantechiGnesiPassaroSemini_RE@next_23.pdf) | 2.77MiB |
Thu 7 SepDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
10:45 - 12:15 | |||
10:45 30mPaper | Prompting Creative Requirements via Traceable and Adversarial Examples in Deep Learning Research Papers A: Hemanth Gudaparthi Governors State University, A: Nan Niu University of Cincinnati, A: Boyang Wang University of Cincinnati, A: Tanmay Bhowmik Mississippi State University, A: Hui Liu Beijing Institute of Technology, A: Jianzhang Zhang , A: Juha Savolainen Danfoss, A: Glen Horton University of Cincinnati, A: Sean Crowe University of Cincinnati, A: Thomas Scherz University of Cincinnati, A: Lisa Haitz University of Cincinnati | ||
11:15 30mPaper | Zero-shot Learning for Named Entity Recognition in Software Specification Documents Research Papers A: Souvick Das , A: Novarun Deb Assistant Professor, Indian Institute of Information Technology, Vadodara, A: Agostino Cortesi Ca’ Foscari University of Venice, A: Nabendu Chaki | ||
11:45 30mPaper | Inconsistency Detection in Natural Language Requirements using ChatGPT: a Preliminary Evaluation} RE@Next! Papers A: Alessandro Fantechi University of Florence, A: Stefania Gnesi Istituto di Scienza e Tecnologie dell'Informazione "Alessandro Faedo" , A: Lucia Passaro University of Pisa, A: Laura Semini Università di Pisa - Dipartimento di Informatica File Attached | ||