Automated Handling of Anaphoric Ambiguity in Requirements: A Multi-solution Study
Wed 11 May 2022 13:10 - 13:15 at ICSE room 2-odd hours - Requirements and Reuse Chair(s): Andrea Zisman
Wed 25 May 2022 11:35 - 11:40 at Room 301+302 - Papers 6: Machine Learning with and for SE 1 Chair(s): Baishakhi Ray
Ambiguity is a pervasive issue in natural-language requirements. A common source of ambiguity in requirements is when a pronoun is anaphoric. In requirements engineering, anaphoric ambiguity occurs when a pronoun can plausibly refer to different entities and thus be interpreted differently by different readers. In this paper, we develop an accurate and practical automated approach for handling anaphoric ambiguity in requirements, addressing both ambiguity detection and anaphora interpretation. In view of the multiple competing natural language processing (NLP) and machine learning (ML) technologies that one can utilize, we simultaneously pursue six alternative solutions, empirically assessing each using a collection of ~1,350 industrial requirements. The alternative solution strategies that we consider are natural choices induced by the existing technologies; these choices frequently arise in other automation tasks involving natural-language requirements. A side-by-side empirical examination of these choices helps develop insights about the usefulness of different state-of-the-art NLP and ML technologies for addressing requirements engineering problems. For the ambiguity detection task, we observe that supervised ML outperforms both a large-scale language model, SpanBERT (a variant of BERT), as well as a solution assembled from off-the-shelf NLP coreference resolvers. In contrast, for anaphora interpretation, SpanBERT yields the most accurate solution. In our evaluation, (1) the best solution for anaphoric ambiguity detection has an average precision of ~60% and a recall of 100%, and (2) the best solution for anaphora interpretation (resolution) has an average success rate of ~98%.
Keywords: Requirements Engineering, Natural-language Requirements, Ambiguity, Natural Language Processing (NLP), Machine Learning (ML), Language Models, BERT.
Tue 10 MayDisplayed time zone: Eastern Time (US & Canada) change
04:00 - 05:00 | Requirements Engineering 1Technical Track / Journal-First Papers / SEIP - Software Engineering in Practice at ICSE room 4-even hours Chair(s): Irum Inayat National University of Computer and Emerging Sciences | ||
04:00 5mTalk | On Systematically Building a CNL for Functional Requirements Journal-First Papers Alvaro Veizaga Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg, Luxembourg, Mauricio Alferez Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg, Damiano Torre University of Luxembourg, Mehrdad Sabetzadeh University of Ottawa, Lionel Briand University of Luxembourg; University of Ottawa Link to publication DOI Pre-print Media Attached | ||
04:05 5mTalk | How Templated Requirements Specifications Inhibit Creativity in Software Engineering Journal-First Papers Rahul Mohanani University of Jyväskylä, Paul Ralph Dalhousie University, Burak Turhan University of Oulu, Vladimir Mandić Faculty of Technical Sciences, University of Novi Sad Link to publication DOI Pre-print Media Attached | ||
04:10 5mTalk | Automated Labeling and Classification of Business Rules from Software Requirement Specifications SEIP - Software Engineering in Practice Preethu Rose Anish TCS Research, Prashant Lawhatre TCS Research, Ranit Chatterjee TCS Research, Vivek Joshi TCS Research, Smita Ghaisas TCS Research File Attached | ||
04:15 5mTalk | Automated Handling of Anaphoric Ambiguity in Requirements: A Multi-solution Study Technical Track Saad Ezzini University of Luxembourg, Sallam Abualhaija University of Luxembourg, Chetan Arora Deakin University, Mehrdad Sabetzadeh University of Ottawa Pre-print Media Attached | ||
04:20 5mTalk | DeepSTL - From English Requirements to Signal Temporal Logic Technical Track Jie He Technische Universität Wien, Ezio Bartocci , Dejan Nickovic Austrian Institute of Technology, Haris Isakovic Technische Universität Wien, Radu Grosu Vienna University of Technology DOI Pre-print Media Attached |
Wed 11 MayDisplayed time zone: Eastern Time (US & Canada) change
13:00 - 14:00 | Requirements and ReuseTechnical Track / Journal-First Papers at ICSE room 2-odd hours Chair(s): Andrea Zisman The Open University | ||
13:00 5mTalk | On Systematically Building a CNL for Functional Requirements Journal-First Papers Alvaro Veizaga Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg, Luxembourg, Mauricio Alferez Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg, Damiano Torre University of Luxembourg, Mehrdad Sabetzadeh University of Ottawa, Lionel Briand University of Luxembourg; University of Ottawa Link to publication DOI Pre-print Media Attached | ||
13:05 5mTalk | Präzi: From Package-based to Call-based Dependency Networks Journal-First Papers Joseph Hejderup Endor Labs & Delft University of Technology, Moritz Beller Facebook, USA, Konstantinos Triantafyllou University of Athens, Georgios Gousios Endor Labs & Delft University of Technology DOI Pre-print Media Attached | ||
13:10 5mTalk | Automated Handling of Anaphoric Ambiguity in Requirements: A Multi-solution Study Technical Track Saad Ezzini University of Luxembourg, Sallam Abualhaija University of Luxembourg, Chetan Arora Deakin University, Mehrdad Sabetzadeh University of Ottawa Pre-print Media Attached | ||
13:15 5mTalk | DeepSTL - From English Requirements to Signal Temporal Logic Technical Track Jie He Technische Universität Wien, Ezio Bartocci , Dejan Nickovic Austrian Institute of Technology, Haris Isakovic Technische Universität Wien, Radu Grosu Vienna University of Technology DOI Pre-print Media Attached | ||
13:20 5mTalk | SnR: Constraint-Based Type Inference for Incomplete Java Code Snippets Technical Track Yiwen Dong University of Waterloo, Tianxiao Gu Tiktok Inc., Yongqiang Tian University of Waterloo, Chengnian Sun University of Waterloo Pre-print Media Attached |