Thu 18 Aug 2022 21:40 - 22:10 at Quokka - Regulations Chair(s): Travis Breaux

Software systems are increasingly subject to regulatory compliance. Extracting compliance requirements from regulations is challenging. Ideally, locating compliance-related information in a regulation requires a joint effort from requirements engineers and legal experts, whose availability is limited. However, regulations are typically long documents spanning hundreds of pages, containing legal jargon, applying complicated natural language structures, and including cross-references, thus making their analysis effort-intensive. In this paper, we propose an automated question-answering (QA) approach that assists requirements engineers in finding the legal text passages relevant to compliance requirements. Our approach utilizes large-scale language models fine-tuned for QA, including BERT and three variants. We evaluate our approach on 107 question-answer pairs, manually curated by subject-matter experts, for four different European regulatory documents. Among these documents is the general data protection regulation (GDPR) – a major source for privacy-related requirements. Our empirical results show that, in ~94% of the cases, our approach finds the text passage containing the answer to a given question among the top five passages that our approach marks as most relevant. Further, our approach successfully demarcates, in the selected passage, the right answer with an average accuracy of ~91%.

Thu 18 Aug

Displayed time zone: Hobart change

21:40 - 22:40
RegulationsJournal-First / Research Papers at Quokka
Chair(s): Travis Breaux Carnegie Mellon University
Automated Question Answering for Improved Understanding of Compliance Requirements: A Multi-Document StudyBest Paper
Research Papers
Sallam Abualhaija University of Luxembourg, Chetan Arora Deakin University, Amin Sleimi SnT, University of Luxembourg, Lionel Briand University of Luxembourg; University of Ottawa
GoRIM: A Model-Driven Method for Enhancing Regulatory Intelligence
Okhaide Akhigbe University of Ottawa, Daniel Amyot University of Ottawa, Gregory Richards University of Ottawa, Lysanne Lessard University of Ottawa