ICSE 2024
Fri 12 - Sun 21 April 2024 Lisbon, Portugal
Fri 19 Apr 2024 11:00 - 11:15 at Grande Auditório - LLM, NN and other AI technologies 5 Chair(s): Sarah Nadi

Exploratory testing leverages the tester’s knowledge and creativity to design test cases for effectively uncovering system-level bugs from the end user’s perspective. Researchers have worked on test scenario generation to support exploratory testing based on a system knowledge graph, enriched with scenario and oracle knowledge from bug reports. Nevertheless, the adoption of this approach is hindered by difficulties in handling bug reports of inconsistent quality and varied expression styles, along with the infeasibility of the generated test scenarios. To overcome these limitations, we utilize the superior natural language understanding (NLU) capabilities of Large Language Models (LLMs) to construct a System KG of User Tasks and Failures (SysKG-UTF). Leveraging the system and bug knowledge from the KG, along with the logical reasoning capabilities of LLMs, we generate test scenarios with high feasibility and coherence. Particularly, we design chain-of-thought (CoT) reasoning to extract human-like knowledge and logical reasoning from LLMs, simulating a developer’s process of validating test scenario feasibility. Our evaluation shows that our approach significantly enhances the KG construction, particularly for bug reports with low quality. Furthermore, our approach generates test scenarios with high feasibility and coherence. The user study further proves the effectiveness of our generated test scenarios in supporting exploratory testing. Specifically, 8 participants find 36 bugs from 8 seed bugs in two hours using our test scenarios, a significant improvement over the 21 bugs found by the state-of-the-art baseline.

Fri 19 Apr

Displayed time zone: Lisbon change

11:00 - 12:30
LLM, NN and other AI technologies 5Software Engineering Education and Training / Software Engineering in Practice / Research Track at Grande Auditório
Chair(s): Sarah Nadi New York University Abu Dhabi, UAE; University of Alberta, Canada
11:00
15m
Talk
Enhancing Exploratory Testing by Large Language Model and Knowledge Graph
Research Track
Yanqi Su Australian National University, Dianshu Liao Australian National University, Zhenchang Xing , Qing Huang School of Computer Information Engineering, Jiangxi Normal University, Mulong Xie CSIRO's Data61, Qinghua Lu CSIRO’s Data61, Xiwei (Sherry) Xu CSIRO’s Data61
11:15
15m
Talk
LLMParser: An Exploratory Study on Using Large Language Models for Log Parsing
Research Track
Zeyang Ma Concordia University, An Ran Chen University of Alberta, Dong Jae Kim Concordia University, Tse-Hsun (Peter) Chen Concordia University, Shaowei Wang University of Manitoba
11:30
15m
Talk
Enhancing Text-to-SQL Translation for Financial System Design
Software Engineering in Practice
Yewei Song University of Luxembourg, Saad Ezzini Lancaster University, Xunzhu Tang University of Luxembourg, Cedric Lothritz University of Luxembourg, Jacques Klein University of Luxembourg, Tegawendé F. Bissyandé University of Luxembourg, Andrey Boytsov Banque BGL BNP Paribas, Ulrick Ble Banque BGL BNP Paribas, Anne Goujon Banque BGL BNP Paribas
11:45
15m
Talk
Towards Building AI-CPS with NVIDIA Isaac Sim: An Industrial Benchmark and Case Study for Robotics Manipulation
Software Engineering in Practice
Zhehua Zhou University of Alberta, Jiayang Song University of Alberta, Xuan Xie University of Alberta, Zhan Shu University of Alberta, Lei Ma The University of Tokyo / University of Alberta, Dikai Liu NVIDIA AI Tech Centre, Jianxiong Yin NVIDIA AI Tech Centre, Simon See NVIDIA AI Tech Centre
12:00
15m
Talk
Let's Ask AI About Their Programs: Exploring ChatGPT's Answers To Program Comprehension Questions
Software Engineering Education and Training
Teemu Lehtinen Aalto University, Charles Koutcheme Aalto University, Arto Hellas Aalto University
12:15
15m
Talk
Experience Report: Identifying common misconceptions and errors of novice programmers with ChatGPT
Software Engineering Education and Training
Hua Leong Fwa Singapore Management University
Media Attached
Hide past events