ICSR 2025
Sun 27 - Mon 28 April 2025 Ottawa, Ontario, Canada
co-located with ICSE 2025
Sun 27 Apr 2025 16:45 - 17:15 at 204 - Session 4: Reusable models and Testing Chair(s): Dalila Tamzalit

Flaky tests refer to test cases that exhibit inconsistent behaviors across multiple executions, potentially passing or failing unpredictably. They are frequently associated with suboptimal design practices that testers may utilize when crafting test cases, which undermine the quality of software testing. Identifying the root causes of flaky tests is crucial for fixing them. Currently, inspired by the success of the Large Language Models (LLMs), researchers leverage the pre-trained language model to embed flaky test code as vectors and predict its root cause category based on vector similarity measures. However, such code embeddings generated by LLM mainly focus on capturing general semantic features but lack sufficient comprehension of the behavioral patterns involved in test scenarios, leading to the ineffectiveness of root cause identification. Test smells, which reflect poor coding practices or habits when writing test cases, provide complementary information in root cause identification of test flakiness. Therefore, this paper proposes a flaky test root cause identification method based on test smells, which leverages test smells to abstract and express behavioral patterns of test codes and integrates general semantic features extracted via vector embeddings to enhance the feature representation of flaky tests. Furthermore, to capture the complex nonlinear relationships between test smell features and code embeddings, a Feedforward Neural Network is constructed to categorize the root causes of test flakiness. To validate the effectiveness of our method, we performed evaluations on a dataset consisting of 451 Java flaky test cases. The experimental results indicate that our method achieves an F1-score of 80%, which is 7% higher than that of the baseline model that does not incorporate test smells.

Predicting the Root Cause of Flaky Tests Based on Test Smells (Predicting_the_Root_Cause_of_Flaky_Tests_Based_on_Test_Smells.pdf)955KiB

Sun 27 Apr

Displayed time zone: Eastern Time (US & Canada) change

16:00 - 17:30
Session 4: Reusable models and TestingICSR at 204
Chair(s): Dalila Tamzalit Nantes Université
16:00
30m
Paper
DockInsight: A Knowledge-Augmented Dependency Extraction Approach for Dockerfile
ICSR
Zhiling Zhu Zhejiang University of Technology, Tieming Chen Zhejiang University of Technology, Yunjin Zhong Zhejiang University of Technology, Qijie Song Zhejiang University of Technology
16:30
15m
Paper
Porting an LLM based Application from ChatGPT to an On-Premise Environment
ICSR
Teemu Paloniemi University of Jyväskylä, Manu Setälä Solita Oy, Tommi Mikkonen University of Jyvaskyla
Pre-print
16:45
30m
Paper
Predicting the Root Cause of Flaky Tests Based on Test Smells
ICSR
Jing Wang College of Information Science and Technology, Beijing University of Chemical Technology, Weixi Zhang College of Information Engineering, Beijing Institute of Petrochemical Technology, Beijing, China, Weixi Zhang College of Information Engineering, Beijing Institute of Petrochemical Technology, Beijing, China, Ruilian Zhao Beijing University of Chemical Technology, Ying Shang Beijing University of Chemical Technology
File Attached
17:15
15m
Paper
Towards Patterns for a Reference Assurance Case for Autonomous Inspection Robots
ICSR
Dhaminda B. Abeywickrama Department of Computer Science, The University of Manchester, UK, Michael Fisher University of Manchester, UK, Frederic Wheeler Regulatory Support Directorate, Amentum, Louise Dennis The University of Manchester