Data Conflicts-Guided Interleaved Thread Scheduling for Flaky Test Detection in Multithreaded Programs
Flaky tests can non-deterministically pass or fail on the same version of code. Their occurrence prevents using test results to determine if there are bugs in the program, thereby reducing the credibility of software testing. Concurrency is a main cause of flaky tests, where different concurrent thread interleavings may lead to uncertain program execution results. Essentially, only when threads with data dependencies are interleaved can inconsistent test execution results occur. But the existing work scheduled threads randomly with no guide during flaky test detection, resulting in a vast interleaving space and low efficiency. To address this issue, we propose a data conflict-guided flaky test detection approach, which prioritizes threads with data dependency for scheduling to try to alter the test results, thereby detecting flaky tests more effectively. In more detail, the dynamic execution trace of test cases is tracked in multithreaded programs. Then, by analyzing read-write operations in the trace, data conflicts are identified. On this basis, a Bayesian Network is put forward to evaluate the degree of impact of data conflicts on test flakiness, which implies potential flaky risk. That is, the greater the flaky risk, the more likely the data conflict is to cause a flaky test. So, the scheduling strategy prioritizes threads with data conflict pairs that have a significant impact on test flakiness, to trigger test flakiness as soon as possible. To verify the effectiveness of our approach, we conduct experiments on multithreaded Java programs. The experimental results show that our approach can discover the flakiness of tests within an average of 118 seconds, and the accuracy in detecting flaky tests can reach 75%.
Wed 4 DecDisplayed time zone: Beijing, Chongqing, Hong Kong, Urumqi change
16:00 - 17:30 | Session (5)Technical Track at Room 2 (Xiangshan Ballroom) Chair(s): Haoye Tian University of Melbourne | ||
16:00 30mTalk | SwFuzz: Structure-Sensitive WebAssembly Fuzzing Technical Track Jiashui Wang Zhejiang University, Ziyi Guo Northwestern University, Xinlei Ying Ant Group, Peng Qian Zhejiang University, Yan Chen Northwestern University | ||
16:30 30mTalk | Data Conflicts-Guided Interleaved Thread Scheduling for Flaky Test Detection in Multithreaded Programs Technical Track Tianzi Wang College of lnformation Science and Technology, Beijing University of chemical Technology, Beijing, China, Ruilian Zhao Beijing University of Chemical Technology, Weiwei Wang College of lnformation Science and Technology, Beijing University of chemical Technology, Beijing, China, Weixi Zhang College of Information Engineering, Beijing Institute of Petrochemical Technology, Beijing, China | ||
17:00 30mTalk | AS-Fuzzer: An Optimized ADS Fuzzing Method via Scenario Segmentation and Parallel Evolution Technical Track Fansong Chen Institute of Information Engineering, Chinese Academy of Sciences, Shenghao Lin Institute of Information Engineering, Chinese Academy of Sciences, Weicheng Lin Institute of Information Engineering, Chinese Academy of Sciences, Laile Xi Institute of Information Engineering, Chinese Academy of Sciences , Yongji Liu Institute of Information Engineering, Chinese Academy of Sciences, Lun Li Institute of Information Engineering at Chinese Academy of Sciences, Hongsong Zhu Institute of Information Engineering at Chinese Academy of Sciences; University of Chinese Academy of Sciences |