FDG: A Precise Measurement of Fault Diagnosability Gain of Test Cases
Fri 22 Jul 2022 18:20 - 18:40 at ISSTA 1 - Session 3-13: Oracles, Models, and Measurement F Chair(s): Stefan Winter
The performance of many Fault Localisation (FL) techniques directly depends on the quality of the used test suites. Consequently, it is extremely useful to be able to precisely measure how much diagnostic power each test case can introduce when added to a test suite used for FL. Such a measure can help us not only to prioritise and select test cases to be used for FL, but also to effectively augment test suites that are too weak to be used with FL techniques. We propose FDG, a new measure of Fault Diagnosability Gain for individual test cases. The design of FDG is based on our analysis of existing metrics that are designed to prioritise test cases for better FL. Unlike other metrics, FDG exploits the ongoing FL results to emphasise the parts of the program for which more information is needed. Our evaluation of FDG with Defects4J shows that it can successfully help the augmentation of test suites for better FL. When given only a few failing test cases (2.3 test cases on average), FDG can effectively augment the given test suite by prioritising the test cases generated automatically by EvoSuite: the augmentation can improve the acc@1 and acc@10 of the FL results by 11.6x and 2.2x on average, after requiring only ten human judgements on the correctness of the assertions EvoSuite generates.
Wed 20 JulDisplayed time zone: Seoul change
07:00 - 08:20 | Session 2-1: Oracles, Models, and Measurement ETechnical Papers at ISSTA 1 Chair(s): Christoph Csallner University of Texas at Arlington | ||
07:00 20mTalk | On the Use of Evaluation Measures for Defect Prediction Studies Technical Papers DOI | ||
07:20 20mTalk | Combining Solution Reuse and Bound Tightening for Efficient Analysis of Evolving SystemsACM SIGSOFT Distinguished Paper Technical Papers DOI | ||
07:40 20mTalk | Evolution-Aware Detection of Order-Dependent Flaky Tests Technical Papers DOI | ||
08:00 20mTalk | FDG: A Precise Measurement of Fault Diagnosability Gain of Test Cases Technical Papers DOI Pre-print | ||
Fri 22 JulDisplayed time zone: Seoul change
18:00 - 19:00 | Session 3-13: Oracles, Models, and Measurement FTechnical Papers at ISSTA 1 Chair(s): Stefan Winter LMU Munich | ||
18:00 20mTalk | jTrans: Jump-Aware Transformer for Binary Code Similarity Detection Technical Papers Hao Wang Tsinghua University, Wenjie Qu Huazhong University of Science and Technology, Gilad Katz Ben-Gurion University of the Negev, Wenyu Zhu Tsinghua University, Zeyu Gao University of Science and Technology of China, Han Qiu Tsinghua University, Jianwei Zhuge Tsinghua University, Chao Zhang Tsinghua University DOI Pre-print | ||
18:20 20mTalk | FDG: A Precise Measurement of Fault Diagnosability Gain of Test Cases Technical Papers DOI Pre-print | ||
18:40 20mTalk | Hunting Bugs with Accelerated Optimal Graph Vertex Matching Technical Papers Xiaohui Zhang Renmin University of China, Yuanjun Gong Renmin University of China, Bin Liang Renmin University of China, China, Jianjun Huang Renmin University of China, China, Wei You Renmin University of China, Wenchang Shi Renmin University of China, China, Jian Zhang Institute of Software at Chinese Academy of Sciences, China DOI | ||