ISSTA 2019
Mon 15 - Fri 19 July 2019 Beijing, China
Thu 18 Jul 2019 11:45 - 12:07 at Grand Ballroom - Regression Testing Chair(s): Dan Hao

The execution latency of a test suite strongly depends on the degree of concurrency with which test cases are being executed. However, if test cases have not been designed for concurrent execution, they may interfere, which can lead to result deviations compared to sequential execution. To prevent such interferences, each test case can be provided with an isolated execution environment, but this entails performance overheads that diminish the merit of parallel testing. In this paper, we present a large-scale analysis of the Debian Buster package repository, showing that existing test suites in C projects make limited use of parallelization. We then present an approach to (a) analyze the potential of C test suites for safe concurrent execution, i.e., result invariance compared to sequential execution, and (b) execute tests concurrently with different parallelization strategies using processes or threads if it is found to be safe in step (a). Using this approach on nine C projects, we find that most of them cannot safely execute tests in parallel due to unsafe test code or unsafe usage of shared variables or files within the program code. The parallel execution of tests shows a significant acceleration over sequential execution for most projects. We find that multi-threading rarely outperforms multi-processing. Finally, we observe that the lack of a common test framework for C leaves testers with make as the standard driver for running tests, which introduces unnecessary performance overheads for test execution.

Thu 18 Jul
Times are displayed in time zone: Beijing, Chongqing, Hong Kong, Urumqi change

11:00 - 12:30: Regression TestingTechnical Papers at Grand Ballroom
Chair(s): Dan HaoPeking University
11:00 - 11:22
Root Causing Flaky Tests in a Large-scale Industrial Setting
Technical Papers
Wing LamUniversity of Illinois at Urbana-Champaign, Patrice GodefroidMicrosoft Research, Suman NathMicrosoft Corporation, Anirudh SanthiarIndian Institute of Science, Suresh Thummalapenta
11:22 - 11:45
Mitigating the Effects of Flaky Tests on Mutation Testing
Technical Papers
August ShiUniversity of Illinois at Urbana-Champaign, Jonathan BellGeorge Mason University, Darko MarinovUniversity of Illinois at Urbana-Champaign
Pre-print Media Attached
11:45 - 12:07
Assessing the State and Improving the Art of Parallel Testing for CArtifacts ReusableArtifacts Functional
Technical Papers
Oliver SchwahnTU Darmstadt, Nicolas CoppikTU Darmstadt, Stefan WinterTU Darmstadt, Neeraj Suri
12:07 - 12:30
Failure Clustering Without Coverage
Technical Papers