During test execution, automated software tests can interfere, i.e., their results can deviate depending on their (possibly interleaved) execution order. Such interference imposes severe restrictions on regression testing, when execution order is not or cannot be controlled for, as they can lead to non-deterministic deviations of test results giving false indication of regressions in the code base. While the phenomenon has been extensively studied for Java and Python projects, it remains unclear if or how the obtained results apply for other languages with different practices of test organization and execution. Our study contributes to filling that gap by reporting results from a large-scale study on test interference in 134 C projects. To cope with the combinatorial explosion necessitated by testing all possible test orders, we propose and evaluate four novel dynamic permutation reduction strategies. As these strategies are specific to the resources that tests interfere on, rather than the language in which the code is written, we expect them to be useful for the study of test interference in other languages beyond C.

Based on the results obtained with these reductions, our results indicate that compared to Java and Python, test order dependencies are far less common in C projects.