Context: White-box test generation is a technique used for automatically selecting test inputs using only the code under test. However, such techniques encounter challenges when applying them to complex programs. One of the challenges is handling invocations to external modules or dependencies in the code under test.
Objective: Without using proper isolation, like mocks, generated tests cannot cover all parts of the source code. Moreover, invoking external dependencies may cause unexpected side effects (e.g., accessing the file system or network). Our goal was to tackle this issue while maintaining the advantages of white-box test generation.
Method: In this paper, we present an automated approach addressing the external dependency challenge for white-box test generation. This technique isolates the test generation and execution by transforming the code under test and creating a parameterized sandbox with generated mocks. We implemented the approach in a ready-to-use tool using Microsoft Pex as a test generator, and evaluated it on 10 open-source projects from GitHub having more than 38.000 lines of code in total.
Results: The results from the evaluation indicate that if the lack of isolation hinders white-box test generation, then our approach is able to help: it increases the code coverage reached by the automatically generated tests, while it prevents invoking any external module or dependency. Also, our results act as a unique baseline for the test generation performance of Microsoft Pex on open-source projects.
Conclusion: Based on the results, our technique might serve well for handling external dependencies in white-box test generation as it increases the coverage reached in such situations, while maintaining the practical applicability of the tests generated on the isolated code.
Wed 13 OctDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
14:20 - 15:15 | Testing & Security 3Emerging Results and Vision papers / Journal-first Papers / Technical Papers at ESEM ROOM Chair(s): Robert Feldt Chalmers University of Technology, Sweden | ||
14:20 15mTalk | On (Mis)Perceptions of Testing Effectiveness: An Empirical Study Journal-first Papers Sira Vegas Universidad Politecnica de Madrid, Patricia Riofrio , Esperanza Marcos Universidad Rey Juan Carlos, Natalia Juristo Universidad Politecnica de Madrid | ||
14:35 15mTalk | Testing Smart Contracts: Which Technique Performs Best? Technical Papers Sefa Akca Uniersity of Edinburgh, Chao Peng University of Edinburgh, UK, Ajitha Rajan University of Edinburgh | ||
14:50 15mTalk | Automated isolation for white-box test generation Journal-first Papers Link to publication DOI | ||
15:05 10mTalk | Contextual Understanding and Improvement of Metamorphic Testing in Scientific Software Development Emerging Results and Vision papers Zedong Peng University of Cincinnati, Upulee Kanewala University of North Florida, Nan Niu University of Cincinnati |