Are Unit and Integration Test Definitions Still Valid for Modern Java Projects? An Empirical Study on Open-Source Projects
Tue 27 Oct 2020 01:45 - 02:15 at Farfetch (D. Maria) - Journal First 1 Chair(s): Fabiano Ferrari
Context: Unit and integration testing are popular testing techniques. However, while the software development context evolved over time, the definitions remained unchanged. There is no empirical evidence, if these commonly used definitions still fit to modern software development.
Objective: We analyze, if the existing standard definitions of unit and integration tests are still valid in modern software development contexts. Hence, we analyze if unit and integration tests detect different types of defects, as expected from the standard literature.
Method: We classify 38,782 test cases into unit and integration tests according to the definition of the IEEE and use mutation testing to assess their defect detection capabilities. All integrated mutations are classified into five different defect types. Afterwards, we evaluate if there are any statistically significant differences in the results between unit and integration tests.
Results: We could not find any evidence that one test type is more capable of detecting certain defect types than the other one. Our results suggest that the currently used definitions do not fit modern software development contexts.
Conclusions: This finding implies that we need to reconsider the definitions of unit and integration tests and suggest that the current property-based definitions may be exchanged with usage-based definitions.
Mon 26 OctDisplayed time zone: Lisbon change
13:45 - 15:15 | Journal First 1Journal-First Papers at Farfetch (D. Maria) +11h Chair(s): Maurício Aniche Delft University of Technology, Netherlands | ||
13:45 30mTalk | Modeling Foundations for Executable Model-Based Testing of Self-Healing Cyber-Physical Systems Journal-First Papers Tao Ma Simula Research Laboratory, Shaukat Ali Simula Research Laboratory, Tao Yue Nanjing University of Aeronautics and Astronautics Link to publication DOI | ||
14:15 30mTalk | Empirical evaluation of mutation-based test case prioritization techniques Journal-First Papers Donghwan Shin University of Luxembourg, Shin Yoo Korea Advanced Institute of Science and Technology, Mike Papadakis University of Luxembourg, Luxembourg, Doo-Hwan Bae Korea Advanced Institute of Science and Technology Link to publication DOI | ||
14:45 30mTalk | Are Unit and Integration Test Definitions Still Valid for Modern Java Projects? An Empirical Study on Open-Source Projects Journal-First Papers Fabian Trautsch University of Göttingen, Steffen Herbold University of Göttingen, Jens Grabowski University of Göttingen Link to publication DOI |
Tue 27 OctDisplayed time zone: Lisbon change
00:45 - 02:15 | Journal First 1Journal-First Papers at Farfetch (D. Maria) Chair(s): Fabiano Ferrari Federal University of São Carlos | ||
00:45 30mTalk | Modeling Foundations for Executable Model-Based Testing of Self-Healing Cyber-Physical Systems Journal-First Papers Tao Ma Simula Research Laboratory, Shaukat Ali Simula Research Laboratory, Tao Yue Nanjing University of Aeronautics and Astronautics Link to publication DOI | ||
01:15 30mTalk | Empirical evaluation of mutation-based test case prioritization techniques Journal-First Papers Donghwan Shin University of Luxembourg, Shin Yoo Korea Advanced Institute of Science and Technology, Mike Papadakis University of Luxembourg, Luxembourg, Doo-Hwan Bae Korea Advanced Institute of Science and Technology Link to publication DOI | ||
01:45 30mTalk | Are Unit and Integration Test Definitions Still Valid for Modern Java Projects? An Empirical Study on Open-Source Projects Journal-First Papers Fabian Trautsch University of Göttingen, Steffen Herbold University of Göttingen, Jens Grabowski University of Göttingen Link to publication DOI |