Code, Test and Coverage Evolution in Mature Software Systems: Changes over the Past Decade
This program is tentative and subject to change.
Despite the central role of test suites in the software development process, there is surprisingly limited information on how code and tests co-evolve to exercise different parts of the codebase.
Ten years ago, the Covrig project examined the code, test, and coverage evolution in six mature open-source projects, spanning a combined development time of twelve years. In this study, we significantly expand the analysis to nine mature projects and a combined period of 78 years of development time. Our focus is on understanding how development practices have changed and how these changes have impacted the way in which software is tested.
We report on the co-evolution of code and tests; the adoption of CI, coverage, and fuzzing services; the changes to the overall code coverage achieved by developer test suites; the distribution of patch coverage across revisions; how different kinds of code changes impact coverage; and the occurrence and evolution of flaky tests.
Our large-scale study paints a mixed picture in terms of how software development and testing have changed over the past ten years. While developers put more emphasis on software testing and the overall code coverage achieved by developer test suites has increased in most projects, coverage and fuzzing services are not widely adopted, many patches are still poorly tested, and the fraction of flaky tests has increased.
This program is tentative and subject to change.
Fri 4 AprDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
14:00 - 15:28 | Advances in Software TestingResearch Papers / Short Papers, Vision and Emerging Results / Industry / Education at Room A1 Chair(s): Annibale Panichella Delft University of Technology | ||
14:00 15mTalk | Code, Test and Coverage Evolution in Mature Software Systems: Changes over the Past Decade Research Papers | ||
14:15 15mTalk | Coverage Metrics for T-Wise Feature Interactions Research Papers Sabrina Böhm University of Ulm, Germany, Tim Jannik Schmidt University of Ulm, TU Braunschweig, Germany, Sebastian Krieter TU Braunschweig, Germany, Tobias Pett Karlsruhe Institute for Technology (KIT), Germany, Thomas Thüm TU Braunschweig, Germany, Malte Lochau University of Siegen | ||
14:30 15mTalk | LLM-based Labelling of Recorded Automated GUI-based Test cases Industry Diogo Buarque Franzosi Blekinge Institute of Technology, Emil Alégroth Blekinge Institute of Technology, Maycel Isaac Synteda | ||
14:45 15mTalk | A Tool-assisted Training Approach for Empowering Localization and Internationalization Testing Proficiency Education Maria Couto Federal University of Pernambuco, Breno Miranda Centro de Informática at Universidade Federal de Pernambuco, Kiev Gama Universidade Federal de Pernambuco | ||
15:00 7mShort-paper | Pre-trained Models for Bytecode Instructions Short Papers, Vision and Emerging Results Donggyu Kim Handong Global University, Taemin Kim , Jiho Shin York University, Song Wang York University, Heeyoul Choi , Jaechang Nam Handong Global University Pre-print | ||
15:07 7mShort-paper | EnCus: Customizing Search Space for Automated Program Repair Short Papers, Vision and Emerging Results Seongbin Kim , Sechang Jang , Jindae Kim Seoul National University of Science and Technology, Jaechang Nam Handong Global University Pre-print | ||
15:14 7mTalk | Test Generation from Use Case Specifications for IoT Systems: Custom, LLM-Based, and Hybrid Approaches Short Papers, Vision and Emerging Results Zacharie Chenail-Larcher École de technologie supérieure (ÉTS), Jean Baptiste Minani Concordia University, Montreal, Canada, Naouel Moha École de Technologie Supérieure (ETS) | ||