Test Behaviors, Not Methods! Detecting Tests Obsessed by Methods
Best testing practices state that tests should verify a single functionality or behavior of the system. Tests that verify multiple behaviors are harder to understand, lack focus, and are more coupled to the production code. An attempt to identify this issue is the test smell Eager Test, which aims to capture tests that verify too much functionality based on the number of production method calls. Unfortunately, prior research suggests that counting production method calls is an inaccurate measure, as these calls do not reliably serve as a proxy for functionality. We envision a complementary solution based on runtime analysis: we hypothesize that some tests that verify multiple behaviors will likely cover multiple paths of the same production methods. Thus, we propose a novel test smell named Test Obsessed by Method, a test method that covers multiple paths of a single production method. We provide an initial empirical study to explore the presence of this smell in 2,054 tests provided by 12 test suites of the Python Standard Library. (1) We detect 44 Tests Obsessed by Methods in 11 of the 12 test suites. (2) Each smelly test verifies a median of two behaviors of the production method. (3) The 44 smelly tests could be split into 118 novel tests. (4) 23% of the smelly tests have code comments recognizing that distinct behaviors are being tested. We conclude by discussing benefits, limitations, and further research.
Sun 12 AprDisplayed time zone: Brasilia, Distrito Federal, Brazil change
16:00 - 17:30 | Session 3 - Tools and Interfaces for Software AnalysisEarly Research Achievements (ERA) / Replications and Negative Results (RENE) / Tool Demonstration / ICPC Program / Research Track at Europa II Chair(s): Nicole Novielli University of Bari | ||
16:00 10mTalk | A Comparative Study of Dual-Monitor and Augmented-Reality Interfaces for Digital-Twin-Based Debugging Research Track Lucas Kreber Trier University, Nico Feld Trier University, Stephan Diehl Computer Science, University Trier, Germany, Benjamin Weyers Trier University | ||
16:10 5mTalk | Understanding CI/CD Workflow Runs Through Interactive and Animated Visualizations Early Research Achievements (ERA) Pablo Valenzuela-Toledo University of Bern, Universidad de La Frontera, Sebastiano Panichella University of Bern, Timo Kehrer University of Bern | ||
16:15 5mTalk | CrossLink: A Tool for Cross-Language (Java-C/C++) Code Comprehension Tool Demonstration Ajinkya Sawarkar Indian Institute of Technology Tirupati, Sridhar Chimalakonda Indian Institute of Technology Tirupati, Anirudh Arrepu Indian Institute of Technology Tirupati Media Attached File Attached | ||
16:20 5mTalk | BF-CBOM: Uncovering Cryptographic Assets Through Comparative CBOM Analysis at Scale Tool Demonstration DOI Pre-print Media Attached | ||
16:25 5mTalk | COBMaker - An Interactive Block-Based Programming Environment for COBOL Tool Demonstration Satish Pati IIT Tirupati, Raghavendra Pappu IIT Tirupati, Rahul Krishna Gaddam IIT Tirupati, Sridhar Chimalakonda Indian Institute of Technology Tirupati Media Attached | ||
16:30 5mTalk | CPPJoules: An Energy Measurement Tool for C++ Tool Demonstration Shivadharshan S Indian Institute of Technology Tirupati, Akilesh P Indian Institute of Technology Tirupati, Rajrupa Chattaraj Indian Institute of Technology Tirupati, India, Sridhar Chimalakonda Indian Institute of Technology Tirupati Media Attached | ||
16:35 10mTalk | Can We Spot Energy Regressions Using Developers Tests? An Industrial Replication Replications and Negative Results (RENE) Louay Khrouf Berger-Levrault, Anas Shatnawi Berger-Levrault, Romain Rouvoy Univ. Lille / Inria / IUF | ||
16:45 10mTalk | Evaluating the use of Augmented Reality for Dependency Graph Analysis: A Controlled Experiment Replications and Negative Results (RENE) Juan Pablo Sandoval Alcocer Pontificia Universidad Católica de Chile, Dussan Freire-Pozo Centro de Investigación en Ciencias Exactas e Ingenierías, Universidad Católica Boliviana, Tiara Rojas-Stambuk Centro de Investigación en Ciencias Exactas e Ingenierías, Universidad Católica Boliviana, Alison Fernandez-Blanco Pontificia Universidad Católica de Chile, Leonel Merino Pontificia Universidad Católica de Chile | ||
16:55 5mTalk | Test Behaviors, Not Methods! Detecting Tests Obsessed by Methods Early Research Achievements (ERA) Pre-print Media Attached | ||
17:00 10mTalk | Déjà Vu: A Replication Study on Code Smells and Faults in JavaScript Projects Replications and Negative Results (RENE) Kevin Pacifico University of Salerno, Giammaria Giordano University of Salerno, Valeria Pontillo Gran Sasso Science Institute, Massimiliano Di Penta University of Sannio, Italy, Damian Andrew Tamburri University of Sannio - JADS/NXP Semiconductors, Fabio Palomba University of Salerno Link to publication | ||
17:10 10mTalk | Industrial Replication of COLOR: Locator Repair in Continuous Integration and Failure Patterns under UI/DOM Evolution Replications and Negative Results (RENE) Hilal Taha University of Luxembourg, Luxembourg, Mike Papadakis University of Luxembourg, Joel Muller BGL BNP Paribas DOI File Attached | ||
17:20 10mLive Q&A | Joint QA and Discussion ICPC Program | ||