Tue 27 Oct 2020 02:30 - 03:00 at Farfetch (D. Maria) - RT9 - Empirical Analysis and Program Repair Chair(s): Corina S. Pasareanu
Modern software engineers automate as many tests as possible. Test automation allows tests to be run hundreds or thousands of times: hourly, daily, and sometimes continuously. This saves time and money, ensures reproducibility, and ultimately leads to software that is better and cheaper. Automated tests must include code to check that the output of the program on the test matches expected behavior. This code is called the test oracle and is typically implemented in assertions that flag the test as passing if the assertion evaluates to true and failing if not. Since automated tests require programming, many problems can occur. Some lead to false positives, where incorrect behavior is marked as correct, and others to false negatives, where correct behavior is marked as incorrect. This paper identifies and studies a common problem where test assertions are written incorrectly, leading to incorrect behavior that is not recognized. We call these tests blind because the test does not see the incorrect behavior. Blind tests cause false positives, essentially wasting the tests. This paper presents results from several human-based studies to assess the frequency of blind tests with different software and different populations of users. In our studies, the percent of blind tests ranged from a low of 39% to a high of 95%.
Mon 26 OctDisplayed time zone: Lisbon change
15:30 - 17:00 | RT9 - Empirical Analysis and Program RepairResearch Papers at Farfetch (D. Maria) +11h Chair(s): Corina S. Pasareanu Carnegie Mellon University Silicon Valley, NASA Ames Research Center | ||
15:30 30mTalk | An Empirical Analysis of Blind Tests Research Papers Link to publication DOI | ||
16:00 30mTalk | A Family of Experiments to Assess the Impact of Page Object Pattern in Web Test Suite Development Research Papers Maurizio Leotta Università di Genova, Matteo Biagiola Fondazione Bruno Kessler (FBK), Filippo Ricca Università di Genova, Mariano Ceccato University of Verona, Paolo Tonella Università della Svizzera Italiana (USI) Link to publication DOI | ||
16:30 30mTalk | Human-In-The-Loop Automatic Program Repair Research Papers Marcel Böhme Monash University, Australia, Charaka Geethal Monash University, Van-Thuan Pham Monash University Link to publication DOI |
Tue 27 OctDisplayed time zone: Lisbon change
02:30 - 04:00 | RT9 - Empirical Analysis and Program RepairResearch Papers at Farfetch (D. Maria) Chair(s): Corina S. Pasareanu Carnegie Mellon University Silicon Valley, NASA Ames Research Center | ||
02:30 30mTalk | An Empirical Analysis of Blind Tests Research Papers Link to publication DOI | ||
03:00 30mTalk | A Family of Experiments to Assess the Impact of Page Object Pattern in Web Test Suite Development Research Papers Maurizio Leotta Università di Genova, Matteo Biagiola Fondazione Bruno Kessler (FBK), Filippo Ricca Università di Genova, Mariano Ceccato University of Verona, Paolo Tonella Università della Svizzera Italiana (USI) Link to publication DOI | ||
03:30 30mTalk | Human-In-The-Loop Automatic Program Repair Research Papers Marcel Böhme Monash University, Australia, Charaka Geethal Monash University, Van-Thuan Pham Monash University Link to publication DOI |