Write a Blog >>
ICST 2020
Sat 24 - Wed 28 October 2020 Porto, Portugal

Modern software engineers automate as many tests as possible. Test automation allows tests to be run hundreds or thousands of times: hourly, daily, and sometimes continuously. This saves time and money, ensures reproducibility, and ultimately leads to software that is better and cheaper. Automated tests must include code to check that the output of the program on the test matches expected behavior. This code is called the test oracle and is typically implemented in assertions that flag the test as passing if the assertion evaluates to true and failing if not. Since automated tests require programming, many problems can occur. Some lead to false positives, where incorrect behavior is marked as correct, and others to false negatives, where correct behavior is marked as incorrect. This paper identifies and studies a common problem where test assertions are written incorrectly, leading to incorrect behavior that is not recognized. We call these tests blind because the test does not see the incorrect behavior. Blind tests cause false positives, essentially wasting the tests. This paper presents results from several human-based studies to assess the frequency of blind tests with different software and different populations of users. In our studies, the percent of blind tests ranged from a low of 39% to a high of 95%.

Mon 26 Oct

Displayed time zone: Lisbon change

15:30 - 17:00
RT9 - Empirical Analysis and Program RepairResearch Papers at Farfetch (D. Maria) +11h
Chair(s): Corina S. Pasareanu Carnegie Mellon University Silicon Valley, NASA Ames Research Center
15:30
30m
Talk
An Empirical Analysis of Blind Tests
Research Papers
Kesina Baral George Mason University, Jeff Offutt George Mason University
Link to publication DOI
16:00
30m
Talk
A Family of Experiments to Assess the Impact of Page Object Pattern in Web Test Suite Development
Research Papers
Maurizio Leotta Università di Genova, Matteo Biagiola Fondazione Bruno Kessler (FBK), Filippo Ricca Università di Genova, Mariano Ceccato University of Verona, Paolo Tonella Università della Svizzera Italiana (USI)
Link to publication DOI
16:30
30m
Talk
Human-In-The-Loop Automatic Program Repair
Research Papers
Marcel Böhme Monash University, Australia, Charaka Geethal Monash University, Van-Thuan Pham Monash University
Link to publication DOI

Tue 27 Oct

Displayed time zone: Lisbon change

02:30 - 04:00
RT9 - Empirical Analysis and Program RepairResearch Papers at Farfetch (D. Maria)
Chair(s): Corina S. Pasareanu Carnegie Mellon University Silicon Valley, NASA Ames Research Center
02:30
30m
Talk
An Empirical Analysis of Blind Tests
Research Papers
Kesina Baral George Mason University, Jeff Offutt George Mason University
Link to publication DOI
03:00
30m
Talk
A Family of Experiments to Assess the Impact of Page Object Pattern in Web Test Suite Development
Research Papers
Maurizio Leotta Università di Genova, Matteo Biagiola Fondazione Bruno Kessler (FBK), Filippo Ricca Università di Genova, Mariano Ceccato University of Verona, Paolo Tonella Università della Svizzera Italiana (USI)
Link to publication DOI
03:30
30m
Talk
Human-In-The-Loop Automatic Program Repair
Research Papers
Marcel Böhme Monash University, Australia, Charaka Geethal Monash University, Van-Thuan Pham Monash University
Link to publication DOI