Call Me Maybe: Using NLP to Automatically Generate Unit Test Cases Respecting Temporal Constraints
A class may need to obey temporal properties in order to function correctly. For example, the correct usage protocol for an iterator is to always check whether there is a next element before asking for it; iterating over a collection when there are no items left leads to a NoSuchElementException. Automatic test case generation tools such as Randoop and Evo- Suite do not have any notion of these temporal properties. Gener- ating test cases by randomly invoking methods on a new instance of the class under test may raise run-time exceptions that do not necessarily expose software faults, but are rather a consequence of violations of temporal properties.
This paper presents Call Me Maybe, a novel technique that uses natural language processing to analyze Javadoc comments to identify temporal properties. This information can guide a test case generator towards executing sequences of method calls that respect the temporal properties. Our evaluation on 73 subjects from seven popular Java systems show that Randoop flags over 10K fewer false negatives and enriches over 12K correctly failing test cases due to violations of temporal properties with clear explanation that can help software developers.
Tue 11 OctDisplayed time zone: Eastern Time (US & Canada) change
10:30 - 12:30 | Technical Session 2 - Debugging and TroubleshootingResearch Papers / Industry Showcase / Late Breaking Results at Banquet A Chair(s): Andrew Begel Carnegie Mellon University, Software and Societal Systems Department | ||
10:30 20mResearch paper | Call Me Maybe: Using NLP to Automatically Generate Unit Test Cases Respecting Temporal Constraints Research Papers Arianna Blasi Meta; prev. Università della Svizzera italiana, Alessandra Gorla IMDEA Software Institute, Michael D. Ernst University of Washington, Mauro Pezze USI Lugano; Schaffhausen Institute of Technology | ||
10:50 20mResearch paper | CoditT5: Pretraining for Source Code and Natural Language Editing Research Papers Jiyang Zhang University of Texas at Austin, Sheena Panthaplackel UT Austin, Pengyu Nie University of Texas at Austin, Junyi Jessy Li University of Texas at Austin, USA, Milos Gligoric University of Texas at Austin Pre-print | ||
11:10 20mIndustry talk | Automated Identification of Security-Relevant Configuration Settings Using NLP Industry Showcase Patrick Stöckle Technical University of Munich (TUM), Theresa Wasserer Technical University of Munich, Bernd Grobauer Siemens AG, Alexander Pretschner TU Munich Pre-print | ||
11:30 20mResearch paper | Is this Change the Answer to that Problem? Correlating Descriptions of Bug and Code Changes for Evaluating Patch Correctness Research Papers Haoye Tian University of Luxembourg, Xunzhu Tang University of Luxembourg, Andrew Habib SnT, University of Luxembourg, Shangwen Wang National University of Defense Technology, Kui Liu Huawei Software Engineering Application Technology Lab, Xin Xia Huawei Software Engineering Application Technology Lab, Jacques Klein University of Luxembourg, Tegawendé F. Bissyandé SnT, University of Luxembourg Pre-print | ||
11:50 10mPaper | A real-world case study for automated ticket team assignment using natural language processing and explainable modelsVirtual Late Breaking Results | ||