Developers write assertions in their test cases to validate program behaviour. Although these assertions are not complex, they are written in code that must be correct in order for the behavioural validation to be useful. In this work we show that most test assertions are straightforward and repetitive, making them well-suited to automation. We demonstrate that this automation can also reduce developer effort and improve test case assertion quality. This paper investigates over 33,000 assertions from 105 projects and finds that assertions fall into a small set of high-level categories. To automate test case assertions, we created AutoAssert, a human-in-the-loop tool that is able to fit into a developer’s test-writing workflow. The tool automatically generates test assertions for TypeScript and JavaScript unit tests. AutoAssert is a dynamic approach: it injects code into the test case as it runs to ensure that the generated assertions will not fail at runtime for the developer. The developer is then free to modify the generated assertions as they see fit. We show that AutoAssert is able to generate over 80% of test assertions in a sample of over 1,000 assertions. Through a field study with 17 industrial developers, we show that developers find assertion generation to be useful when applied to their own projects.
This Journal First paper was accepted to TSE in October, 2022 and has not appeared in any other venue. The approach describes a tool that could be described and demonstrated in the ICSE Journal First track. The talk would be the first author’s first, and only, ICSE presentation.
Fri 19 MayDisplayed time zone: Hobart change
13:45 - 15:15 | Code generationJournal-First Papers / Technical Track at Meeting Room 101 Chair(s): Iftekhar Ahmed University of California at Irvine | ||
13:45 15mTalk | Learning Deep Semantics for Test Completion Technical Track Pengyu Nie University of Texas at Austin, Rahul Banerjee The University of Texas at Austin, Junyi Jessy Li University of Texas at Austin, USA, Raymond Mooney The University of Texas at Austin, Milos Gligoric University of Texas at Austin | ||
14:00 15mTalk | Dynamic Human-in-the-Loop Assertion Generation Journal-First Papers Lucas Zamprogno University of British Columbia, Braxton Hall University of British Columbia, Reid Holmes University of British Columbia, Joanne M. Atlee University of Waterloo | ||
14:15 15mTalk | SkCoder: A Sketch-based Approach for Automatic Code Generation Technical Track Jia Li Peking University, Yongmin Li Peking University, Ge Li Peking University, Zhi Jin Peking University, Xing Hu Zhejiang University Pre-print | ||
14:30 15mTalk | An Empirical Comparison of Pre-Trained Models of Source Code Technical Track Changan Niu Software Institute, Nanjing University, Chuanyi Li Nanjing University, Vincent Ng Human Language Technology Research Institute, University of Texas at Dallas, Richardson, TX 75083-0688, Dongxiao Chen Software Institute, Nanjing University, Jidong Ge Nanjing University, Bin Luo Nanjing University Pre-print | ||
14:45 15mTalk | On the Robustness of Code Generation Techniques: An Empirical Study on GitHub Copilot Technical Track Antonio Mastropaolo Università della Svizzera italiana, Luca Pascarella ETH Zurich, Emanuela Guglielmi University of Molise, Matteo Ciniselli Università della Svizzera Italiana, Simone Scalabrino University of Molise, Rocco Oliveto University of Molise, Gabriele Bavota Software Institute, USI Università della Svizzera italiana | ||
15:00 15mTalk | Source Code Recommender Systems: The Practitioners' Perspective Technical Track Matteo Ciniselli Università della Svizzera Italiana, Luca Pascarella ETH Zurich, Emad Aghajani Software Institute, USI Università della Svizzera italiana, Simone Scalabrino University of Molise, Rocco Oliveto University of Molise, Gabriele Bavota Software Institute, USI Università della Svizzera italiana | ||