Nessie: Automatically Testing JavaScript APIs with Asynchronous Callbacks
Wed 11 May 2022 20:10 - 20:15 at ICSE room 3-even hours - Validation and Verification 5 Chair(s): Saba Alimadadi
Thu 26 May 2022 11:30 - 11:35 at Room 301+302 - Papers 15: Software Testing 2 Chair(s): Rohan Padhye
Previous algorithms for feedback-directed unit test generation iteratively create sequences of API calls by executing partial tests and by adding new API calls at the end of the test. These algorithms are challenged by a popular class of APIs: higher-order functions that receive callback arguments, which often are invoked asynchronously. Existing test generators cannot effectively test such APIs because they only sequence API calls, but do not nest one call into the callback function of another. This paper presents Nessie, the first feedback-directed unit test generator that supports nesting of API calls and that tests asynchronous callbacks. Nesting API calls enables a test to use values produced by an API that are available only once a callback has been invoked, and is often necessary to ensure that methods are invoked in a specific order. The core contributions of our approach are a tree-based representation of unit tests with callbacks and a novel algorithm to iteratively generate such tests in a feedback-directed manner. We evaluate our approach on ten popular JavaScript libraries with both asynchronous and synchronous callbacks. The results show that, in a comparison with LambdaTester, a state of the art test generation technique that only considers sequencing of method calls, Nessie finds more behavioral differences and achieves slightly higher coverage. Notably, Nessie needs to generate significantly fewer tests to achieve and exceed the coverage achieved by the state of the art.
Tue 10 MayDisplayed time zone: Eastern Time (US & Canada) change
Wed 11 MayDisplayed time zone: Eastern Time (US & Canada) change
Thu 26 MayDisplayed time zone: Eastern Time (US & Canada) change
11:00 - 12:30 | Papers 15: Software Testing 2Technical Track / SEIP - Software Engineering in Practice at Room 301+302 Chair(s): Rohan Padhye Carnegie Mellon University | ||
11:00 5mTalk | CONFETTI: Amplifying Concolic Guidance for Fuzzers Technical Track James Kukucka George Mason University, Luís Pina University of Illinois at Chicago, Paul Ammann George Mason University, USA, Jonathan Bell Northeastern University Pre-print Media Attached | ||
11:05 5mTalk | Surveying the Developer Experience of Flaky Tests SEIP - Software Engineering in Practice Owain Parry The University of Sheffield, Gregory Kapfhammer Allegheny College, Michael Hilton Carnegie Mellon University, USA, Phil McMinn University of Sheffield Pre-print Media Attached | ||
11:10 5mTalk | Natural Attack for Pre-trained Models of Code Technical Track Zhou Yang Singapore Management University, Jieke Shi Singapore Management University, Junda He Singapore Management University, David Lo Singapore Management University DOI Pre-print Media Attached | ||
11:15 5mTalk | FADATest: Fast and Adaptive Performance Regression Testing of Dynamic Binary Translation Systems Technical Track Jin Wu Harbin Institute of Technology, Jian Dong Harbin Institute Of Technology, Ruili Fang University of Georgia, Wen Zhang University of Georgia, Wenwen Wang University of Georgia, Decheng Zuo Harbin Institute of Technology Pre-print Media Attached | ||
11:20 5mTalk | Repairing Order-Dependent Flaky Tests via Test Generation Technical Track Chengpeng Li University of Texas at Austin, Chenguang Zhu University of Texas at Austin, Wenxi Wang University of Texas at Austin, August Shi University of Texas at Austin Link to publication DOI Media Attached | ||
11:25 5mTalk | BeDivFuzz: Integrating Behavioral Diversity into Generator-based Fuzzing Technical Track Pre-print Media Attached | ||
11:30 5mTalk | Nessie: Automatically Testing JavaScript APIs with Asynchronous Callbacks Technical Track Ellen Arteca Northeastern University, Sebastian Harner University of Stuttgart, Michael Pradel University of Stuttgart, Frank Tip Northeastern University Pre-print Media Attached | ||
11:35 5mTalk | EAGLE: Creating Equivalent Graphs to Test Deep Learning Libraries Technical Track Jiannan Wang Purdue University, Thibaud Lutellier University of Waterloo, Shangshu Qian Purdue University, Hung Viet Pham University of Waterloo, Lin Tan Purdue University Pre-print Media Attached |