Adaptive Random Testing (ART) has faced criticism, particularly for its computational inefficiency, as highlighted by Arcuri and Briand. Their analysis clarified how ART requires a quadratic number of distance computations as the number of test executions increases, which limits its scalability in scenarios requiring extensive testing to uncover faults. Simulation results support this, showing that the computational overhead of these distance calculations often outweighs ART’s benefits. While various ART variants have attempted to reduce these costs, they frequently do so at the expense of fault detection, lack complexity guarantees, or are restricted to specific input types, such as numerical or discrete data.
In this paper, we introduce a novel framework for adaptive random testing that replaces pairwise distance computations with a compact aggregation of past executions, such as counting the Qgrams observed in previous runs. Test case selection then leverages this aggregated data to measure diversity (e.g., entropy of Qgrams), allowing us to reduce the computational complexity from quadratic to linear.
Experiments with a benchmark of six web applications, show that ART with Qgrams covers, on average, 4× more unique targets than random testing, and 3.5× more than ART using traditional distance-based methods.
Wed 25 JunDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
14:00 - 15:20 | Testing 4Industry Papers / Research Papers / Demonstrations at Cosmos 3D Chair(s): Antonio Mastropaolo William and Mary, USA | ||
14:00 20mTalk | Detecting and Reducing the Factual Hallucinations of Large Language Models with Metamorphic Testing Research Papers Weibin Wu Sun Yat-sen University, Yuhang Cao Sun Yat-sen University, Ning Yi Sun Yat-sen University, Rongyi Ou Sun Yat-sen University, Zibin Zheng Sun Yat-sen University DOI | ||
14:20 10mTalk | A Tool for Generating Exceptional Behavior Tests With Large Language Models Demonstrations Linghan Zhong University of Texas Austin, Samuel Yuan The University of Texas at Austin, Jiyang Zhang University of Texas at Austin, Yu Liu Meta, Pengyu Nie University of Waterloo, Junyi Jessy Li University of Texas at Austin, USA, Milos Gligoric The University of Texas at Austin | ||
14:30 20mTalk | Using Large Language Models to Support the Workflow of Differential Testing Industry Papers Arun Krishna Vajjala George Mason University, Ajay Krishna Vajjala George Mason University, Carmen Badea Microsoft Research, Christian Bird Microsoft Research, Jade D'Souza Microsoft, Robert DeLine Microsoft Research, Mikhail Demyanyuk Microsoft, Jason Entenmann Microsoft Research, Nicole Forsgren Microsoft Research, Aliaksandr Hramadski Microsoft, Haris Mohammad Microsoft, Sandeepan Sanyal Microsoft, Oleg Surmachev Microsoft, Thomas Zimmermann University of California, Irvine | ||
14:50 20mTalk | Adaptive Random Testing with Qgrams: the Illusion Comes True Research Papers Matteo Biagiola Università della Svizzera italiana, Robert Feldt Chalmers | University of Gothenburg, Paolo Tonella USI Lugano DOI Pre-print | ||
15:10 10mTalk | Dynamic Application Security Testing for Kubernetes Deployment: An Experience Report from Industry Industry Papers Shazibul Islam Shamim Kennesaw State University, Hanyang Hu Company A, Akond Rahman Auburn University Pre-print | ||
Cosmos 3D is the fourth room in the Cosmos 3 wing.
When facing the main Cosmos Hall, access to the Cosmos 3 wing is on the left, close to the stairs. The area is accessed through a large door with the number “3”, which will stay open during the event.