FSE 2025
Mon 23 - Fri 27 June 2025 Trondheim, Norway
Tue 24 Jun 2025 15:20 - 15:30 at Cosmos 3C - Testing 3 Chair(s): Dan Hao

Software in the wild exhibits various dynamic and nondeterministic behaviors, such as runtime invocation from dynamic language features and randomness from probabilistic algorithms. These behaviors complicate software testing because they require meaningful inputs for invocation, introduce additional branching decisions at runtime, and produce divergent outcomes for identical inputs.

To address these challenges, we propose an effective and efficient testing framework that leverages commonality analysis. This paper presents our preliminary research, which leverages input commonalities to explore dynamic behaviors in modern Java software, to demonstrate the utility of our framework. Specifically, we introduce SynToken, a two-fold testing tool that aims to improve (1) the reachability of runtime invocation sites and (2) testing efficiency. First, to address the challenge that existing string mutation operators cannot easily synthesize meaningful inputs for runtime invocation, SynToken constructs a token tree to prioritize tokens that are likely to be used for invocation based on common vocabularies such as prefixes or suffixes. Second, to address the challenge that branching decisions and execution paths are not deterministic, SynToken enhances traditional branch coverage guided fuzzing by monitoring invocation behaviors based on class hierarchies and call graphs.

We preliminarily evaluated SynToken on 71 open-source subjects. SynToken accelerated test generation by up to 10 × with tree-based input prioritization. The string substitution based input mutation enabled SynToken to generate all loadable targets within two minutes for each subject and replicated 280 injected bugs.

In the future, we plan to extend our test generation framework for dynamic behaviors to include nondeterministic behaviors. Nondeterminism often stems from probabilities and randomization, causing programs to branch into multiple outcomes despite identical inputs. We want to systematically explore as many of these outcomes as possible. Leveraging the insight from SynToken that there exist common patterns such as shared prefixes, we plan to define constraints to enable early termination when identical execution trace prefixes are detected, allowing us to improve testing efficiency while covering diverse execution paths.

Tue 24 Jun

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

14:00 - 15:30
14:00
20m
Talk
LlamaRestTest: Effective REST API Testing with Small Language Models
Research Papers
Myeongsoo Kim Georgia Institute of Technology, Saurabh Sinha IBM Research, Alessandro Orso University of Georgia, USA
DOI
14:20
20m
Talk
Testing Updated Apps by Adapting Learned Models
Journal First
Chanh Duc Ngo University of Luxembourg, Fabrizio Pastore University of Luxembourg, Lionel Briand University of Ottawa, Canada; Lero centre, University of Limerick, Ireland
Link to publication
14:40
20m
Talk
Automated Testing of COBOL to Java Transformation
Industry Papers
Sandeep Hans IBM India Research Lab, Atul Kumar IBM Research India, Toshiaki Yasue IBM Research - Tokyo, Kohichi Ono IBM Research - Tokyo, Saravanan Krishnan IBM India Research Lab, Devika Sondhi IBM Research, Fumiko Satoh IBM Research - Tokyo, Gerald Mitchell IBM Software, Sachin Kumar IBM Software, Diptikalyan Saha IBM Research India
15:00
20m
Talk
TerzoN: Human-in-the-Loop Software Testing with a Composite Oracle
Research Papers
Matthew C. Davis Carnegie Mellon University, Amy Wei University of Michigan, Brad A. Myers Carnegie Mellon University, Joshua Sunshine Carnegie Mellon University
Link to publication DOI
15:20
10m
Talk
Efficient Test Generation for Dynamic Behaviors Leveraging Token-Level Input Commonalities
Ideas, Visions and Reflections
Yuxin Qiu University of California at Riverside, Qian Zhang University of California at Riverside

Information for Participants
Tue 24 Jun 2025 14:00 - 15:30 at Cosmos 3C - Testing 3 Chair(s): Dan Hao
Info for room Cosmos 3C:

Cosmos 3C is the third room in the Cosmos 3 wing.

When facing the main Cosmos Hall, access to the Cosmos 3 wing is on the left, close to the stairs. The area is accessed through a large door with the number “3”, which will stay open during the event.