TCSE logo 
 Sigsoft logo
Sustainability badge
Wed 30 Apr 2025 16:00 - 16:15 at 214 - AI for Testing and QA 2 Chair(s): Michael Pradel

As software systems become more complex and configurable, more performance problems tend to arise from the configuration designs. This has caused some configuration options to unexpectedly degrade performance which deviates from their original expectations designed by the developers. Such discrepancies, namely configuration performance bugs (CPBugs), are devastating and can be deeply hidden in the source code. Yet, efficiently testing CPBugs is difficult, not only due to the test oracle is hard to set, but also because the configuration measurement is expensive and there are simply too many possible configurations to test. As such, existing testing tools suffer from lengthy runtime or have been ineffective in detecting CPBugs when the budget is limited, compounded by inaccurate test oracle.

In this paper, we seek to achieve significantly faster CPBug testing by neurally prioritizing the testing at both the configuration option and value range levels with automated oracle estimation. Our proposed tool, dubbed NDP, is a general framework that works with different heuristic generators. The idea is to leverage two neural language models: one to estimate the CPBug types that serve as the oracle while, more vitally, the other to infer the probabilities of an option being CPBug-related, based on which the options and the value ranges to be searched can be prioritized. Experiments on several widely-used systems of different versions reveal that NDP can, in general, better predict CPBug type in 87% cases and find more CPBugs with up to 88.88$\times$ testing efficiency speedup over the state-of-the-art tools.

Wed 30 Apr

Displayed time zone: Eastern Time (US & Canada) change

16:00 - 17:30
AI for Testing and QA 2Research Track / SE In Practice (SEIP) at 214
Chair(s): Michael Pradel University of Stuttgart
16:00
15m
Talk
Faster Configuration Performance Bug Testing with Neural Dual-level PrioritizationArtifact-FunctionalArtifact-AvailableArtifact-Reusable
Research Track
Youpeng Ma University of Electronic Science and Technology of China, Tao Chen University of Birmingham, Ke Li University of Exeter
Pre-print
16:15
15m
Talk
Metamorphic-Based Many-Objective Distillation of LLMs for Code-related TasksArtifact-FunctionalArtifact-AvailableArtifact-Reusable
Research Track
Annibale Panichella Delft University of Technology
16:30
15m
Talk
NIODebugger: A Novel Approach to Repair Non-Idempotent-Outcome Tests with LLM-Based Agent
Research Track
Kaiyao Ke University of Illinois at Urbana-Champaign
16:45
15m
Talk
Test Intention Guided LLM-based Unit Test Generation
Research Track
Zifan Nan Huawei, Zhaoqiang Guo Software Engineering Application Technology Lab, Huawei, China, Kui Liu Huawei, Xin Xia Huawei
17:00
15m
Talk
What You See Is What You Get: Attention-based Self-guided Automatic Unit Test Generation
Research Track
Xin Yin Zhejiang University, Chao Ni Zhejiang University, xiaodanxu College of Computer Science and Technology, Zhejiang university, Xiaohu Yang Zhejiang University
Pre-print
17:15
15m
Talk
Improving Code Performance Using LLMs in Zero-Shot: RAPGen
SE In Practice (SEIP)
Spandan Garg Microsoft Corporation, Roshanak Zilouchian Moghaddam Microsoft, Neel Sundaresan Microsoft
:
:
:
: