AI-driven Java Performance Testing: Balancing Result Quality with Testing Time
Performance testing aims at uncovering efficiency issues of software systems. In order to be both effective and practical, the design of a performance test must achieve a reasonable trade-off between result quality and testing time. This becomes particularly challenging in Java context, where the software undergoes a warm-up phase of execution, due to just-in-time compilation. During this phase, performance measurements are subject to severe fluctuations, which may adversely affect quality of performance test results. Both practitioners and researchers have proposed approaches to mitigate this issue. Practitioners typically rely on a fixed number of iterated executions that are used to warm-up the software before starting to collect performance measurements (state-of-practice). Researchers have developed techniques that can dynamically stop warm-up iterations at runtime (state-of-the-art). However, these approaches often provide suboptimal estimates of the warm-up phase, resulting in either insufficient or excessive warm-up iterations, which may degrade result quality or increase testing time. There is still a lack of consensus on how to properly address this problem. Here, we propose and study an AI-based framework to dynamically halt warm-up iterations at runtime. Specifically, our framework leverages recent advances in AI for Time Series Classification (TSC) to predict the end of the warm-up phase during test execution. We conduct experiments by training three different TSC models on half a million of measurement segments obtained from JMH microbenchmark executions. We find that our framework significantly improves the accuracy of the warm-up estimates provided by state-of-practice and state-of-the-art methods. This higher estimation accuracy results in a net improvement in either result quality or testing time for up to +35.3% of the microbenchmarks. Our study highlights that integrating AI to dynamically estimate the end of the warm-up phase can enhance the cost-effectiveness of Java performance testing.
Tue 29 OctDisplayed time zone: Pacific Time (US & Canada) change
15:30 - 17:00 | |||
15:30 15mTalk | AI-driven Java Performance Testing: Balancing Result Quality with Testing Time Research Papers Luca Traini University of L'Aquila, Federico Di Menna University of L'Aquila, Vittorio Cortellessa University of L'Aquila DOI Pre-print | ||
15:45 15mTalk | MLOLET - Machine Learning Optimized Load and Endurance Testing: An industrial experience report Industry Showcase | ||
16:00 15mTalk | Dynamic Scoring Code Token Tree: A Novel Decoding Strategy for Generating High-Performance Code Research Papers Muzi Qu University of Chinese Academy of Sciences, Jie Liu Institute of Software, Chinese Academy of Sciences, Liangyi Kang Institute of Software, Chinese Academy of Sciences, Shuai Wang Institute of Software, Chinese Academy of Sciences, Dan Ye Institute of Software, Chinese Academy of Sciences, Tao Huang Institute of Software at Chinese Academy of Sciences | ||
16:15 10mTalk | BenchCloud: A Platform for Scalable Performance Benchmarking Tool Demonstrations DOI Pre-print Media Attached | ||
16:25 10mTalk | A Formal Treatment of Performance BugsRecorded Talk NIER Track Omar I. Al Bataineh Gran Sasso Science Institute (GSSI) |