ASE 2024
Sun 27 October - Fri 1 November 2024 Sacramento, California, United States
Tue 29 Oct 2024 15:30 - 15:45 at Carr - Performance and load

Performance testing aims at uncovering efficiency issues of software systems. In order to be both effective and practical, the design of a performance test must achieve a reasonable trade-off between result quality and testing time. This becomes particularly challenging in Java context, where the software undergoes a warm-up phase of execution, due to just-in-time compilation. During this phase, performance measurements are subject to severe fluctuations, which may adversely affect quality of performance test results. Both practitioners and researchers have proposed approaches to mitigate this issue. Practitioners typically rely on a fixed number of iterated executions that are used to warm-up the software before starting to collect performance measurements (state-of-practice). Researchers have developed techniques that can dynamically stop warm-up iterations at runtime (state-of-the-art). However, these approaches often provide suboptimal estimates of the warm-up phase, resulting in either insufficient or excessive warm-up iterations, which may degrade result quality or increase testing time. There is still a lack of consensus on how to properly address this problem. Here, we propose and study an AI-based framework to dynamically halt warm-up iterations at runtime. Specifically, our framework leverages recent advances in AI for Time Series Classification (TSC) to predict the end of the warm-up phase during test execution. We conduct experiments by training three different TSC models on half a million of measurement segments obtained from JMH microbenchmark executions. We find that our framework significantly improves the accuracy of the warm-up estimates provided by state-of-practice and state-of-the-art methods. This higher estimation accuracy results in a net improvement in either result quality or testing time for up to +35.3% of the microbenchmarks. Our study highlights that integrating AI to dynamically estimate the end of the warm-up phase can enhance the cost-effectiveness of Java performance testing.

Tue 29 Oct

Displayed time zone: Pacific Time (US & Canada) change

15:30 - 17:00
15:30
15m
Talk
AI-driven Java Performance Testing: Balancing Result Quality with Testing Time
Research Papers
Luca Traini University of L'Aquila, Federico Di Menna University of L'Aquila, Vittorio Cortellessa University of L'Aquila
DOI Pre-print
15:45
15m
Talk
MLOLET - Machine Learning Optimized Load and Endurance Testing: An industrial experience report
Industry Showcase
Arthur Vitui Concordia University, Tse-Hsun (Peter) Chen Concordia University
16:00
15m
Talk
Dynamic Scoring Code Token Tree: A Novel Decoding Strategy for Generating High-Performance Code
Research Papers
Muzi Qu University of Chinese Academy of Sciences, Jie Liu Institute of Software, Chinese Academy of Sciences, Liangyi Kang Institute of Software, Chinese Academy of Sciences, Shuai Wang Institute of Software, Chinese Academy of Sciences, Dan Ye Institute of Software, Chinese Academy of Sciences, Tao Huang Institute of Software at Chinese Academy of Sciences
16:15
10m
Talk
BenchCloud: A Platform for Scalable Performance Benchmarking
Tool Demonstrations
Dirk Beyer LMU Munich, Po-Chun Chien LMU Munich, Marek Jankola LMU Munich
DOI Pre-print Media Attached
16:25
10m
Talk
A Formal Treatment of Performance BugsRecorded Talk
NIER Track
Omar I. Al Bataineh Gran Sasso Science Institute (GSSI)