On-Demand Performance Regression Detection with Test Selection and Amplification
Performance regressions negatively impact the execution time and memory usage of software systems, leading to user dissatisfaction or even failure of the software in the market. Performance testing, such as microbenchmarks, allows developers to detect regressions early and address them before they affect overall system efficiency. In practice, however, existing microbenchmarks might not cover recent code changes or be sufficient to detect performance regressions. To address this problem, we introduce an on-demand performance testing approach that expands existing benchmarks only when code changes occur rather than exhaustively benchmarking the entire system. The approach is integrated into a continuous integration workflow. First, it analyzes code modifications to determine whether existing microbenchmarks already exercise affected methods. When this is not the case, it incrementally augments the performance microbenchmarks by either converting JUnit tests into JMH microbenchmarks or generating new microbenchmarks using large language models. We conduct a preliminary evaluation of our approach on Byte Buddy, a widely used open-source Java library, to assess the effectiveness of the existing and generated microbenchmarks in detecting performance regressions. Our results show that manually written microbenchmarks cover 10 of 26 modified methods but detect only 5 performance regressions. In turn, the augmentation approach detects 17 performance regressions, increasing the detection rate from 19.2% to 65.4% (+46.2%).
Mon 13 AprDisplayed time zone: Brasilia, Distrito Federal, Brazil change
14:00 - 15:30 | |||
14:00 5mTalk | Testing the Untrusted: An Empirical Study of Custom Model Codes in Model-Sharing Platforms Journal Ahead Workshop (JAWs) Mohammed Latif Siddiq University of Notre Dame, Beatrice Casey University of Notre Dame, Joanna C. S. Santos University of Notre Dame | ||
14:05 5mTalk | Towards the Systematic Testing of Regular Expression Engines Journal Ahead Workshop (JAWs) Berk Çakar Purdue University, Dongyoon Lee Stony Brook University, James C. Davis Purdue University Pre-print | ||
14:10 5mTalk | Rethinking Automated Testing for LLM-Based Systems Journal Ahead Workshop (JAWs) Domenico Amalfitano University of Naples Federico II, Marco Autili University of L'Aquila, Italy, Antonia Bertolino Gran Sasso Science Institute, Ricardo Caldas Gran Sasso Science Institute (GSSI), Patrizio Pelliccione Gran Sasso Science Institute, L'Aquila, Italy | ||
14:15 5mTalk | CONGRAD: Controllable Scenario Generation for Autonomous Driving Systems via Gradient-Guided Falsification Journal Ahead Workshop (JAWs) Shuncheng Tang University of Science and Technology of China, Lejin Li Kyushu University, Deyun Lyu National Institute of Informatics, Yinxing Xue Institute of AI for Industries, Chinese Academy of Sciences , Zhenya Zhang Kyushu University, Paolo Arcaini National Institute of Informatics | ||
14:20 5mTalk | Persuasion-guided Automated Testing for Harmful Content Degeneration in Large Language Models Journal Ahead Workshop (JAWs) Francesco Renato Negri Politecnico di Milano, Simone Corbo , Livia Lestingi DEIB, Politecnico di Milano, Matteo Camilli Politecnico di Milano | ||
14:25 5mTalk | On-Demand Performance Regression Detection with Test Selection and Amplification Journal Ahead Workshop (JAWs) Antonio Trovato University of Salerno, Luana Martins University of Salerno, Daniele Fabiano University of Salerno, Daniele Di Pompeo University of L'Aquila, Michele Tucci University of L'Aquila, Dario Di Nucci University of Salerno | ||
14:30 5mTalk | The Intelligent Sidecar: Injecting Semantic Guidance into Smart Contract Fuzzing Journal Ahead Workshop (JAWs) Xin Li The University of Tokyo, Yuheng Huang The University of Tokyo, Andy M. Lee Mamori, Lei Ma The University of Tokyo & University of Alberta | ||
14:35 5mTalk | A Roadmap for Quantum Program Analysis: From Classical to Quantum Challenges Journal Ahead Workshop (JAWs) Shangzhou Xia Kyushu University, Junjie Luo Kyushu University, Qiming Li Kyushu University, Xiaoyu Guo Kyushu University, Minqi Shao Kyushu University, Qichen Wang Kyushu University, Chunkai Chen Kyushu University, Jianjun Zhao Kyushu University | ||
14:40 5mTalk | APIfying COBOL Monoliths: Static Analysis with Pattern-Based Call Simplification Journal Ahead Workshop (JAWs) Vini Kanvar IBM Research, Ashwin Dhinesh Kumar IBM Research, USA, Srikanth G. Tamilselvam IBM India Research Lab, Keerthi Narayan Raghunath IBM, India | ||
14:45 5mTalk | Automated Proof-of-Concept Generation for Software Vulnerabilities: A Survey Journal Ahead Workshop (JAWs) Minglang Li University of Electronic Science and Technology of China, Jiongchi Yu Singapore Management University, Xinfeng Li Nanyang Technological University | ||
14:50 30mPanel | Panel Discussion: Software Testing and Analysis Journal Ahead Workshop (JAWs) | ||
15:20 10mAwards | Selection of award presentations Journal Ahead Workshop (JAWs) | ||