Write a Blog >>
ICSE 2023
Sun 14 - Sat 20 May 2023 Melbourne, Australia

To ensure compilers’ quality, compiler testing has received more and more attention, and test-program generation is the core task. In recent years, some approaches have been proposed to explore test configurations for generating more effective test programs, but they either are restricted by historical bugs or suffer from the cost-effectiveness issue. Here, we propose a novel test-program generation approach (called MCS) to further improving the performance of compiler testing. MCS conducts memoized search via multi-agent reinforcement learning (RL) for guiding the construction of effective test configurations based on the memoization for the explored test configurations during the on-the-fly compiler-testing process. During the process, the elaborate coordination among configuration options can be also well learned by multi-agent RL, which is required for generating bug-triggering test programs. Specifically, MCS considers the diversity among test configurations to efficiently explore the input space and the testing results under each explored configuration to learn which portions of space are more bug-triggering. Our extensive experiments on GCC and LLVM demonstrate the performance of MCS, significantly outperforming the state-of-the-art test-program generation approaches in bug detection. Also, MCS detects 16 new bugs on the latest trunk revisions of GCC and LLVM, and all of them have been confirmed or fixed by developers. MCS has been deployed by a global IT company for testing their in-house compiler, and detects 10 new bugs (covering all the 5 bugs detected by the compared approaches), all of which have been confirmed.

Fri 19 May

Displayed time zone: Hobart change

11:00 - 12:30
Testing of database and low-level softwareTechnical Track / SEIP - Software Engineering in Practice / DEMO - Demonstrations / Journal-First Papers at Meeting Room 109
Chair(s): Michael Pradel University of Stuttgart
11:00
15m
Talk
Compiler Test-Program Generation via Memoized Configuration Search
Technical Track
Junjie Chen Tianjin University, Chenyao Suo College of Intelligence and Computing, Tianjin University, Jiajun Jiang Tianjin University, Peiqi Chen College of Intelligence and Computing, Tianjin University, Xingjian Li College of Intelligence and Computing, Tianjin University
11:15
15m
Talk
Generating Test Databases for Database-Backed Applications
Technical Track
Cong Yan Microsoft Research, Suman Nath Microsoft Research, Shan Lu University of Chicago
11:30
15m
Talk
Testing Database Engines via Query Plan GuidanceDistinguished Paper Award
Technical Track
Jinsheng Ba National University of Singapore, Manuel Rigger National University of Singapore
Pre-print
11:45
15m
Talk
Testing Database Systems via Differential Query Execution
Technical Track
Jiansen Song , Wensheng Dou Institute of Software Chinese Academy of Sciences, Ziyu Cui Institute of Software Chinese Academy of Sciences, Qianwang Dai Institute of Software Chinese Academy of Sciences, Wei Wang , Jun Wei Institute of Software at Chinese Academy of Sciences; University of Chinese Academy of Sciences; University of Chinese Academy of Sciences Chongqing School, Hua Zhong Institute of Software Chinese Academy of Sciences, Tao Huang Institute of Software Chinese Academy of Sciences
Pre-print
12:00
15m
Talk
Achieving Last-Mile Functional Coverage in Testing Chip Design Software Implementations
SEIP - Software Engineering in Practice
Ming Yan College of Intelligence and Computing, Tianjin University, Junjie Chen Tianjin University, Hangyu Mao , Jiajun Jiang Tianjin University, Jianye Hao Tianjin University, Xingjian Li College of Intelligence and Computing, Tianjin University, Zhao Tian Tianjin University, Zhichao Chen , Dong Li , Dong Li , Yanwei Guo , Wulong Liu , Bin Wang , Yuefeng Sun , Yongshun Cui
12:15
7m
Talk
Randomized Differential Testing of RDF Stores
DEMO - Demonstrations
Rui Yang Institute of Software, Chinese Academy of Sciences, Yingying Zheng Institute of Software Chinese Academy of Sciences, Lei Tang Institute of Software Chinese Academy of Sciences, Wensheng Dou Institute of Software Chinese Academy of Sciences, Wei Wang , Jun Wei Institute of Software at Chinese Academy of Sciences; University of Chinese Academy of Sciences; University of Chinese Academy of Sciences Chongqing School
Pre-print
12:22
7m
Talk
Boosting Compiler Testing via Compiler Optimization Exploration
Journal-First Papers
Junjie Chen Tianjin University, Chenyao Suo College of Intelligence and Computing, Tianjin University