Enriching Compiler Testing with Real Program from Bug ReportVirtual
The source code must be compiled into machine code, so that machines can understand the intention of software. Compilers are important, because a bug in compilers can hinder or even crash the compilation. As there are often more than one compiler for each programming language, differential testing has been widely used to detect bugs in compilers. Its basic idea is to compile test programs with different compilers, and compare their compilation results to detect bugs. For this research line, test programs are critical, and researchers have proposed various approaches to generate test programs. The state-of-the-art approaches can be roughly divided into random-based and mutation-based approaches: random-based approaches generate random programs and mutation-based approaches mutate programs to generate more test programs. Both lines of approaches mainly generate random code, but it is more beneficial to use real programs, since it is easier to learn the impacts of compiler bugs and it becomes reasonable to use both valid and invalid code. However, most real programs from code repositories are ineffective to trigger compiler bugs, partially because they are compiled before they are submitted. In this experience paper, we apply two techniques such as differential testing and code snippet extraction to the specific research domain of compiler testing. Based on our observations on the practice of testing compilers, we identify bug reports of compilers as a new source for compiler testing. To illustrate the benefits of the new source, we implement a tool, called LeRe, that extracts test programs from bug reports and uses differential testing to detect compiler bugs with extracted programs. After we enriched the test programs, we have found 156 unique bugs in the latest versions of gcc and clang. Among them, 103 bugs are confirmed as valid, and 9 bugs are already fixed. Our found bugs contain 59 accept-invalid bugs and 33 reject-valid bugs. In these bugs, compilers wrongly accept invalid programs or reject valid programs. The new source enables us detecting accept-invalid and reject-valid bugs that were usually missed by the prior approaches. The prior approaches seldom report the two types of bugs. Besides our found bugs, we also present our analysis on our invalid bug reports. The results are useful for programmers, when they are switching from one compiler to another, and can provide insights, when researchers apply differential testing to detect bugs in more types of software
I received my Ph.D degree from Peking University in 2009. My Ph.D dissertation was nominated for the distinguished Ph.D dissertation award of China Computer Federation. After graduation, I joined Institute of Software, Chinese Academy of Sciences as an assistant professor, and was promoted as an associated professor in 2011. From 2012 to 2014, I was a visiting scholar with University of California, Davis. In 2014, I joined Shanghai Jiao Tong University. I am a recipient of ACM SIGSOFT Distinguished Paper Award, the best paper award of ASE, and the best paper award of APSEC.
Wed 12 OctDisplayed time zone: Eastern Time (US & Canada) change
13:30 - 15:30 | Technical Session 15 - Compilers and LanguagesJournal-first Papers / Research Papers / Industry Showcase at Banquet B Chair(s): Lingming Zhang University of Illinois at Urbana-Champaign | ||
13:30 20mResearch paper | Cornucopia: A Framework for Feedback Guided Generation of Binaries Research Papers Vidush Singhal Purdue University, Akul Abhilash Pillai Purdue University, Charitha Saumya Purdue University, Milind Kulkarni Purdue University, Aravind Machiry Purdue University | ||
13:50 20mPaper | CSMITHEDGE: More Effective Compiler Testing by Handling Undefined Behaviour Less Conservatively Journal-first Papers Karine Even-Mendoza Imperial College London, Cristian Cadar Imperial College London, UK, Alastair F. Donaldson Imperial College London | ||
14:10 20mResearch paper | Compiler Testing using Template Java ProgramsACM SIGSOFT Distinguished Paper Award Research Papers Zhiqiang Zang University of Texas at Austin, Nathan Wiatrek The University of Texas at Austin, Milos Gligoric University of Texas at Austin, August Shi University of Texas at Austin DOI Pre-print | ||
14:30 20mIndustry talk | Towards Understanding the Performance of Rust Industry Showcase Yuchen Zhang Stevens Institute of Technology, Yunhang Zhang The University of Utah, Georgios Portokalidis Stevens Institute of Technology, Jun Xu The University of Utah | ||
14:50 20mResearch paper | TransRepair: Context-aware Program Repair for Compilation ErrorsVirtual Research Papers Xueyang Li SKLOIS, Institute of Information Engineering, Chinese Academy of Sciences, China, Shangqing Liu Nanyang Technological University, Ruitao Feng Nanyang Technological University, Guozhu Meng Institute of Information Engineering, Chinese Academy of Sciences, Xiaofei Xie Singapore Management University, Singapore, Kai Chen SKLOIS, Institute of Information Engineering, Chinese Academy of Sciences, China, Yang Liu Nanyang Technological University | ||
15:10 20mResearch paper | Enriching Compiler Testing with Real Program from Bug ReportVirtual Research Papers Hao Zhong Shanghai Jiao Tong University |