Validating SMT Solvers via Skeleton Enumeration Empowered by Historical Bug-Triggering Inputs
SMT solvers check the satisfiability of logic formulas via first-order theories, which have been utilized in a rich number of critical applications, such as software verification, test case generation, and program synthesis. Bugs hidden in SMT solvers would severely mislead those applications and further cause severe consequences. Therefore, ensuring the reliability and robustness of SMT solvers is of critical importance. Although many approaches have been proposed to test SMT solvers, it is still a challenge to discover bugs effectively. To tackle such a challenge, we conduct an empirical study on the historical bug-triggering formulas in SMT solvers’ bug tracking systems. We observe that the historical bug-triggering formulas contain valuable skeletons (i.e., core structures of formulas) as well as associated atomic formulas which can cast significant impacts on formulas’ ability in triggering bugs. Therefore, we propose a novel approach that utilizes the skeletons extracted from the historical bug-triggering formulas and enumerates atomic formulas under the guidance of association rules derived from historical formulas. In this study, we realized our approach as a practical fuzzing tool HistFuzz and conducted extensive testing on the state-of-the-art SMT solvers Z3 and CVC5. To date, HistFuzz has found 111 confirmed new bugs for Z3 and CVC5, of which 108 have been fixed by the developers, which significantly improved the reliability and correctness of the state-of-the-art SMT solvers. In addition, our experiments also demonstrate that HistFuzz has an excellent performance when compared against the other SMT solver fuzzing tools in terms of achieved code coverage and effectiveness.
Wed 17 MayDisplayed time zone: Hobart change
11:00 - 12:30 | Fuzzing: applicationsTechnical Track / DEMO - Demonstrations at Meeting Room 101 Chair(s): Corina S. Păsăreanu Carnegie Mellon University | ||
11:00 15mTalk | Detecting JVM JIT Compiler Bugs via Exploring Two-Dimensional Input Spaces Technical Track Haoxiang Jia Huazhong University of Science and Technology, Ming Wen Huazhong University of Science and Technology, Zifan Xie Huazhong University of Science and Technology, Xiaochen Guo Huazhong University of Science and Technology, Rongxin Wu Xiamen University, Maolin Sun Huazhong University of Science and Technology, Kang Chen Huazhong University of Science and Technology, Hai Jin Huazhong University of Science and Technology Pre-print | ||
11:15 15mTalk | JITfuzz: Coverage-guided Fuzzing for JVM Just-in-Time Compilers Technical Track Mingyuan Wu Southern University of Science and Technology, Minghai Lu Southern University of Science and Technology, Heming Cui University of Hong Kong, Junjie Chen Tianjin University, Yuqun Zhang Southern University of Science and Technology, Lingming Zhang University of Illinois at Urbana-Champaign | ||
11:30 15mTalk | Validating SMT Solvers via Skeleton Enumeration Empowered by Historical Bug-Triggering Inputs Technical Track Maolin Sun Huazhong University of Science and Technology, Yibiao Yang Nanjing University, Ming Wen Huazhong University of Science and Technology, Yongcong Wang Huazhong University of Science and Technology, Yuming Zhou Nanjing University, Hai Jin Huazhong University of Science and Technology Pre-print | ||
11:45 15mTalk | Regression Fuzzing for Deep Learning Systems Technical Track Hanmo You College of Intelligence and Computing, Tianjin University, Zan Wang Tianjin University, China, Junjie Chen Tianjin University, Shuang Liu Tianjin University, Shuochuan Li College of Intelligence and Computing, Tianjin University | ||
12:00 15mTalk | Operand-Variation-Oriented Differential Analysis for Fuzzing Binding Calls in PDF Readers Technical Track Suyue Guo Renmin University of China, Xinyu Wan Renmin University of China, Wei You Renmin University of China, Bin Liang Renmin University of China, China, Wenchang Shi Renmin University of China, China, Yiwei Zhang Renmin University of China, Jianjun Huang Renmin University of China, China, Jian Zhang State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences, Beijing, China Pre-print | ||
12:15 7mTalk | JAttack: Java JIT Testing using Template Programs DEMO - Demonstrations Zhiqiang Zang University of Texas at Austin, Fu-Yao Yu The University of Texas at Austin, Nathan Wiatrek The University of Texas at Austin, Milos Gligoric University of Texas at Austin, August Shi University of Texas at Austin Pre-print | ||