Finding and Understanding Incompleteness Bugs in SMT SolversVirtual
We propose Janus, an approach for finding incompleteness bugs in SMT solvers. The key insight is to mutate SMT formulas with local weakening and strengthening rules that preserve the satisfiability of the seed formula. The generated mutants are used to test SMT solvers for incompleteness bugs, i.e., inputs on which the SMT solver unexpectedly returns unknown. We realized Janus on top of the SMT solver fuzzing framework YinYang. From June to August 2021, we stress-tested the two state-of-the-art SMT solvers Z3 and CVC5 with Janus and totally reported 32 incompleteness bugs. Out of these, 24 have been confirmed as unique bugs and 8 are already fixed by the developers. Our diverse bug findings uncovered functional, regression, and performance bugs—several surprising enough to trigger discussions among the developers sharing their in-depth analysis.
Thu 13 OctDisplayed time zone: Eastern Time (US & Canada) change
16:00 - 18:00 | Technical Session 32 - Formal Methods and Models IITool Demonstrations / Journal-first Papers / Research Papers at Banquet B Chair(s): Khouloud Gaaloul University of Michigan - Dearborn | ||
16:00 10mDemonstration | CBMC-SSM: Bounded Model Checking of C Programs with Symbolic Shadow Memory Tool Demonstrations Bernd Fischer Stellenbosch University, South Africa, Salvatore La Torre Università degli Studi di Salerno, Gennaro Parlato University of Molise, Peter Schrammel University of Sussex and Diffblue Ltd | ||
16:10 20mResearch paper | Tseitin or not Tseitin? The Impact of CNF Transformations on Feature-Model Analyses Research Papers Elias Kuiter Otto-von-Guericke-University Magdeburg, Sebastian Krieter University of Ulm, Chico Sundermann University of Ulm, Thomas Thüm University of Ulm, Gunter Saake University of Magdeburg, Germany | ||
16:30 20mPaper | A three-valued model abstraction framework for PCTL* stochastic model checkingVirtual Journal-first Papers Yang Liu Shanghai Maritime University/National University of Singapore, Yan Ma Nanjing University of Finance and Economics / National University of Singapore, Yongsheng Yang Shanghai Maritime University File Attached | ||
16:50 20mResearch paper | Finding and Understanding Incompleteness Bugs in SMT SolversVirtual Research Papers | ||
17:10 20mResearch paper | Checking LTL Satisfiability via End-to-end LearningVirtual Research Papers Weilin Luo School of Computer Science and Engineering, Sun Yat-sen University, Hai Wan School of Data and Computer Science, Sun Yat-sen University, Delong Zhang SUN YAT-SEN UNIVERSITY, Jianfeng Du Guangdong University of Foreign Studies, Hengdi Su SUN YAT-SEN UNIVERSITY | ||
17:30 20mResearch paper | QVIP: An ILP-based Formal Verification Approach for Quantized Neural NetworksVirtual Research Papers Yedi Zhang ShanghaiTech University, Zhe Zhao ShanghaiTech University, Guangke Chen ShanghaiTech University, Fu Song ShanghaiTech University, Min Zhang East China Normal University, Taolue Chen Birkbeck University of London, Jun Sun Singapore Management University | ||