Diver: Oracle-Guided SMT Solver Testing with Unrestricted Random Mutations
Fri 19 May 2023 14:00 - 14:15 at Meeting Room 105 - Fault injection and mutation Chair(s): Lingxiao Jiang
We present Diver, a novel technique for effectively finding critical bugs in SMT solvers. Ensuring the correctness of SMT solvers is becoming increasingly important as many applications use solvers as a foundational basis. In response, several approaches for testing SMT solvers, which are classified into differential testing and oracle-guided approaches, have been proposed until recently. However, they are still unsatisfactory in that (1) differential testing approaches cannot validate unique yet important features of solvers, and (2) oracle-guided approaches cannot generate diverse tests due to their reliance on limited mutation rules. Diver aims to complement these shortcomings, particularly focusing on finding bugs that are missed by existing approaches. To this end, we present a new testing technique that supports unrestricted random mutations without differential testing. We have used Diver to validate the most recent versions of three popular SMT solvers: CVC5, Z3 and dReal. In total, Diver found 25 confirmed and unique bugs, of which 21 are critical and directly affect the reliability of the solvers. We also empirically prove Diver’s own strength by showing that existing tools are unlikely to find the bugs discovered by Diver.
Wed 17 MayDisplayed time zone: Hobart change
Fri 19 MayDisplayed time zone: Hobart change
13:45 - 15:15 | Fault injection and mutationJournal-First Papers / NIER - New Ideas and Emerging Results / SEIP - Software Engineering in Practice / DEMO - Demonstrations / Technical Track at Meeting Room 105 Chair(s): Lingxiao Jiang Singapore Management University | ||
13:45 15mTalk | Coverage Guided Fault Injection for Cloud Systems Technical Track Yu Gao Institute of Software, Chinese Academy of Sciences, China, Wensheng Dou Institute of Software Chinese Academy of Sciences, Dong Wang Institute of software, Chinese academy of sciences, Wenhan Feng Institute of Software Chinese Academy of Sciences, Jun Wei Institute of Software at Chinese Academy of Sciences; University of Chinese Academy of Sciences; University of Chinese Academy of Sciences Chongqing School, Hua Zhong Institute of Software Chinese Academy of Sciences, Tao Huang Institute of Software Chinese Academy of Sciences Pre-print | ||
14:00 15mTalk | Diver: Oracle-Guided SMT Solver Testing with Unrestricted Random Mutations Technical Track | ||
14:15 15mTalk | Identifying Defect Injection Risks from Analysis and Design Diagrams: An Industrial Case Study at Sony SEIP - Software Engineering in Practice Yoji Imanishi Sony Global Manufacturing&Operations, Kazuhiro Kumon Sony Global Manufacturing&Operations, Shuji Morisaki Nagoya University | ||
14:30 7mTalk | DaMAT: A Data-driven Mutation Analysis Tool DEMO - Demonstrations Enrico Viganò University of Luxembourg, Oscar Cornejo SnT Centre, University of Luxembourg, Fabrizio Pastore University of Luxembourg, Lionel Briand University of Luxembourg; University of Ottawa Pre-print | ||
14:37 7mTalk | Mutation testing in the wild: findings from GitHub Journal-First Papers Ana B. Sánchez University of Seville, Pedro Delgado-Pérez Universidad de Cádiz, Inmaculada Medina-Bulo Universidad de Cádiz, Sergio Segura University of Seville Link to publication DOI | ||
14:45 7mTalk | An Experimental Assessment of Using Theoretical Defect Predictors to Guide Search-Based Software Testing Journal-First Papers Anjana Perera Oracle Labs, Australia, Aldeida Aleti Monash University, Burak Turhan University of Oulu, Marcel Böhme MPI-SP, Germany and Monash University, Australia Link to publication DOI | ||
14:52 7mTalk | Assurance Cases as Data: A Manifesto NIER - New Ideas and Emerging Results Claudio Menghi McMaster University, Canada, Torin Viger , Alessio Di Sandro University of Toronto, Chris Rees Critical Systems Labs, Jeffrey Joyce Critical System Labs Inc., Marsha Chechik University of Toronto | ||
15:00 7mTalk | Predictive Mutation Analysis via Natural Language Channel in Source Code Journal-First Papers Jinhan Kim KAIST, Juyoung Jeon Handong Global University, Shin Hong Handong Global University, Shin Yoo KAIST Link to publication Pre-print |