OptionFuzz: Fuzzing SMT Solvers with Optimized Option Exploration via Large Language Models
This program is tentative and subject to change.
—Satisfiability Modulo Theory (SMT) solvers play a crucial role in various domains and applications. Therefore, ensuring their correctness and robustness becomes increasingly vital. Fuzzing is an efficient and effective method for validating the quality of SMT solvers, utilizing inputs that consist of solving formulas and configuration options. However, existing fuzzing methods focus solely on generating formulas or simply combining options and formulas, neglecting the complex interactions between options. Yet, randomly combining multiple options can lead to a combinatorial explosion and result in numerous invalid inputs. To overcome these limitations, we propose OptionFuzz, a fuzzer that optimizes option exploration by identifying relationships between solver options, reducing invalid inputs and mitigating combinatorial explosion. OptionFuzz identifies option relationships using large language models (LLMs), which analyze official documentation of options. These identified relationships are transformed to a relation graph, enabling efficient traversal to derive related option combinations and generate high-quality fuzz inputs. To evaluate OptionFuzz’s effectiveness, we conduct comprehensive evaluations on two state-of-the-art SMT solvers, Z3 and CVC5. OptionFuzz demonstrates its effectiveness by accurately extracting option relationships with an accuracy of 95.23% and a recall rate of 90.10%. Leveraging these relationships, OptionFuzz reduces the number of options combinations to be tested by 70.11%. Notably, OptionFuzz has detected 34 unique bugs, 20 of which have been fixed by developers, and 5 have been assigned CVE IDs due to their severity.
This program is tentative and subject to change.
Thu 11 SepDisplayed time zone: Auckland, Wellington change
10:30 - 12:00 | Session 7 - Testing 2Registered Reports / Research Papers Track / Journal First Track / Tool Demonstration Track / Industry Track / NIER Track at Case Room 3 260-055 Chair(s): Jiajun Jiang Tianjin University | ||
10:30 15m | OptionFuzz: Fuzzing SMT Solvers with Optimized Option Exploration via Large Language Models Research Papers Track Yuhao Peng (Institute of Software, Chinese Academy of Sciences; University of Chinese Academy of Sciences), Jingzheng Wu Institute of Software, The Chinese Academy of Sciences, Xiang Ling Institute of Software, Chinese Academy of Sciences, Zhiyuan Li , Tianyue Luo (Institute of Software Chinese Academy of Sciences), Yanjun Wu Institute of Software, Chinese Academy of Sciences | ||
10:45 15m | Nüwa: Enhancing MLIR Fuzzing with LLM-Driven Generation and Adaptive Mutation Research Papers Track Bocan Cao Northwest University, Weiyuan Tong Northwest University, Zhanyong Tang Northwest University, Zixu Wang Northwest University, Hao Huang Northwest University, Yuheng Yan Northwest University | ||
11:00 10m | MediumDarwin: LittleDarwin Grows with Performance and Research-oriented Extensions Tool Demonstration Track Sajjad Hesamipour Khelejan School of Computer Science and Statistics, Trinity College Dublin & Research Ireland Lero, Thomas Laurent School of Computer Science and Statistics, Trinity College Dublin & Research Ireland Lero, Anthony Ventresque School of Computer Science and Statistics, Trinity College Dublin & Research Ireland Lero | ||
11:10 10m | Rethinking Cognitive Complexity for Unit Tests: Toward a Readability-Aware Metric Grounded in Developer Perception NIER Track Wendkuuni Arzouma Marc Christian OUEDRAOGO University of Luxembourg, Yinghua Li University of Luxembourg, Xueqi Dang University of Luxembourg, SnT, Xin Zhou Singapore Management University, Singapore, Anil Koyuncu Bilkent University, Jacques Klein University of Luxembourg, David Lo Singapore Management University, Tegawendé F. Bissyandé University of Luxembourg | ||
11:20 15m | Targeted Test Selection Approach in Continuous Integration Industry Track Pavel Plyusnin T-Technologies, Aleksey Antonov T-Technologies, Vasilii Ermakov T-Technologies, Aleksandr Khaybriev T-Technologies, Margarita Kikot T-Technologies, Nikolay Bushkov T-Technologies, Stanislav Moiseev T-Technologies | ||
11:35 15m | An Empirical Investigation into the Capabilities of Anomaly Detection Approaches for Test Smell Detection Journal First Track Valeria Pontillo Gran Sasso Science Institute, Luana Martins University of Salerno, Ivan Machado Federal University of Bahia - UFBA, Fabio Palomba University of Salerno, Filomena Ferrucci Università di Salerno DOI Pre-print | ||
11:50 10mResearch paper | Assessing Reliability of Statistical Maximum Coverage Estimators in Fuzzing Registered Reports Danushka Liyanage University of Sydney, Australia, Nelum Attanayake University of Sydney, Australia, Zijian Luo University of Sydney, Australia, Rahul Gopinath University of Sydney DOI Pre-print |