Thanos: DBMS Bug Detection via Storage Engine Rotation Based Differential Testing
Award Winner
Differential testing is a prevalent strategy for establishing test oracles in automated DBMS testing. However, meticulously selecting equivalent DBMSs with diverse implementations and compatible input syntax requires huge manual efforts. In this paper, we propose Thanos, a framework that finds DBMS bugs via storage engine rotation based differential testing. Our key insight is that a DBMS with different storage engines must provide consistent basic storage functionalities. Therefore, it’s feasible to construct equivalent DBMSs based on storage engine rotation, ensuring that the same SQL test cases to these equivalent DBMSs yield consistent results. The framework involves four main steps: 1) select the appropriate storage engines; 2) extract equivalence information among the selected storage engines; 3) synthesize feature-orient test cases that ensure the DBMS equivalence; and 4) send test cases to the DBMSs with selected storage engines and compare the results. We evaluate Thanos on three widely used and extensively tested DBMSs, namely MySQL, MariaDB, and Percona against state-of-the-art fuzzers SQLancer, SQLsmith, and Squirrel. Thanos outperforms them on branch coverage by 24%–116%, and also finds many bugs missed by other fuzzers. More importantly, the vendors have confirmed 32 previously unknown bugs found by Thanos, with 29 verified as Critical.
Wed 30 AprDisplayed time zone: Eastern Time (US & Canada) change
16:00 - 17:30 | Databases and BusinessResearch Track / SE In Practice (SEIP) / Demonstrations / Journal-first Papers at 104 Chair(s): Lu Xiao Stevens Institute of Technology | ||
16:00 15mTalk | Optimization of Automated and Manual Software Tests in Industrial Practice: A Survey and Historical Analysis Journal-first Papers Roman Haas Saarland University; CQSE, Raphael Nömmer Saarbr�cken Graduate School of Computer Science, CQSE, Elmar Juergens CQSE GmbH, Sven Apel Saarland University Link to publication Pre-print | ||
16:15 15mTalk | A-COBREX : A Tool for Identifying Business Rules in COBOL Programs Demonstrations Samveg Shah Indian Institute of Technology, Tirupati, Shivali Agarwal IBM, Saravanan Krishnan IBM India Research Lab, Vini Kanvar IBM Research, Sridhar Chimalakonda Indian Institute of Technology Tirupati | ||
16:30 15mTalk | Thanos: DBMS Bug Detection via Storage Engine Rotation Based Differential TestingAward Winner Research Track Ying Fu National University of Defense Technology, Zhiyong Wu Tsinghua University, China, Yuanliang Zhang National University of Defense Technology, Jie Liang , Jingzhou Fu School of Software, Tsinghua University, Yu Jiang Tsinghua University, Shanshan Li National University of Defense Technology, Liao Xiangke National University of Defense Technology | ||
16:45 15mTalk | Coni: Detecting Database Connector Bugs via State-Aware Test Case Generation Research Track Wenqian Deng Tsinghua University, Zhiyong Wu Tsinghua University, China, Jie Liang , Jingzhou Fu School of Software, Tsinghua University, Mingzhe Wang Tsinghua University, Yu Jiang Tsinghua University | ||
17:00 15mTalk | Puppy: Finding Performance Degradation Bugs in DBMSs via Limited-Optimization Plan Construction Research Track Zhiyong Wu Tsinghua University, China, Jie Liang , Jingzhou Fu School of Software, Tsinghua University, Mingzhe Wang Tsinghua University, Yu Jiang Tsinghua University | ||
17:15 15mTalk | Safe Validation of Pricing Agreements SE In Practice (SEIP) John C. Kolesar Yale University, Tancrède Lepoint Amazon, Martin Schäf Amazon Web Services, Willem Visser Amazon Web Services |