Write a Blog >>
Tue 11 Oct 2022 14:40 - 15:00 at Banquet B - Technical Session 7 - Fuzzing II Chair(s): Karine Even-Mendoza

The tremendous advancements in deep learning techniques have empowered question answering(QA) systems with the capability of dealing with various tasks. Many commercial QA systems, such as Siri, Google Home, and Alexa, have been deployed to assist people in different daily activities. However, modern QA systems are often designed to deal with different topics and task formats, which makes both the test collection and labeling tasks difficult and thus threats their quality.

To alleviate this challenge, in this paper, we design and implement a fuzzing framework for QA systems, namely QATest, based on the metamorphic testing theory. It provides the first uniform solution to generate tests with oracle information automatically for various QA systems, such as machine reading comprehension, open-domain QA, and QA on knowledge bases. To further improve testing efficiency and generate more tests detecting erroneous behaviors, we design N-gram coverage criteria and perplexity priority based on the features of the question data to guide the generation process. To evaluate the performance of QATest, we experiment it on four QA systems that are designed for different tasks. The experiment results show that the tests generated by QATest detect hundreds of erroneous behaviors of QA systems efficiently. Also, the results confirm that the testing criteria can improve test diversity and fuzzing efficiency.

Tue 11 Oct

Displayed time zone: Eastern Time (US & Canada) change

14:00 - 15:30
Technical Session 7 - Fuzzing IIResearch Papers at Banquet B
Chair(s): Karine Even-Mendoza Imperial College London
14:00
20m
Research paper
So Many Fuzzers, So Little Time - Experience from Evaluating Fuzzers on the Contiki-NG Network (Hay)StackVirtual
Research Papers
Clement Poncelet Uppsala University, Konstantinos (Kostis) Sagonas Uppsala University and Nat. Tech. Univ. of Athens, Nicolas Tsiftes RISE Research Institutes of Sweden
DOI Pre-print
14:20
20m
Research paper
FuzzerAid: Grouping Fuzzed Crashes Based On Fault Signatures
Research Papers
Ashwin Kallingal Joshy Iowa State University, Wei Le Iowa State University
14:40
20m
Research paper
QATest: A Uniform Fuzzing Framework for Question Answering SystemsVirtualACM SIGSOFT Distinguished Paper Award
Research Papers
Zixi Liu Nanjing University, Yang Feng Nanjing University, Yining Yin Nanjing University, China, Jingyu Sun Nanjing University, Zhenyu Chen Nanjing University, Baowen Xu Nanjing University
15:00
20m
Research paper
Effectively Generating Vulnerable Transaction Sequences in Smart Contracts with Reinforcement Learning-guided FuzzingVirtual
Research Papers
Jianzhong Su Sun Yat-sen University, Hong-Ning Dai Hong Kong Baptist University, Lingjun Zhao Sun Yat-sen University, Zibin Zheng School of Data and Computer Science, Sun Yat-sen University, Xiapu Luo Hong Kong Polytechnic University