Programs expecting structured inputs often consist of both a syntactic analysis stage, which parses raw input, and a semantic analysis stage, which conducts checks on the parsed input and executes the core logic of the program. Generator-based testing tools in the lineage of QuickCheck are a promising way to generate random syntactically valid test inputs for these programs. But often, to effectively explore the semantic analysis stage, these tools require tedious manual tuning of the generators. We present Zest, a technique which automatically guides QuickCheck-like random input generators to better explore the semantic analysis stage of test programs. Zest converts random input generators into deterministic parametric input generators. We present the key insight that mutations in the untyped parameter domain map to structural mutations in the input domain. Zest leverages this insight to perform generator-based mutational fuzzing, directed by program feedback in the form of code coverage and input validity. We implement Zest in Java and evaluate it against AFL and QuickCheck on five real-world benchmarks: Maven, Ant, BCEL, the Google Closure compiler, and Mozilla Rhino. We find that Zest outperforms baseline techniques in terms of code coverage within the semantic analysis stages of these benchmarks. Further, we find 10 new bugs in the semantic analysis stage across these benchmarks. Zest is the most effective technique in finding these bugs, requiring at most 10 minutes on average to find each such bug.
Fri 19 JulDisplayed time zone: Beijing, Chongqing, Hong Kong, Urumqi change
16:00 - 17:30 | PotpourriTechnical Papers at Grand Ballroom Chair(s): Andreas Zeller CISPA Helmholtz Center for Information Security | ||
16:00 22mTalk | Semantic Fuzzing with Zest Technical Papers Rohan Padhye University of California, Berkeley, Caroline Lemieux University of California, Berkeley, Koushik Sen University of California, Berkeley, Mike Papadakis University of Luxembourg, Yves Le Traon University of Luxembourg Link to publication DOI Pre-print | ||
16:22 22mTalk | Detecting Memory Errors at Runtime with Source-Level Instrumentation Technical Papers Zhe Chen Nanjing University of Aeronautics and Astronautics, Junqi Yan Nanjing University of Aeronautics and Astronautics, Shuanglong Kan Nanjing University of Aeronautics and Astronautics, Ju Qian Nanjing University of Aeronautics and Astronautics, Jingling Xue UNSW Sydney | ||
16:45 22mTalk | Optimal Context-Sensitive Dynamic Partial Order Reduction with Observers Technical Papers Elvira Albert , Maria Garcia de la Banda Monash University, Miguel Gómez-Zamalloa Complutense University of Madrid, Miguel Isabel Complutense University of Madrid, Peter J. Stuckey Monash University | ||
17:07 22mTalk | Exploiting The Laws of Order in Smart Contracts Technical Papers Aashish Kolluri , Ivica Nikolić National University Of Singapore, Ilya Sergey Yale-NUS College and National University of Singapore, Aquinas Hobor , Prateek Saxena National University Of Singapore |