Greybox fuzzing is one of the fuzzing techniques that is extensively being researched and used in practice. Plenty of publications propose improvements to greybox fuzzing. However, the extent to which these improvements really work and generalize is not yet understood: our preliminary study of the recent literature in greybox fuzzing showed that most papers evaluated their fuzzers in terms of runtime code coverage or bug-finding capability, although the improvements made were to the fuzzer’s internals. Results drawn from such experiments are insufficient to judge the impact the changes in the fuzzer’s internals have on its performance.

To understand fuzzing better, we thus propose to evaluate fuzzers more in depth. To this extent, we suggest (1) developing a fuzzing-specific visualization framework to support different analytic tasks and which is scalable and facilitates the effective comparison of fuzzing internals, and (2) developing an evaluation specification to automate the evaluation process and to enable one to evaluate claims directly using visualization analysis.

Realizing this vision will allow us to finally answer the following questions: How can one effectively visualize and compare fuzzing internals? And what internal changes between the fuzzers are responsible for their performance deviations?