The security guarantee of SSL/TLS critically depends on the correct validation of X.509 certificate. Therefore, it is important to check whether certificate validation in SSL/TLS is implemented correctly. Differential testing has been used successfully to find semantic bugs in this domain. However, existing differential testing tools suffer from three limitations: (1) inputs are not guaranteed to be syntactically correct. (2) the diversity of inputs is not enough. (3) requiring a large number inputs for finding each semantic bug.

This paper tackles these problems by introducing SADT, a novel syntax-aware differential testing framework for testing certificate validation code. Our core insight is to mutate input by tree-based mutation to ensure generated inputs are syntactically correct, and diversify inputs by share interesting inputs among all tested SSL/TLS implementations. The generated certificates are then employed to reveal discrepancies (potential bugs) among certificate validation in all SSL/TLS implementations.

We have implemented the new syntax-aware differential testing framework, named SADT, and evaluated it against other differential testing frameworks (such as NEZHA and RFCcert) and the fuzzer AFL. In our experiment, SADT yields 64 unique discrepancies when 6 SSL/TLS implementations are tested while NEZHA, RFCcert and AFL yield 31, 15 and 2 unique discrepancies respectively. In adition, we have been reporting bugs found by SADT to the software developers. Until now, 13 bugs have been confirmed or fixed, 10 of which were previously unknown bugs among all projects.