Malicious users can exploit undiscovered software vulnerabilities i.e., undiscovered weaknesses in software, to cause serious consequences, such as large-scale data breaches. A systematic approach that synthesizes strategies used by security testers can aid practitioners to identify latent vulnerabilities. The goal of this paper is to help practitioners identify software vulnerabilities by categorizing vulnerability discovery strategies using open source software bug reports. We categorize vulnerability discovery strategies by applying qualitative analysis on 312 OSS bug reports. Next, we quantify the frequency and evolution of the identified strategies by analyzing 1,632 OSS bug reports collected from five software projects spanning across 2009 to 2019. The five software projects are Chrome, Eclipse, Mozilla, OpenStack, and PHP.

We identify four vulnerability discovery strategies: diagnostics, malicious payload construction, misconfiguration, and pernicious execution. For Eclipse and OpenStack, the most frequently used strategy is diagnostics, where security testers inspect source code and build/debug logs. For three web-related software projects namely, Chrome, Mozilla, and PHP, the most frequently occurring strategy is malicious payload construction i.e., creating malicious files, such as malicious certificates and malicious videos.