Locating Software Vulnerabilities With Static Analyzers: How Far Are We?
Software vulnerabilities pose a substantial risk to the security and reliability of software. Automated static analysis tools are available; however, their adoption among developers is limited due to high false positive rates and low accuracy. While improving the accuracy of static analysis remains an ongoing challenge, there has been little focus on assessing how precisely warnings can localize actual patch. In this paper, we present novel measurements of the distances between the locations in source code reported by static analysis tools and the corresponding locations patched by real developers. We undertake a comprehensive exploratory analysis of real vulnerabilities and corresponding patches obtained from 243 open source projects on GitHub. We examine the disparity between the locations in source code reported by five popular static analyzers using different analysis mechanisms compared to the locations of source code patched by real developers. Notably, we observe that these distances exhibit varying distributions based on both the severity of the vulnerability and the specific analyzer methodology employed. Furthermore, we report correlations between our new distance metrics and the ease with which developers can identify vulnerabilities in software.