Context. Detecting and especially assessing software vulnerabilities continues to be a challenge in the vulnerability prediction field mainly due to the poor quality and/or low amount of data curated [1]. Many works were conducted aiming to create datasets of security patches based on software repositories data [2,3,4,5]. However, there are still very few known gold standard datasets for comparison/evaluation of the different approaches [6]. One way to detect/assess software vulnerabilities is by extracting security-related information from commit messages. Yet, automating the detection and assessment of vulnerabilities upon security commit messages is still challenging due to the lack of structured and clear messages.

Are security-relevant commit messages informative? We conducted an empirical analysis of 2k security commit messages collected from GitHub commits included in CVE reports references; and, confirmed that 23% of the commit messages used to patch publicly known vulnerabilities are either 1) cryptic/poorly documented, or 2) do not seem security-related (unclear). Results suggest that best practices/templates are necessary to help security engineers create better security commit messages; and further technology development upon this type of repository data, i.e., commits messages.

How to write a good security commit messages? We searched for conventions or guidelines on writing security commits messages. But we only found guidelines to write better generic commit messages which do not consider crucial security-related information such as the CWE-ID, CVE-ID, impact/score of the vulnerability, and more. These bits of security-related information are essential in detecting and assessing vulnerabilities through commit messages for both humans and tools. Therefore, we created a convention for security commit messages that structure and contemplate information about the vulnerabilities.

SECOM: A convention for security commit messages. This convention was created upon well-known sources on writing better commits messages—provided at the end of our website—to facilitate its adoption. The structure and set of fields included in the convention were inferred 1) from the conclusions retained from our empirical analysis of security-related commits messages; and, 2) from feedback collected by presenting SECOM in two Open Source Security Foundation working groups. The full convention, details, and examples are available here: https://tqrg.github.io/secom/.

Feedback and Future Ideas: In general, the community sees value in SECOM and would like to see it as a standard practice. We are currently working with the Open Source Vulnerability Database Google team to gather internal feedback from their teams. Writing more structured and informative commit messages for vulnerability disclosure/patching will further the detection and assessment of security vulnerabilities through commit messages. In the future, new technologies can be developed on top of SECOM to boost team productivity with tools to assess compliance or to assist developers in writing better commit messages with recommendations and auto-completion.