Towards Summarizing Code Snippets Using Pre-Trained TransformersICPCICPC Full paper
When comprehending code, a helping hand may come from the natural language comments documenting it that, unfortunately, are not always there. To support developers in such a scenario, several techniques have been presented to automatically generate natural language summaries for a given code. Most recent approaches exploit deep learning (DL) to automatically document classes or functions, while very little effort has been devoted to more fine-grained documentation (e.g., documenting code snippets or even a single statement). Such a design choice is dictated by the availability of training data: For example, in the case of Java, it is easy to create datasets composed of pairs $<$$method$, $javadoc$$>$ that can be fed to DL models to teach them how to summarize a method. Such a comment-to-code linking is instead non-trivial when it comes to inner comments (i.e., comments within a function) documenting a few statements. In this work, we take all steps needed to train a DL model to automatically document code snippets. First, we manually built a dataset featuring 6.6k comments that have been (i) classified based on their type (e.g., code summary, TODO), and (ii) linked to the code statements they document. Second, we used such a dataset to train a multi-task DL model taking as input a comment and being able to (i) classify whether it represents a ``code summary'' or not, and (ii) link it to the code statements it documents. Our trained model identifies code summaries with 83% accuracy and is able to link them to the documented lines of code with recall and precision higher than 80%. Third, we run this model on 10k open-source projects, automatically identifying code summaries and linking them to the related documented code. This allowed building a large-scale dataset of documented code snippets that has then been used to train a new DL model able to automatically document code snippets. A comparison with state-of-the-art baselines shows the superiority of the proposed approach, which, however, is still far from representing an accurate solution for snippet summarization.