Write a Blog >>
ICSE 2021
Mon 17 May - Fri 4 June 2021

This program is tentative and subject to change.

Learning code representations has found many uses in software engineering, such as code classification, code search, code comment generation, and bug prediction. Although repre- sentations of code in tokens, syntax trees, dependency graphs, paths in trees, or the combinations of their variants have been proposed, existing learning techniques have a major limitation that these models are often trained on datasets labeled for specific downstream tasks, and the code representations may not be suitable for other tasks. Even though some techniques generate representations from unlabeled code, their effectiveness when applied to downstream tasks are far from satisfactory. To overcome the limitations, this paper proposes InferCode, which adapts the self-supervised learning idea from natural language processing to abstract syntax trees (ASTs) of code. The key novelty lies in the training of code representations by predicting subtrees automatically identified from the context of ASTs. With InferCode, subtrees in ASTs are treated as the labels for training the code representations without any human labeling effort or the overhead of expensive graph construction, and the trained representations are no longer tied to any specific downstream tasks or code units. We have trained an instance of InferCode using tree-based convolutional neural network (TBCNN) as the encoder on a large set of Java code. This pre-trained model can then be applied easily to downstream unsupervised tasks such as code clustering, code clone detection, cross-language code search, or be reused under a transfer learning scheme to continue training the model weights for supervised tasks such as code classification and method name prediction. In comparison with prior techniques applied to the same tasks, such as code2vec, code2seq, ASTNN, our pre-trained InferCode model achieves higher results in most of the tasks with a significant margin, including the task involving different programming languages. The implementation of InferCode and the trained embeddings are available at the anonymous link: https://github.com/ICSE21/infercode.

This program is tentative and subject to change.

Thu 27 May
Times are displayed in time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

11:50 - 13:10
3.2.1. Programming: Code Analysis AlgorithmsTechnical Track / Journal-First Papers / SEIP - Software Engineering in Practice at Blended Sessions Room 1
Chair(s): Giuseppe ScannielloUniversity of Basilicata
11:50
20m
Paper
A Differential Testing Approach for Evaluating Abstract Syntax Tree Mapping AlgorithmsTechnical Track
Technical Track
Yuanrui FanCollege of Computer Science and Technology, Zhejiang University, Xin XiaMonash University, David LoSingapore Management University, Ahmed E. HassanSchool of Computing, Queen's University, Yuan WangHuawei Sweden Research Center, Shanping LiZhejiang University
Pre-print
12:10
20m
Paper
InferCode: Self-Supervised Learning of Code Representations by Predicting SubtreesTechnical Track
Technical Track
Nghi D. Q. BuiSingapore Management University, Singapore, Yijun YuThe Open University, UK, Lingxiao JiangSingapore Management University
Pre-print
12:30
20m
Paper
Modular Tree Network for Source Code Representation LearningJournal-First
Journal-First Papers
Wenhan WangPeking University, Ge LiPeking University, Sijie ShenPeking University, Xin XiaMonash University, Zhi JinPeking University
Link to publication Pre-print
12:50
20m
Paper
Case Study on Data-driven Deployment of Program Analysis on an Open Tools StackSEIP
SEIP - Software Engineering in Practice
Anton LjungbergLund University, David ÅkermanAxis Communications, Emma SöderbergLund University, Gustaf LundhAxis Communications, Jon StenAxis Communications, Luke ChurchUniversity of Cambridge | Lund University | Lark Systems
Pre-print