Self-Supervised Learning of Smart Contract Representations
Learning smart contract representations can greatly facilitate the development of smart contracts in many tasks such as bug and clone detection. Existing approaches for learning program representations are difficult to apply to smart contracts which have insufficient data and significant homogenization. To overcome these challenges, in this paper, we propose SRCL, a novel, self-supervised approach for learning smart contract representations. Unlike existing supervised methods, which are tied on task-oriented data labels, SRCL leverages large-scale unlabeled data by self-supervised learning of both local and global information of smart contracts. It automatically extracts structural sequences from abstract syntax trees (ASTs). Then, two discriminators (local and global) are designed to guide the Transformer encoder to learn local and global semantic features of smart contracts. We evaluate SRCL on a dataset of 55,007 smart contracts collected from Etherscan. Experimental results show that SRCL considerably outperforms the state-of-theart code representation models on three downstream tasks.
Sun 15 MayDisplayed time zone: Eastern Time (US & Canada) change
22:30 - 23:20 | Session 2: Program Representation 1Research at ICPC room Chair(s): Fatemeh Hendijani Fard University of British Columbia | ||
22:30 7mTalk | Zero-Shot Program Representation Learning Research Nan Cui Shanghai Jiao Tong University, Yuze Jiang Shanghai Jiao Tong University, Xiaodong Gu Shanghai Jiao Tong University, China, Beijun Shen School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University Pre-print Media Attached | ||
22:37 7mTalk | On The Cross-Modal Transfer from Natural Language to Code through Adapter Modules Research Divyam Goel Indian Institute of Technology Roorkee, Ramansh Grover Delhi Technological University, Fatemeh Hendijani Fard University of British Columbia Pre-print Media Attached | ||
22:44 7mTalk | Self-Supervised Learning of Smart Contract Representations Research Shouliang Yang School of Software, Shanghai Jiao Tong University, Beijun Shen School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Xiaodong Gu Shanghai Jiao Tong University, China Pre-print Media Attached | ||
22:51 7mTalk | An Exploratory Study on Code Attention in BERT Research Rishab Sharma University of British Columbia, Fuxiang Chen University of British Columbia, Fatemeh Hendijani Fard University of British Columbia, David Lo Singapore Management University Pre-print Media Attached | ||
22:58 7mTalk | Accurate Generation of Trigger-Action Programs with Domain-Adapted Sequence-to-Sequence Learning Research Imam Nur Bani Yusuf Singapore Management University, Lingxiao Jiang Singapore Management University, David Lo Singapore Management University DOI Pre-print Media Attached | ||
23:05 15mLive Q&A | Q&A-Paper Session 2 Research | ||