VulExplainer: A Transformer-Based Hierarchical Distillation for Explaining Vulnerability Types
Deep learning-based vulnerability prediction approaches are proposed to help under-resourced security practitioners detect vulnerable functions. However, security practitioners still do not know what type of vulnerabilities correspond to a given prediction (aka CWE-ID). Thus, a novel approach to explain the type of vulnerabilities for a given prediction is imperative. In this paper, we propose VulExplainer, an approach to explain the type of vulnerabilities. We represent VulExplainer as a vulnerability classification task. However, vulnerabilities have diverse characteristics (i.e., CWE-IDs) and the number of labeled samples in each CWE-ID is highly imbalanced (known as a highly imbalanced multi-class classification problem), which often lead to inaccurate predictions. Thus, we introduce a Transformer-based hierarchical distillation for software vulnerability classification in order to address the highly imbalanced types of software vulnerabilities. Specifically, we split a complex label distribution into sub-distributions based on CWE abstract types (i.e., categorizations that group similar CWE-IDs). Thus, similar CWE-IDs can be grouped and each group will have a more balanced label distribution. We learn TextCNN teachers on each of the simplified distributions respectively, however, they only perform well in their group. Thus, we build a transformer student model to generalize the performance of TextCNN teachers through our hierarchical knowledge distillation framework. Through an extensive evaluation using the real-world 8,636 vulnerabilities, our approach outperforms all of the baselines by 5%-29%. The results also demonstrate that our approach can be applied to Transformer-based architectures such as CodeBERT, GraphCodeBERT, and CodeGPT. Moreover, our method maintains compatibility with any Transformer-based model without requiring any architectural modifications but only adds a special distillation token to the input. These results highlight our significant contributions towards the fundamental and practical problem of explaining software vulnerability.
Wed 17 AprDisplayed time zone: Lisbon change
11:00 - 12:30 | AI & Security 1Research Track / Journal-first Papers at Grande Auditório Chair(s): Tevfik Bultan University of California at Santa Barbara | ||
11:00 15mTalk | Towards More Practical Automation of Vulnerability Assessment Research Track Shengyi Pan Zhejiang University, Lingfeng Bao Zhejiang University, Jiayuan Zhou Huawei, Xing Hu Zhejiang University, Xin Xia Huawei Technologies, Shanping Li Zhejiang University | ||
11:15 15mTalk | VGX: Large-Scale Sample Generation for Boosting Learning-Based Software Vulnerability Analyses Research Track Yu Nong Washington State University, Richard Fang Washington State University, Guangbei Yi Washington State University, Kunsong Zhao The Hong Kong Polytechnic University, Xiapu Luo The Hong Kong Polytechnic University, Feng Chen University of Texas at Dallas, Haipeng Cai Washington State University | ||
11:30 15mTalk | MalCertain: Enhancing Deep Neural Network Based Android Malware Detection by Tackling Prediction Uncertainty Research Track haodong li Beijing University of Posts and Telecommunications, Guosheng Xu Beijing University of Posts and Telecommunications, Liu Wang Beijing University of Posts and Telecommunications, Xusheng Xiao Arizona State University, Xiapu Luo The Hong Kong Polytechnic University, Guoai Xu Harbin Institute of Technology, Shenzhen, Haoyu Wang Huazhong University of Science and Technology | ||
11:45 15mTalk | Pre-training by Predicting Program Dependencies for Vulnerability Analysis Tasks Research Track Zhongxin Liu Zhejiang University, Zhijie Tang Zhejiang University, Junwei Zhang Zhejiang University, Xin Xia Huawei Technologies, Xiaohu Yang Zhejiang University | ||
12:00 15mTalk | Investigating White-Box Attacks for On-Device Models Research Track Mingyi Zhou Monash University, Xiang Gao Beihang University, Jing Wu Monash University, Kui Liu Huawei, Hailong Sun Beihang University, Li Li Beihang University | ||
12:15 7mTalk | VulExplainer: A Transformer-Based Hierarchical Distillation for Explaining Vulnerability Types Journal-first Papers Michael Fu Monash University, Van Nguyen Monash University, Kla Tantithamthavorn Monash University, Trung Le Monash University, Australia, Dinh Phung Monash University, Australia Link to publication DOI | ||
12:22 7mTalk | SIEGE: A Semantics-Guided Safety Enhancement Framework for AI-enabled Cyber-Physical Systems Journal-first Papers Jiayang Song University of Alberta, Xuan Xie University of Alberta, Lei Ma The University of Tokyo & University of Alberta DOI | ||