SAEL: Leveraging Large Language Models with Adaptive Mixture-of-Experts for Smart Contract Vulnerability Detection
With the increasing security issues in blockchain, smart contract vulnerability detection has become a research focus. Existing vulnerability detection methods have their limitations: 1) Static analysis methods struggle with complex scenarios. 2) Methods based on specialized pre-trained models perform well on specific datasets but have limited generalization capabilities. In contrast, general-purpose Large Language Models (LLMs) demonstrate impressive ability in adapting to new vulnerability patterns. However, they often underperform on specific vulnerability types compared to methods based on specialized pre-trained models. We also observe that explanations generated by general-purpose LLMs can provide fine-grained code understanding information, contributing to improved detection performance.
Inspired by these observations, we propose SAEL, a LLM-based framework for smart contract vulnerability detection. First, we design prompts targeting specific smart contract vulnerabilities to guide general-purpose LLMs in detecting vulnerabilities and providing explanations. The detection results generated by LLMs serve as prediction features. Then, we employ prompt-tuning on CodeT5 and T5 respectively to process contract code and explanations, enhancing model performance on specific tasks. To leverage the strengths of each component, we introduce Adaptive Mixture-of-Experts, a dynamic architecture for smart contract vulnerability detection. This mechanism dynamically adjusts feature weights through a Gating Network, which selects the most relevant features by applying TopK filtering and Softmax normalization, and a Multi-Head Self-Attention mechanism, which enhances cross-feature relationships by processing multiple attention heads in parallel. This design ensures that prediction results for LLMs, explanation features, and contract code features are effectively integrated through gradient optimization. The loss function focuses on the independent prediction performance of each feature and the overall performance of weighted predictions. Experimental results show that SAEL outperforms existing methods in detecting various vulnerabilities.
Thu 11 SepDisplayed time zone: Auckland, Wellington change
15:30 - 17:00 | Session 12 - Security 1NIER Track / Research Papers Track / Tool Demonstration Track / Journal First Track at Case Room 2 260-057 Chair(s): Dhanushka Jayasuriya University of Auckland | ||
15:30 15m | Retrieve, Refine, or Both? Using Task-Specific Guidelines for Secure Python Code Generation Research Papers Track Catherine Tony Hamburg University of Technology, Emanuele Iannone Hamburg University of Technology, Riccardo Scandariato Hamburg University of Technology Pre-print | ||
15:45 15m | SAEL: Leveraging Large Language Models with Adaptive Mixture-of-Experts for Smart Contract Vulnerability Detection Research Papers Track Lei Yu Institute of Software, Chinese Academy of Sciences, University of Chinese Academy of Sciences, China, Shiqi Cheng Institute of Software, Chinese Academy of Sciences, China, Zhirong Huang Institute of Software, Chinese Academy of Sciences, University of Chinese Academy of Sciences, China, Jingyuan Zhang Institute of Software, Chinese Academy of Sciences, University of Chinese Academy of Sciences, China, Chenjie Shen Institute of Software, Chinese Academy of Sciences, University of Chinese Academy of Sciences, China, Junyi Lu Institute of Software, Chinese Academy of Sciences, University of Chinese Academy of Sciences, China, Li Yang Institute of Software, Chinese Academy of Sciences, Fengjun Zhang Institute of Software, Chinese Academy of Sciences, China, Jiajia Ma Institute of Software, Chinese Academy of Sciences, China Pre-print | ||
16:00 15m | Evaluating the maintainability of Forward-Porting vulnerabilities in fuzzer benchmarks Research Papers Track Timothée Riom Umeå Universitet, Sabine Houy Umeå Universitet, Bruno Kreyssig Umeå University, Alexandre Bartel Umeå University | ||
16:15 10m | VulGuard: An Unified Tool for Evaluating Just-In-Time Vulnerability Prediction Models Tool Demonstration Track Duong Nguyen Hanoi University of Science and Technology, Manh Tran-Duc Hanoi University of Science and Technology, Le-Cong Thanh The University of Melbourne, Triet Le The University of Adelaide, Muhammad Ali Babar School of Computer Science, The University of Adelaide, Quyet Thang Huynh Hanoi University of Science and Technology | ||
16:25 10m | Explicit Vulnerability Generation with LLMs: An Investigation Beyond Adversarial Attacks NIER Track Emir Bosnak Bilkent University, Sahand Moslemi Yengejeh Bilkent University, Mayasah Lami Bilkent University, Anil Koyuncu Bilkent University Pre-print | ||
16:35 15m | Vulnerabilities in Infrastructure as Code: What, How Many, and Who? Journal First Track Aïcha War University of Luxembourg, Alioune Diallo University of Luxembourg, Andrew Habib ABB Corporate Research, Germany, Jacques Klein University of Luxembourg, Tegawendé F. Bissyandé University of Luxembourg |