Efficient Adaptation of Large Language Models for Smart Contract Vulnerability Detection
Smart contracts underpin decentralized applications but face significant security risks from vulnerabilities, while traditional analysis methods have limitations. Large Language Models (LLMs) offer promise for vulnerability detection, yet adapting these powerful models efficiently, particularly generative ones, remains challenging. This paper investigates two key strategies for the efficient adaptation of LLMs for Solidity smart contract vulnerability detection: (1) replacing token-level generation with a dedicated classification head during fine-tuning, and (2) selectively freezing lower transformer layers using Low-Rank Adaptation (LoRA). Our empirical evaluation demonstrates that the classification head approach enables models like Llama 3.2 3B to achieve high accuracy (77.5%), rivaling the performance of significantly larger models such as the fine-tuned GPT-3.5. Furthermore, we show that selectively freezing bottom layers reduces training time and memory usage by approximately 10-20% with minimal impact on accuracy. Notably, larger models (3B vs. 1B parameters) exhibit greater resilience to layer freezing, maintaining high accuracy even with a large proportion of layers frozen, suggesting a localization of general code understanding in lower layers versus task-specific vulnerability patterns in upper layers. These findings present practical insights for developing and deploying performant LLM-based vulnerability detection systems efficiently, particularly in resource-constrained settings.
Thu 26 JunDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
16:00 - 18:00 | |||
16:00 15mTalk | Leveraging LLM Enhanced Commit Messages to Improve Machine Learning Based Test Case Prioritization PROMISE 2025 Yara Q Mahmoud Ontario Tech University, Akramul Azim Ontario Tech University, Ramiro Liscano Ontario Tech University, Kevin Smith International Business Machines Corporation (IBM), Yee-Kang Chang International Business Machines Corporation (IBM), Gkerta Seferi International Business Machines Corporation (IBM), Qasim Tauseef International Business Machines Corporation (IBM) | ||
16:16 14mTalk | Designing and Optimizing Alignment Datasets for IoT Security: A Synergistic Approach with Static Analysis Insights PROMISE 2025 | ||
16:31 14mTalk | Efficient Adaptation of Large Language Models for Smart Contract Vulnerability Detection PROMISE 2025 Fadul Sikder Department of Computer Science and Engineering, The University of Texas at Arlington, Jeff Yu Lei University of Texas at Arlington, Yuede Ji Department of Computer Science and Engineering, The University of Texas at Arlington | ||
16:46 14mTalk | A Combined Approach to Performance Regression Testing Resource Usage Reduction PROMISE 2025 Milad Abdullah Charles University, David Georg Reichelt Lancaster University Leipzig, Leipzig, Germany, Vojtech Horky Charles University, Lubomír Bulej Charles University, Tomas Bures Charles University, Czech Republic, Petr Tuma Charles University | ||
17:01 14mTalk | Security Bug Report Prediction Within and Across Projects: A Comparative Study of BERT and Random Forest PROMISE 2025 Farnaz Soltaniani TU Clausthal, Mohammad Ghafari TU Clausthal, Mohammed Sayagh ETS Montreal, University of Quebec | ||
17:16 9mTalk | Towards Build Optimization Using Digital Twins PROMISE 2025 Henri Aïdasso École de technologie supérieure (ÉTS), Francis Bordeleau École de Technologie Supérieure (ETS), Ali Tizghadam TELUS | ||
17:26 4mDay closing | Closing PROMISE 2025 |