Friends or Foes? Combining Static Analysis Tools and LLMs for Vulnerability Detection
Cybersecurity has been a priority for enterprises. Web applications have been at the center of enterprises’ concern, as they are the most targeted due to their widespread use and persistence of vulnerabilities in their code. Security analysts have resorted to different techniques to test and improve web application security, with static analysis tools (SASTs) being the preferred choice, alongside the new trend of employing large language models (LLMs) with prompt engineering for vulnerability detection. Although they have proven useful in this area, both techniques tend to generate false positives, unnecessarily increasing manual effort in the search for non-existent vulnerabilities; moreover, SASTs tend to miss vulnerabilities. In contrast, fine-tuned LLMs have proven effective at reasoning and classification tasks, but often require expensive training with balanced corpora. In this paper, we study SAST tools and LLM models, and their combination to improve overall vulnerability detection in web applications. We tested two modern SAST tools and two LLM models, both open-source and commercial, across seven datasets for SQL injection (SQLi) vulnerability detection, with instances ranging from the simplest to the more complex SQLi expressiveness in web application code. Our findings reveal that combining the results of multiple solutions can improve vulnerability detection. The best combination integrates both LLMs and a SAST, where $i)$ the fine-tuned LLM, together with the SAST, reduces false positives, mainly produced by the prompt-engineering LLM, and $ii)$ both LLMs overcome the limitation of SAST in missing vulnerabilities. On average, the F1-Score increases by 17-60% when SASTS and LLMs are used in combination for vulnerability detection. In particular, it can improve from 6% (with a standalone solution) to ~100% when LLMs are combined with SASTs.
Mon 18 MayDisplayed time zone: Seoul change
14:00 - 15:30 | Session IIIITEQS at Room 103 Chair(s): Mehrdad Saadatmand RISE Research Institutes of Sweden, Fitash Ul Haq Luxembourg Institute of Science and Technology | ||
14:00 22mTalk | Friends or Foes? Combining Static Analysis Tools and LLMs for Vulnerability Detection ITEQS A: Rafael Ramires LASIGE, DI, Faculdade de Ciencias da Universidade de Lisboa, A: Sarmad Bashir RISE Research Institutes of Sweden, A: Muhammad Abbas Khan RISE Research Institutes of Sweden, A: Mehrdad Saadatmand RISE Research Institutes of Sweden, A: Ibéria Medeiros LaSIGE, Faculdade de Ciências da Universidade de Lisboa Pre-print | ||
14:22 22mTalk | Comp-AFL: Towards Complete Fuzzing ITEQS A: Sangharatna Godboley NIT Warangal, A: Kanika Gupta National Institute of Technology, Warangal, A: Golla Monika Rani , A: P. Radha Krishna National Institute of Technology Warangal, Warangal | ||
14:44 16mDay closing | ITEQS 2026 Workshop Closing ITEQS | ||