Explicit Vulnerability Generation with LLMs: An Investigation Beyond Adversarial Attacks
Large Language Models (LLMs) are increasingly used as code assistants, yet their behavior when explicitly asked to generate insecure code remains poorly understood. While prior research has focused on unintended vulnerabilities, this study examines a more direct threat: open-source LLMs generating vulnerable code when prompted. We propose a dual experimental design: (1) Dynamic Prompting, which systematically varies vulnerability type, user persona, and prompt phrasing across structured templates; and (2) Reverse Prompting, which derives natural-language prompts from real vulnerable code samples. We evaluate three open-source 7B-parameter models (Qwen2, Mistral, Gemma) using static analysis to assess both the presence and correctness of generated vulnerabilities. Our results show that all models frequently generate the requested vulnerabilities, though with significant performance differences. Gemma achieves the highest correctness for memory vulnerabilities under Dynamic Prompting (e.g., 98.6% for buffer overflows), while Qwen2 demonstrates the most balanced performance across all tasks. We find that professional personas (e.g., “DevOps Engineer”) consistently elicit higher success rates than student personas, and that the effectiveness of direct versus indirect phrasing is inverted depending on the prompting strategy. Vulnerability reproduction accuracy follows a non-linear pattern with code complexity, peaking in a moderate range. Our findings expose how LLMs’ reliance on pattern recall over semantic reasoning creates significant blind spots in their safety alignments, particularly for requests framed as plausible professional tasks.
Thu 11 SepDisplayed time zone: Auckland, Wellington change
| 15:30 - 17:00 | Session 12 - Security 1NIER Track / Research Papers Track / Tool Demonstration Track / Journal First Track at Case Room 2 260-057 Chair(s): Dhanushka Jayasuriya University of Auckland | ||
| 15:3015m | Retrieve, Refine, or Both? Using Task-Specific Guidelines for Secure Python Code Generation Research Papers Track Catherine Tony Hamburg University of Technology, Emanuele Iannone Hamburg University of Technology, Riccardo Scandariato Hamburg University of TechnologyPre-print | ||
| 15:4515m | SAEL: Leveraging Large Language Models with Adaptive Mixture-of-Experts for Smart Contract Vulnerability Detection Research Papers Track Lei Yu Institute of Software, Chinese Academy of Sciences, University of Chinese Academy of Sciences, China, Shiqi Cheng Institute of Software, Chinese Academy of Sciences, China, Zhirong Huang Institute of Software, Chinese Academy of Sciences, University of Chinese Academy of Sciences, China, Jingyuan Zhang Institute of Software, Chinese Academy of Sciences, University of Chinese Academy of Sciences, China, Chenjie Shen Institute of Software, Chinese Academy of Sciences, University of Chinese Academy of Sciences, China, Junyi Lu Institute of Software, Chinese Academy of Sciences, University of Chinese Academy of Sciences, China, Li Yang Institute of Software, Chinese Academy of Sciences, Fengjun Zhang Institute of Software, Chinese Academy of Sciences, China, Jiajia Ma Institute of Software, Chinese Academy of Sciences, ChinaPre-print | ||
| 16:0015m | Evaluating the maintainability of Forward-Porting vulnerabilities in fuzzer benchmarks Research Papers Track Timothée Riom Umeå Universitet, Sabine Houy Umeå Universitet, Bruno Kreyssig Umeå University, Alexandre Bartel Umeå UniversityPre-print | ||
| 16:1510m | VulGuard: An Unified Tool for Evaluating Just-In-Time Vulnerability Prediction Models Tool Demonstration Track Duong Nguyen Hanoi University of Science and Technology, Manh Tran-Duc Hanoi University of Science and Technology, Le-Cong Thanh The University of Melbourne, Triet Le The University of Adelaide, Muhammad Ali Babar School of Computer Science, The University of Adelaide, Quyet Thang Huynh Hanoi University of Science and Technology | ||
| 16:2510m | Explicit Vulnerability Generation with LLMs: An Investigation Beyond Adversarial Attacks NIER Track Emir Bosnak Bilkent University, Sahand Moslemi Yengejeh Bilkent University, Mayasah Lami Bilkent University, Anil Koyuncu Bilkent UniversityPre-print | ||
| 16:3515m | Vulnerabilities in Infrastructure as Code: What, How Many, and Who? Journal First Track Aïcha War University of Luxembourg, Alioune Diallo University of Luxembourg, Andrew Habib ABB Corporate Research, Germany, Jacques Klein University of Luxembourg, Tegawendé F. Bissyandé University of Luxembourg | ||