ICTSS 2025
Wed 17 - Fri 19 September 2025 Limassol, Cyprus
co-located with ECSA 2025
Fri 19 Sep 2025 15:00 - 15:30 at Atrium C - LLMs and Agent-Based Testing Chair(s): Jørn Eirik Betten

Effective cybersecurity testing relies on accurate threat identification to guide test design and risk mitigation. Threat modelling plays a central role in this process by helping analysts anticipate potential vulnerabilities. However, traditional threat modelling is a manual, time-consuming task that requires significant expertise, which can limit its scalability and integration into modern testing workflows.

This study investigates the use of large language models (LLMs) to support and partially automate threat modelling, aiming to improve both the efficiency and coverage of cybersecurity testing. Using the STRIDE framework, we evaluate two workflows: a single-agent approach and a two-agent collaboration. We apply three LLMs—\texttt{o1}, \texttt{o3}, and \texttt{Sonnet}—to a curated dataset comprising 24 system descriptions and 745 known threats.

The results show that LLMs can accelerate the generation of structured threat models and identify plausible threats, including some not explicitly listed in the validation data. While LLM outputs still lack the depth and reliability of expert-created models, their use can help testers identify key risks earlier and focus test efforts more effectively.

These findings suggest that LLMs can augment the threat modelling process as part of cybersecurity testing, reducing analyst workload and enhancing the overall security assurance process.

Fri 19 Sep

Displayed time zone: Athens change

14:00 - 15:30
LLMs and Agent-Based TestingGeneral Track at Atrium C
Chair(s): Jørn Eirik Betten Simula Research Laboratory; Oslo Metropolitan University
14:00
30m
Talk
Reverse Engineering for Input Modeling: Input Parameter Model Inference from Network Traces
General Track
Manuel Leithner SBA Research, Salzburg University of Applied Sciences, Dimitris E. Simos Salzburg University of Applied Sciences, Paris LodronUniversity of Salzburg
14:30
30m
Talk
Automated Exploration of Conversational Agents for the Synthesis of Testing Profiles
General Track
Iván Sotillo del Horno Universidad Autónoma de Madrid, Alejandro del Pozzo Universidad Autónoma de Madrid, Esther Guerra Universidad Autónoma de Madrid, Juan de Lara Autonomous University of Madrid
Pre-print Media Attached
15:00
30m
Talk
Extracting Threats from System Descriptions with LLMs - Comparing One and Two Agents Strategies
General Track
Leonid Zelenskiy Innopolis University, Andrey Sadovykh Softeam