Self-Adaptive Dual-Layer DDoS Mitigation using Autoencoder and Reinforcement Learning
FULL
Distributed Denial of Service (DDoS) attacks have grown in complexity, with attackers dynamically adapting their strategies to maximize disruption. Dynamic DDoS adversaries evolve their attacks by changing targets, modifying botnet infrastructure, or altering traffic patterns to evade detection and maintain attack effectiveness. This dynamic nature poses significant challenges for DDoS defense, particularly in developing scalable and robust adaptive systems capable of real-time response.
This paper introduces a novel, robust, multi-layered defense system called {\em DosSink} that integrates detection and mitigation through variational autoencoders (VAE) and actor-critic deep reinforcement learning (DRL). The VAE effectively reduces the feature space and characterizes traffic to estimate the risk score for each flow. At the same time, the DRL agent uses these risk scores to optimize mitigation policies that include traffic limiting, flow redirection, or puzzle-based source verification actions. Feedback from puzzle inquiries refines VAE risk assessments, enhancing detection accuracy.
Key innovations of this framework include (1) the VAE’s adaptability as an anomaly detector that evolves with DRL actions, avoiding reliance on static rules or predefined thresholds and enhancing the robustness of the overall system adaptation; (2) the separation of traffic characterization (VAE) and decision-making (DRL), improving scalability by reducing the state space; and (3) real-time adaptability to evolving attackers’ strategies through dynamic collaboration between the VAE and DRL.
Our evaluation experiments show that this framework accurately identifies malicious traffic flows, with a true positive rate of over 98% and a false positive rate below 1%. Moreover, it efficiently learns the optimal mitigation strategy in under 20,000 episodes across most experimental settings.
Tue 29 AprDisplayed time zone: Eastern Time (US & Canada) change
11:00 - 12:30 | |||
11:00 25mTalk | Self-Adaptive Dual-Layer DDoS Mitigation using Autoencoder and Reinforcement LearningFULL Research Track Qi Duan Carnegie Mellon University, Ehab Al-Shaer Carnegie Mellon University, USA, David Garlan Carnegie Mellon University | ||
11:25 25mTalk | Analysis of Autonomous Driving Software to Low-Level Sensor Cyber AttacksFULL Research Track Andrew Roberts Tallinn University of Technology, Mohsen Malayjerdi Tallinn University of Technology, Mauro Bellone FinEst Smart City Centre, Raivo Sell Tallinn University of Technology, Olaf Maennel University of Adelaide, Mohammad Hamad Technical University of Munich, Sebastian Steinhorst Technical University of Munich | ||
11:50 15mTalk | Approaching Proactive Self-Adaptation in Nonlinear Cyber-Physical SystemsSHORT Research Track Farid Edrisi Linnaeus University, Diego Perez-Palacin Linnaeus University, Mauro Caporuscio Linnaeus University, Raffaela Mirandola Karlsruhe Institute of Technology (KIT) | ||
12:05 15mTalk | Towards Using Inductive Learning to Adapt Security Controls in Smart HomesSHORT Research Track Kushal Ramkumar Lero@University College Dublin, Wanling Cai Lero@Trinity College Dublin, John McCarthy Lero@University College Cork, Gavin Doherty Lero@Trinity College Dublin, Bashar Nuseibeh The Open University, UK; Lero, University of Limerick, Ireland, Liliana Pasquale University College Dublin & Lero File Attached | ||
12:20 10mOther | Discussion Session 6 Research Track |