CAST: Automated Resilience Testing for Production Cloud Service Systems
The distributed nature of microservice architecture introduces significant resilience challenges. Traditional testing methods often suffer from extensive manual effort and oversimplified test environments, failing to capture the complexity of production systems. To address these limitations, we present Cast, an automated, end-to-end framework for microservice resilience testing in production. Cast achieves high test fidelity by replaying production traffic against a comprehensive library of application-level faults to rigorously exercise internal error-handling logic. To manage the combinatorial test space, Cast employs a complexity-driven strategy to systematically prune redundant tests and prioritize high-value tests targeting the most critical service execution paths. Cast automates the testing lifecycle through a three-phrase pipeline (i.e., startup, fault injection, and recovery) and uses a multi-faceted oracle to automatically verify system resilience against nuanced criteria. Deployed in Huawei Cloud for over eight months, Cast has been adopted by many service teams to proactively identify and address resilience vulnerabilities. Our in-depth analysis on four large-scale applications with millions of traces reveals 137 potential resilience vulnerabilities, 89 of which are confirmed by developers. To further quantify its performance, Cast is evaluated on a benchmark set of 48 reproduced bugs, achieving a high coverage of 90%. The results show that Cast is a practical and effective solution for systematically improving the reliability of industrial microservice systems.
Wed 15 AprDisplayed time zone: Brasilia, Distrito Federal, Brazil change
14:00 - 15:30 | Testing and Analysis 5SE In Practice (SEIP) / Research Track / Journal-first Papers at Oceania II Chair(s): Gabriele Bavota Software Institute @ Università della Svizzera Italiana | ||
14:00 15mTalk | Parallelization in System-level Testing: Novel Approaches to Manage Test Suite Dependencies Journal-first Papers Pasquale Polverino USI Università della Svizzera italiana, Fabio Di Lauro USI Università della Svizzera italiana, Matteo Biagiola University of St. Gallen and Università della Svizzera italiana, Paolo Tonella USI Lugano, Antonio Carzaniga Università della Svizzera italiana DOI Pre-print | ||
14:15 15mTalk | Automated Network-Level Fault Injection Testing of Microservice Architectures Research Track Delano Flipse Delft University of Technology (TU Delft), Hakan Simsek ASML, Jérémie Decouchant Delft University of Technology (TU Delft), Burcu Kulahcioglu Ozkan Delft University of Technology | ||
14:30 15mTalk | Predicting Failures in Smart Human-Centric EcoSystems Research Track Niccolò Puccinelli Università della Svizzera Italiana, Davide Molinelli Constructor Institute of Technology, Noura El Moussa USI Lugano; Schaffhausen Institute of Technology, Matteo Ciniselli Università della Svizzera Italiana, Mauro Pezze Università della Svizzera italiana (USI) and Università degli Studi di Milano Bicocca | ||
14:45 15mTalk | PerfScout: An Adaptive Workload Generator in Software Performance Testing SE In Practice (SEIP) Yongqian Sun Nankai University, Qingliang Zhang Nankai University, Xiao Xiong Nankai University, Mengyao Li Nankai University, Yimin Zuo Nankai University, Shenglin Zhang Nankai University, Xidao Wen BizSeer, Wenwei Gu Nankai University, Huandong Zhuang Huawei Cloud, Bowen Deng Huawei Cloud, Ruiyuan Wan , Dan Pei Tsinghua University Media Attached | ||
15:00 15mTalk | Scaling Mobile Chaos Testing with AI-Driven Test Execution SE In Practice (SEIP) Juan Marcano Uber Technologies, Ashish Samant Uber Technologies, Inc, Kai Song Uber Technologies, Inc, Lingchao Chen Uber Technologies, Kaelan Mikowicz Uber Technologies, Inc., Tim Smyth Uber Technologies, Inc., Mengdie Zhang Uber Technologies, Inc., Ali Zamani Uber Technologies, Inc., Arturo Bravo Rovirosa Uber Technologies, Inc., Sowjanya Puligadda Uber Technologies, Inc., Srikanth Prodduturi Uber Technologies, Inc., Mayank Bansal Uber Technologies, Inc. Link to publication DOI Pre-print | ||
15:15 15mTalk | CAST: Automated Resilience Testing for Production Cloud Service Systems SE In Practice (SEIP) Zhuangbin Chen Sun Yat-sen University, Zhiling Deng School of Software Engineering, Sun Yat-sen University, Kaiming Zhang School of Software Engineering, Sun Yat-sen University, Yang Liu Nanyang Technological University, Cheng Cui Huawei Cloud, Jinfeng Zhong Huawei Cloud, Zibin Zheng Sun Yat-sen University | ||