AIOpsArena: Scenario-Oriented Evaluation and Leaderboard for AIOps Algorithms in Microservices
AIOps algorithms play a crucial role in the maintenance of microservice systems. Many previous benchmarks’ performance leaderboard provides valuable guidance for selecting appropriate algorithms. However, existing AIOps benchmarks mainly utilize offline static datasets to evaluate algorithms. They cannot consistently evaluate the performance of algorithms using real-time datasets, and the operation scenarios for evaluation are static, which is insufficient for effective algorithm selection. To address these issues, we propose an evaluation-consistent and scenario-oriented evaluation framework named AIOpsArena. The core idea is to build a live microservice benchmark to generate real-time datasets and consistently simulate the specific operation scenarios on it. AIOpsArena supports different leaderboards by selecting specific algorithms and datasets according to the operation scenarios. It also supports the deployment of various types of algorithms, enabling algorithms hot-plugging. At last, we test AIOpsArena with typical microservice operation scenarios to demonstrate its efficiency and usability.
Thu 6 MarDisplayed time zone: Eastern Time (US & Canada) change
14:00 - 15:30 | Tool Demo and ShowcaseTool Demo Track at M-2401 Chair(s): Brittany Reid Nara Institute of Science and Technology | ||
14:00 7mTalk | AIOpsArena: Scenario-Oriented Evaluation and Leaderboard for AIOps Algorithms in Microservices Tool Demo Track Yongqian Sun Nankai University, Jiaju Wang nankai university, Zhengdan Li Nankai University, Xiaohui Nie Computer Network Information Center at Chinese Academy of Sciences, Minghua Ma Microsoft Research, Shenglin Zhang Nankai University, Yuhe Ji Nankai University, Lu Zhang Peking University, Wen Long Nankai University, Yongnan Luo Nankai University, Hengmao Chen BizSeer, Dan Pei Tsinghua University | ||
14:07 7mTalk | AutoGuard: Reporting breaking changes of REST APIs from Java Spring Boot source code Tool Demo Track Alexander Lercher University of Klagenfurt, Clemens Bauer University of Klagenfurt, Christian Macho University of Klagenfurt, Martin Pinzger Universität Klagenfurt File Attached | ||
14:14 7mTalk | ContractViz: Extending Eclipse Trace Compass for Smart Contract Transaction Analysis Tool Demo Track Xiaolin Liu KTH Royal Institute of Technology, Adel Belkhiri École Polytechnique de Montréal, Mónica Jin KTH Royal Institute of Technology, Yi Li Nanyang Technological University, Cyrille Artho KTH Royal Institute of Technology, Sweden | ||
14:21 7mTalk | DATSO: A Difficulty Assessment Tool for Stack Overflow Questions Tool Demo Track Aman Swaraj Dept. of Computer Science & Engineering, Indian Institute of Technology, Roorkee, India, Neha Gujar Dept. of Computer Science & Engineering, Indian Institute of Technology, Roorkee, India, Manashree Kalode Dept. of Computer Science & Engineering, Indian Institute of Technology, Roorkee, India, Bhoomi Bonal Dept. of Computer Science & Engineering, Indian Institute of Technology, Roorkee, India, Krishna Agarwal Dept. of Computer Science & Engineering, Indian Institute of Technology, Roorkee, India, Sandeep Kumar Dept. of Computer Science & Engineering, Indian Institute of Technology, Roorkee, India | ||
14:28 7mTalk | DragonRadar: Fuzzing Linux Kernel Deployed in Cloud-Native Environment Tool Demo Track Heyuan Shi Central South University, Weibo Zhang Central South University, Runzhe Wang Alibaba Group, Xiaohai Shi Alibaba Group, Guoyu Yin Central South University, Shijun Chen Central South University, Yuhan Chen Central South Sniversity, Qiang Zhang Hunan University, Jianzhong Liu Tsinghua University, Yuheng Shen Tsinghua University | ||
14:35 7mTalk | GHAminer: An Open Source Tool to Extract GitHub Actions Build Metrics Tool Demo Track Jasem Khelifi ETS Montreal, University of Quebec, Yacine Benzina ETS Montreal, University of Quebec, Moataz Chouchen Department of Electrical and Computer Engineering, Concordia University, Montreal, Canada, Ali Ouni ETS Montreal, University of Quebec, Mohammed SAYAGH Queen's University, Salah Bouktif United Arab Emirates University | ||
14:42 7mTalk | IFKG: An Intelligent Fault Diagnosis Tool with Knowledge Graph and Generative LLM Tool Demo Track Xixuan Yang School of Software and Microelectronics, Peking University, Tong Jia Institute for Artificial Intelligence, Peking University, Beijing, China, Ying Li School of Software and Microelectronics, Peking University, Beijing, China, Gang Huang School of Software and Microelectronics, Peking University | ||
14:49 7mTalk | LeakageDetector: An Open Source Data Leakage Analysis Tool in Machine Learning Pipelines Tool Demo Track Eman Abdullah AlOmar Stevens Institute of Technology, USA, Catherine DeMario Stevens Institute of Technology, Roger Shagawat Stevens Institute of Technology, Brandon Kreiser Stevens Institute of Technology | ||
14:56 7mTalk | MDRE-LLM: A Tool for Analysing and Applying LLMs in Software Reverse Engineering Tool Demo Track |