Abstract AI agents built on large language models are rapidly moving from research prototypes into daily workflows, where they execute long-horizon tasks such as software maintenance, data analysis, and business process automation. We argue that the reliability of these agent systems must be engineered. We use the term \textbf{Agent Reliability Engineering}, in analogy to classical software reliability engineering: faults and failures in agent execution should be systematically predicted, detected, mitigated, and prevented. Because agents act autonomously over steps, run-time monitoring of their execution, and the ability to diagnose and repair faults, is critical for trustworthy deployment.

This tutorial first reviews the current capabilities and real-world deployments of AI agents, then identifies the reliability challenges they raise, and finally presents recent solutions, organized into four parts. Part 1 covers the \textbf{foundations and infrastructure} for monitoring agents at runtime, centered on the agent trajectory: the recorded sequence of reasoning, actions, and observations. Part 2 characterizes common failure modes in agent execution and introduces \textbf{benchmarks} for locating faults in long trajectories. Part 3 presents techniques for \textbf{automated fault detection and diagnosis}, including noise reduction over long, noisy trajectories and diagnosis guided by observable failure signals such as test reports. Part 4 covers \textbf{intervention and mitigation practices}, and shows how diagnosed failures can feed back into agent improvement, enabling self-evolving systems and closing the loop of long-term reliability engineering. We conclude with industry practices, a deployment-readiness checklist, and open research questions for the community.

Students, researchers, and practitioners are all welcome to attend!

Brief Table of Contents: Introduction of the reliability engineering in the agentic era Foundations: trajectories and run-time monitoring infrastructure Failure characterization and benchmarks Fault detection and diagnosis techniques Intervention, mitigation, and self-evolving systems Industry practice, open problems, and discussion

Short bios of the speakers

Yintong Huo is an Assistant Professor in the School of Computing and Information Systems at Singapore Management University. She received her PhD from the Chinese University of Hong Kong. Her research focuses on software reliability engineering and agentic AI. She has published extensively in top-tier venues in software engineering and natural language processing, including ICSE, FSE, ASE, ISSTA, ACL, and KDD. She has also built widely adopted software reliability tools for AIOps. She is a recipient of the Google Academic Research Award for AI safety, and her doctoral dissertation was the runner-up for the Best Dissertation Award in performance engineering.

Changhua Pei is an Associate Professor at the Computer Network Information Center, Chinese Academy of Sciences. He received both his PhD and bachelor’s degree in Computer Science and Technology from Tsinghua University. His research focuses on software reliability engineering, AIOps, and agentic AI, with particular interests in anomaly detection, root-cause analysis, and the reliable operation of large-scale and multi-agent systems. He has published more than 50 papers in leading venues, including SIGCOMM, ICML, ASE, FSE, KDD, ACL and WWW. His research has received the ISSRE 2023 Best Research Paper Award, while another paper was selected as a Best Industry Paper candidate at ISSRE 2025. His work at RecSys 2019 was also recognized as a Best Paper candidate. He is a co-author of BERT4Rec, a CIKM 2019 paper ranked among the three most-cited papers in the history of the conference.