From Technical Excellence to Practical Adoption: Lessons Learned Building an ML-Enhanced Trace Analysis Tool
This program is tentative and subject to change.
System tracing has become essential for understanding complex software behavior in modern systems, yet sophisticated trace analysis tools face significant adoption gaps in industrial settings. Through a year-long collaboration with Ericsson Montréal, developing TMLL (Trace-Server Machine Learning Library, now in the Eclipse Foundation), we investigated barriers to trace analysis adoption. Contrary to assumptions about complexity or automation needs, practitioners struggled with translating expert knowledge into actionable insights, integrating analysis into their workflows, and trusting automated results they could not validate. We identified what we called the \emph{Excellence Paradox}: technical excellence can actively impede adoption when conflicting with usability, transparency, and practitioner trust. TMLL addresses this through adoption-focused design that embeds expert knowledge in interfaces, provides transparent explanations, and enables incremental adoption. Validation through Ericsson’s experts’ feedback, Eclipse Foundation’s integration, and a survey of 40 industry and academic professionals revealed consistent patterns: survey results showed that 77.5% prioritize quality and trust in results over technical sophistication, while 67.5% prefer semi-automated analysis with user control, findings supported by qualitative feedback from industrial collaboration and external peer review. Results validate three core principles: cognitive compatibility, embedded expertise, and transparency-based trust. This challenges conventional capability-focused tool development, demonstrating that sustainable adoption requires reorientation toward adoption-focused design with actionable implications for automated software engineering tools.
This program is tentative and subject to change.
Mon 17 NovDisplayed time zone: Seoul change
16:00 - 16:50 | |||
16:00 10mTalk | LogPilot: Intent-aware and Scalable Alert Diagnosis for Large-scale Online Service Systems Industry Showcase Zhihan Jiang The Chinese University of Hong Kong, Jinyang Liu ByteDance, Yichen LI ByteDance, Haiyu Huang CUHK, Xiao He Bytedance, Tieying Zhang ByteDance, Jianjun Chen Bytedance, Yi Li Nanyang Technological University, Rui Shi Bytedance, Michael Lyu The Chinese University of Hong Kong | ||
16:10 10mTalk | Walk the Talk: Is Your Log-based Software Reliability Maintenance System Really Reliable? NIER Track Minghua He Peking University, Tong Jia Institute for Artificial Intelligence, Peking University, Beijing, China, Chiming Duan Peking University, Pei Xiao Peking University, Lingzhe Zhang Peking University, China, Kangjin Wang Alibaba Group, Yifan Wu Peking University, Ying Li School of Software and Microelectronics, Peking University, Beijing, China, Gang Huang Peking University | ||
16:20 10mTalk | Automated Proactive Logging Quality Improvement for Large-Scale Codebases Industry Showcase Yichen LI ByteDance, Jinyang Liu ByteDance, Junsong Pu School of Software Engineering, Sun Yat-sen University, Zhihan Jiang The Chinese University of Hong Kong, Zhuangbin Chen Sun Yat-sen University, Xiao He Bytedance, Tieying Zhang ByteDance, Jianjun Chen Bytedance, Yi Li Nanyang Technological University, Rui Shi Bytedance, Michael Lyu The Chinese University of Hong Kong | ||
16:30 10mTalk | LogSage: An LLM-Based Framework for CI/CD Failure Detection and Remediation with Industrial Validation Industry Showcase Juntao Luo ByteDance, Weiyuan Xu East China Normal University, ByteDance, Tao Huang ByteDance, Kaixin Sui ByteDance, Jie Geng ByteDance, Qijun Ma ByteDance, Isami Akasaka ByteDance, Xiaoxue Shi ByteDance, Jing Tang ByteDance, Peng Cai East China Normal University) | ||
16:40 10mTalk | From Technical Excellence to Practical Adoption: Lessons Learned Building an ML-Enhanced Trace Analysis Tool Industry Showcase Kaveh Shahedi Polytechnique Montréal, Matthew Khouzam Ericsson AB, Heng Li Polytechnique Montréal, Maxime Lamothe Polytechnique Montreal, Foutse Khomh Polytechnique Montréal | ||