Automatically Identifying Shared Root Causes of Test Breakages in SAP HANA
Tue 10 May 2022 22:00 - 22:05 at ICSE room 2-even hours - Software Engineering in Practice 3 Chair(s): Nancy Mead
Wed 25 May 2022 09:35 - 09:40 at Room 301+302 - Papers 2: Software Engineering in Practice Chair(s): Ipek Ozkaya
Wed 25 May 2022 13:30 - 15:00 at Ballroom Gallery - Posters 1
Continuous Integration (CI) of a large-scale software system such as SAP HANA can produce a non-trivial number of test breakages. Each breakage that newly occurs from daily runs needs to be manually inspected, triaged, and eventually assigned to developers for debugging. However, not all new breakages are unique, as some test breakages would share the same root cause; in addition, human errors can produce duplicate bug tickets for the same root cause. An automated identification of breakages with shared root causes will be able to significantly reduce the cost of the (typically manual) post-breakage steps. This paper investigates multiple similarity functions between test breakages to assist and automate the identification of test breakages that are caused by the same root cause. We consider multiple information sources, such as static (i.e., the code itself), historical (i.e., whether the test results have changed in a similar way in the past), as well as dynamic (i.e., whether the coverage of test cases are similar to each other), for the purpose of such automation. We evaluate a total of 27 individual similarity functions, using real-world CI data of SAP HANA from a six-month period. Further, using these individual similarity functions as input features, we construct a classification model that can predict whether two test breakages share the same root cause or not. When trained using ground truth labels extracted from the issue tracker of SAP HANA, our model achieves an F1 score of 0.743 when evaluated using a set of unseen test breakages collected over three months. Our results show that a classification model based on test similarity functions can successfully support the bug triage stage of a CI pipeline.
Tue 10 MayDisplayed time zone: Eastern Time (US & Canada) change
04:00 - 05:00 | Reliability and Safety 1Technical Track / SEIP - Software Engineering in Practice at ICSE room 1-even hours Chair(s): Cristian Cadar Imperial College London, UK | ||
04:00 5mTalk | Automatically Identifying Shared Root Causes of Test Breakages in SAP HANA SEIP - Software Engineering in Practice Gabin An KAIST, Juyeon Yoon Korea Advanced Institute of Science and Technology, Jeongju Sohn University of Luxembourg, Jingun Hong SAP Labs, Dongwon Hwang SAP Labs, Shin Yoo KAIST Pre-print Media Attached | ||
04:05 5mTalk | Record and Replay of Online Traffic for Microservices with Automatic Mocking Point Identification SEIP - Software Engineering in Practice Jiangchao Liu Ant Group, Jierui Liu Ant Group, Peng Di Ant Group, Alex X. Liu Ant Group, Zexin Zhong Ant Group; University of Technology Sydney Pre-print Media Attached | ||
04:10 5mTalk | DeepTraLog: Trace-Log Combined Microservice Anomaly Detection through Graph-based Deep Learning Technical Track Chenxi Zhang Fudan University, Xin Peng Fudan University, Chaofeng Sha Fudan University, Ke Zhang Fudan University, Zhenqing Fu Fudan University, Xiya Wu Fudan University, Qingwei Lin Microsoft Research, Dongmei Zhang Microsoft Research Pre-print Media Attached | ||
04:15 5mTalk | Decomposing Software Verification into Off-the-Shelf Components: An Application to CEGAR Technical Track Dirk Beyer LMU Munich, Germany, Jan Haltermann University of Oldenburg, Thomas Lemberger LMU Munich, Heike Wehrheim Carl von Ossietzky Universität Oldenburg / University of Oldenburg Pre-print Media Attached | ||
04:20 5mTalk | Precise Divide-By-Zero Detection with Affirmative Evidence Technical Track Yiyuan Guo The Hong Kong University of Science and Technology, Ant Group, Jinguo Zhou Ant Group, Peisen Yao The Hong Kong University of Science and Technology, Qingkai Shi Ant Group, Charles Zhang Hong Kong University of Science and Technology DOI Pre-print Media Attached | ||
04:25 5mTalk | Repairing Brain-Computer Interfaces with Fault-based Data Acquisition Technical Track Cailin Winston University of Washington, Caleb Winston University of Washington, Chloe N Winston University of Washington, Claris Winston University of Washington, Cleah Winston , Rajesh PN Rao University of Washington, René Just University of Washington Pre-print Media Attached |
22:00 - 23:00 | Software Engineering in Practice 3SEIP - Software Engineering in Practice at ICSE room 2-even hours Chair(s): Nancy Mead Carnegie Mellon Software Engineering Institute | ||
22:00 5mTalk | Automatically Identifying Shared Root Causes of Test Breakages in SAP HANA SEIP - Software Engineering in Practice Gabin An KAIST, Juyeon Yoon Korea Advanced Institute of Science and Technology, Jeongju Sohn University of Luxembourg, Jingun Hong SAP Labs, Dongwon Hwang SAP Labs, Shin Yoo KAIST Pre-print Media Attached | ||
22:05 5mTalk | Record and Replay of Online Traffic for Microservices with Automatic Mocking Point Identification SEIP - Software Engineering in Practice Jiangchao Liu Ant Group, Jierui Liu Ant Group, Peng Di Ant Group, Alex X. Liu Ant Group, Zexin Zhong Ant Group; University of Technology Sydney Pre-print Media Attached | ||
22:10 5mTalk | Field-based Static Taint Analysis for Industrial Microservices SEIP - Software Engineering in Practice Zexin Zhong Ant Group; University of Technology Sydney, Jiangchao Liu Ant Group, Diyu Wu Ant Group, Peng Di Ant Group, Yulei Sui University of Technology Sydney, Alex X. Liu Ant Group Pre-print Media Attached | ||
22:15 5mTalk | A Cross-Company Ethnographic Study on Software Teams for DevOps and Microservices: Organization, Benefits, and Issues SEIP - Software Engineering in Practice Xin Zhou Nanjing University, China, Huang Huang State Grid Nanjing Power Supply Company, He Zhang Nanjing University, Xin Huang , Dong Shao Nanjing University, Chenxing Zhong Nanjing University Pre-print | ||
22:20 5mTalk | An Industrial Experience Report on Retro-inspection SEIP - Software Engineering in Practice Lanxin Yang Nanjing University, He Zhang Nanjing University, Fuli Zhang Nanjing University, Xiaodong Zhang Nanjing University, Guoping Rong Nanjing University DOI Pre-print Media Attached | ||
22:25 5mTalk | Improving Code Autocompletion with Transfer Learning SEIP - Software Engineering in Practice A: Gareth Aye Facebook, Inc., A: Wen Zhou Facebook, A: Vijayaraghavan Murali Meta Platforms, Inc., A: Seohyun Kim Meta Pre-print |
Wed 25 MayDisplayed time zone: Eastern Time (US & Canada) change
09:30 - 10:30 | Papers 2: Software Engineering in PracticeJournal-First Papers / SEIP - Software Engineering in Practice at Room 301+302 Chair(s): Ipek Ozkaya Carnegie Mellon Software Engineering Institute | ||
09:30 5mTalk | The Agile Success Model: A Mixed-methods Study of a Large-scale Agile Transformation Journal-First Papers Daniel Russo Department of Computer Science, Aalborg University Link to publication DOI Pre-print | ||
09:35 5mTalk | Automatically Identifying Shared Root Causes of Test Breakages in SAP HANA SEIP - Software Engineering in Practice Gabin An KAIST, Juyeon Yoon Korea Advanced Institute of Science and Technology, Jeongju Sohn University of Luxembourg, Jingun Hong SAP Labs, Dongwon Hwang SAP Labs, Shin Yoo KAIST Pre-print Media Attached | ||
09:40 5mTalk | Automatic Anti-Pattern Detection in Microservice Architectures based on Distributed Tracing SEIP - Software Engineering in Practice Tim Hubener ING Bank N.V., Yaping Luo ING; Eindhoven University of Technology, Pieter Vallen ING, Jonck van der Kogel ING Bank N.V., Tom Liefheid ING Bank N.V., Michel Chaudron Eindhoven University of Technology, The Netherlands Media Attached | ||
09:45 5mTalk | Toward Among-Device AI from On-Device AI with Stream Pipelines SEIP - Software Engineering in Practice MyungJoo Ham Samsung Electronics, Sangjung Woo Samsung Electronics, Jaeyun Jung Samsung Electronics, Wook Song Samsung Electronics, Gichan Jang Samsung Electronics, Yongjoo Ahn Samsung Electronics, Hyoungjoo Ahn Samsung Electronics Pre-print Media Attached | ||
09:50 5mTalk | Counterfactual Explanations for Models of Code SEIP - Software Engineering in Practice Jürgen Cito TU Wien and Meta, Işıl Dillig University of Texas at Austin, Vijayaraghavan Murali Meta Platforms, Inc., Satish Chandra Facebook Pre-print Media Attached | ||
09:55 5mTalk | The Unexplored Terrain of Compiler Warnings SEIP - Software Engineering in Practice Gunnar Kudrjavets University of Groningen, Aditya Kumar Snap, Inc., Nachiappan Nagappan Microsoft Research, Ayushi Rastogi University of Groningen, The Netherlands DOI Pre-print Media Attached |