GitBug-Actions: Building Reproducible Bug-Fix Benchmarks with GitHub Actions
Bug-fix benchmarks are fundamental in advancing various sub-fields of software engineering such as automatic program repair (APR) and fault localization (FL). A good benchmark must include recent examples that accurately reflect technologies and development practices of today. To be executable in the long term, a benchmark must feature test suites that do not degrade overtime due to, for example, dependencies that are no longer available. Existing benchmarks fail in meeting both criteria. For instance, Defects4J, one of the foremost Java benchmarks, last received an update in 2020. Moreover, full-reproducibility has been neglected by the majority of existing benchmarks. In this paper, we present GitBug-Actions: a novel tool for building bug-fix benchmarks with modern and fully-reproducible bug-fixes. GitBug-Actions relies on the most popular CI platform, GitHub Actions, to detect bug-fixes and smartly locally execute the CI pipeline in a controlled and reproducible environment. To the best of our knowledge, we are the first to rely on GitHub Actions to collect bug-fixes. To demonstrate our toolchain, we deploy GitBug-Actions to build a proof-of-concept Go bug-fix benchmark containing executable, fully-reproducible bug-fixes from different repositories. A video demonstrating GitBug-Actions is available at: https://youtu.be/aBWwa1sJYBs.
Wed 17 AprDisplayed time zone: Lisbon change
16:00 - 17:30 | Analytics 2Research Track / Journal-first Papers / Demonstrations at Sophia de Mello Breyner Andresen Chair(s): Grace Lewis Carnegie Mellon Software Engineering Institute | ||
16:00 15mTalk | LogShrink: Effective Log Compression by Leveraging Commonality and Variability of Log Data Research Track Xiaoyun Li Sun Yat-sen University, Hongyu Zhang Chongqing University, Van-Hoang Le The University of Newcastle, Pengfei Chen Sun Yat-sen University Pre-print | ||
16:15 15mTalk | Demystifying Compiler Unstable Feature Usage and Impacts in the Rust Ecosystem Research Track Chenghao Li Zhejiang University, Yifei Wu Zhejiang University, Wenbo Shen Zhejiang University, China, Zichen Zhao Zhejiang University, Rui Chang Zhejiang University, Chengwei Liu Nanyang Technological University, Yang Liu Nanyang Technological University, Kui Ren Zhejiang University DOI Pre-print Media Attached | ||
16:30 15mTalk | Resource Usage and Optimization Opportunities in Workflows of GitHub Actions Research Track Pre-print | ||
16:45 15mTalk | Revealing Hidden Threats: An Empirical Study of Library Misuse in Smart Contracts Research Track Mingyuan Huang Sun Yat-Sen University, Jiachi Chen Sun Yat-sen University, Zigui Jiang Sun Yat-sen University, Zibin Zheng Sun Yat-sen University | ||
17:00 7mTalk | A Grounded Theory of Cross-community SECOs: Feedback Diversity vs. Synchronization Journal-first Papers Armstrong Foundjem Queens University, Ellis E. Eghan University of Cape Coast, Ghana, Bram Adams Queen's University | ||
17:07 7mTalk | Studying the Characteristics of AIOps Projects on GitHub Journal-first Papers Roozbeh Aghili Polytechnique Montréal, Heng Li Polytechnique Montréal, Foutse Khomh École Polytechnique de Montréal | ||
17:14 7mTalk | A First Look at Dark Mode in Real-World Android App Journal-first Papers Suyu Ma Monash University, Chunyang Chen Technical University of Munich (TUM), Hourieh Khalajzadeh Deakin University, Australia, John Grundy Monash University Link to publication DOI Pre-print | ||
17:21 7mTalk | GitBug-Actions: Building Reproducible Bug-Fix Benchmarks with GitHub Actions Demonstrations Nuno Saavedra INESC-ID and IST, University of Lisbon, André Silva KTH Royal Institute of Technology, Martin Monperrus KTH Royal Institute of Technology | ||