TCSE logo 
 Sigsoft logo
Sustainability badge

The success of a Pull Request (PR) depends on the responsiveness of the maintainers and the contributor during the review process. Being aware of the expected waiting times can lead to better interactions and managed expectations for both the maintainers and the contributor. In this paper, we propose a machine-learning approach to predict the first response latency of the maintainers following the submission of a PR, and the first response latency of the contributor after receiving the first response from the maintainers. We curate a dataset of 20 large and popular open-source projects on GitHub and extract 21 features to characterize projects, contributors, PRs, and review processes. Using these features, we then evaluate seven types of classifiers to identify the best-performing models. We also conduct permutation feature importance and SHAP analyses to understand the importance and the impact of different features on the predicted response latencies. We find that our CatBoost models are the most effective for predicting the first response latencies of both maintainers and contributors. Compared to a dummy classifier that always returns the majority class, these models achieved an average improvement of 29% in AUC-ROC and 51% in AUC-PR for maintainers, as well as 39% in AUC-ROC and 89% in AUC-PR for contributors across the studied projects. The results indicate that our models can aptly predict the first response latencies using the selected features. We also observe that PRs submitted earlier in the week, containing an average number of commits, and with concise descriptions are more likely to receive faster first responses from the maintainers. Similarly, PRs with a lower first response latency from maintainers, that received the first response of maintainers earlier in the week, and containing an average number of commits tend to receive faster first responses from the contributors. Additionally, contributors with a higher acceptance rate and a history of timely responses in the project are likely to both obtain and provide faster first responses. Moreover, we show the effectiveness of our approach in a cross-project setting. Finally, we discuss key guidelines for maintainers, contributors, and researchers to help facilitate the PR review process.

Wed 30 Apr

Displayed time zone: Eastern Time (US & Canada) change

15:30 - 16:00
15:30
30m
Poster
Non-Autoregressive Line-Level Code Completion
Journal-first Papers
Fang Liu Beihang University, Zhiyi Fu Peking University, Ge Li Peking University, Zhi Jin Peking University, Hui Liu Beijing Institute of Technology, Yiyang Hao Silicon Heart Tech Co., Li Zhang Beihang University
15:30
30m
Poster
FlatD: Protecting Deep Neural Network Program from Reversing Attacks
SE In Practice (SEIP)
Jinquan Zhang The Pennsylvania State University, Zihao Wang Penn State University, Pei Wang Independent Researcher, Rui Zhong Palo Alto Networks, Dinghao Wu Pennsylvania State University
15:30
30m
Talk
Building Domain-Specific Machine Learning Workflows: A Conceptual Framework for the State-of-the-PracticeSE for AI
Journal-first Papers
Bentley Oakes Polytechnique Montréal, Michalis Famelis Université de Montréal, Houari Sahraoui DIRO, Université de Montréal
DOI Pre-print File Attached
15:30
30m
Poster
Predicting the First Response Latency of Maintainers and Contributors in Pull Requests
Journal-first Papers
SayedHassan Khatoonabadi Concordia University, Montreal, Ahmad Abdellatif University of Calgary, Diego Elias Costa Concordia University, Canada, Emad Shihab Concordia University, Montreal
15:30
30m
Talk
LLM-Based Test-Driven Interactive Code Generation: User Study and Empirical Evaluation
Journal-first Papers
Sarah Fakhoury Microsoft Research, Aaditya Naik University of Pennsylvania, Georgios Sakkas University of California at San Diego, Saikat Chakraborty Microsoft Research, Shuvendu K. Lahiri Microsoft Research
Link to publication
15:30
30m
Poster
RustAssistant: Using LLMs to Fix Compilation Errors in Rust Code
Research Track
Pantazis Deligiannis Microsoft Research, Akash Lal Microsoft Research, Nikita Mehrotra Microsoft Research, Rishi Poddar Microsoft Research, Aseem Rastogi Microsoft Research
15:30
30m
Talk
QuanTest: Entanglement-Guided Testing of Quantum Neural Network SystemsQuantum
Journal-first Papers
Jinjing Shi Central South University, Zimeng Xiao Central South University, Heyuan Shi Central South University, Yu Jiang Tsinghua University, Xuelong LI China Telecom
Link to publication

Thu 1 May

Displayed time zone: Eastern Time (US & Canada) change

13:30 - 14:00
13:30
30m
Poster
Non-Autoregressive Line-Level Code Completion
Journal-first Papers
Fang Liu Beihang University, Zhiyi Fu Peking University, Ge Li Peking University, Zhi Jin Peking University, Hui Liu Beijing Institute of Technology, Yiyang Hao Silicon Heart Tech Co., Li Zhang Beihang University
13:30
30m
Talk
LLM-Based Test-Driven Interactive Code Generation: User Study and Empirical Evaluation
Journal-first Papers
Sarah Fakhoury Microsoft Research, Aaditya Naik University of Pennsylvania, Georgios Sakkas University of California at San Diego, Saikat Chakraborty Microsoft Research, Shuvendu K. Lahiri Microsoft Research
Link to publication
13:30
30m
Talk
SusDevOps: Promoting Sustainability to a First Principle in Software Delivery
New Ideas and Emerging Results (NIER)
Istvan David McMaster University / McMaster Centre for Software Certification (McSCert)
13:30
30m
Poster
Predicting the First Response Latency of Maintainers and Contributors in Pull Requests
Journal-first Papers
SayedHassan Khatoonabadi Concordia University, Montreal, Ahmad Abdellatif University of Calgary, Diego Elias Costa Concordia University, Canada, Emad Shihab Concordia University, Montreal
13:30
30m
Poster
RustAssistant: Using LLMs to Fix Compilation Errors in Rust Code
Research Track
Pantazis Deligiannis Microsoft Research, Akash Lal Microsoft Research, Nikita Mehrotra Microsoft Research, Rishi Poddar Microsoft Research, Aseem Rastogi Microsoft Research
13:30
30m
Talk
Relevant information in TDD experiment reporting
Journal-first Papers
Fernando Uyaguari Instituto Superior Tecnológico Wissen, Silvia Teresita Acuña Castillo Universidad Autónoma de Madrid, John W. Castro Universidad de Atacama, Davide Fucci Blekinge Institute of Technology, Oscar Dieste Universidad Politécnica de Madrid, Sira Vegas Universidad Politecnica de Madrid
:
:
:
: