Background: Much research has been conducted to investigate the impact of Continuous Integration (CI) on the productivity and quality of open-source projects. Most of studies have analyzed the impact of adopting a CI online server (e.g, Travis-CI) but did not analyze CI sub-practices.
Aims: We aim to evaluate the impact of five CI sub-practices with respect to the productivity and quality of GitHub open-source projects.
Method: We collect CI sub-practices of 90 relevant open-source projects for a period of 2 years. We use regression models to analyze whether projects upholding the CI sub-practices are more productive and/or generate fewer bugs. We also perform a qualitative document analysis to understand whether CI best practices is related to a higher quality of projects.
Results: Our findings reveal a correlation between the Build Activity and Commit Activity sub-practices and the number of merged PRs and delivered issues. We also observe a correlation between non-core developers and the number of bug-fix-related PRs and Issues. The qualitative analysis reveals that projects with the best values for CI sub-practices face fewer CI-related problems compared to projects that exhibit the worst values for CI sub-practices.
Conclusions: We recommend that projects should strive to uphold the several CI practices as they can impact in the productivity and quality of projects