Attracting and retaining a steady stream of new contributors is crucial to ensuring the long-term survival of open-source software (OSS) projects. But due to the challenges faced by new contributors during the onboarding process, many fail to become long-term contributors. Small OSS projects sometimes face difficulty in attracting newcomers. To assist the newcomers in their onboarding process, researchers have proposed recommendations that newcomers may follow for successful onboarding. However, there are two key research gaps regarding recommendations for onboarding new contributors to OSS projects.

First, most of the existing recommendations are based on a limited number of projects, which raises concerns about their generalizability. If a recommendation yields conflicting results in a different context, it could hinder a newcomer’s onboarding process rather than help them. Second, it’s unclear whether these recommendations also apply to experienced contributors. If certain recommendations are specific to newcomers, continuing to follow them after their initial contributions are accepted could hinder their chances of becoming long-term contributors.

To address these gaps, we conducted a two-stage mixed-method study. In the first stage, we conducted a Systematic Literature Review (SLR) and identified 15 task-related actionable recommendations that newcomers to OSS projects can follow to improve their odds of successful onboarding. In the second stage, we conduct a large-scale empirical study of five Gerrit-based projects and 1,155 OSS projects from GitHub to assess whether those recommendations assist newcomers’ successful onboarding. We conducted a multivariate logistic regression analysis to identify which task-related recommendations are practically influencing newcomer onboarding and which recommendations are not. Our results suggest that four recommendations positively correlate with newcomers’ first patch acceptance in most contexts. Four recommendations are context-dependent, and four indicate significant negative associations for most projects. Our results also found three newcomer-specific recommendations, which OSS joiners should abandon at non-newcomer status to increase their odds of becoming long-term contributors. We have released our dataset and scripts on GitHub, which is publicly available as: \url{https://github.com/WSU-SEAL/newcomer-onboarding}.

This study has important implications for researchers by providing empirical evidence on the effectiveness and real-world impact of task-related onboarding recommendations in OSS projects. By evaluating recommendations across a large and diverse set of projects, the findings help identify which practices generalize across contexts and which are context-dependent or ineffective. The findings are also valuable for OSS project maintainers and contributors. Maintainers can use these insights to refine onboarding guidelines by emphasizing recommendations that positively influence newcomer success and reconsidering those with negative or limited impact. Newcomers can follow the validated recommendations to improve their chances of successful onboarding, while experienced contributors can better understand which practices should be discontinued after the newcomer stage to enhance their long-term contribution potential.