Code completion, a crucial practice in industrial settings, helps developers improve programming efficiency by automatically suggesting code snippets during development. With the emergence of Large Code Models (LCMs), this field has witnessed significant advancements. Due to the natural differences between open-source and industrial codebases, such as coding patterns and unique internal dependencies, it is a common practice to conduct domain adaptation when adopting LCMs in industry. Although there exist studies on proposing adaptation approaches, among which RAG and fine-tuning are the two most popular paradigms, no prior research has explored the trade-off of the two approaches for industrial scenarios.

To mitigate the gap, we comprehensively compare the two paradigms including Retrieval-Augmented Generation (RAG) and Fine-tuning (FT), for industrial code completion in this paper. In collaboration with Tencent’s WXG department, we collect over 160,000 internal C++ files as our codebase. We then compare the two types of alignment approaches from three dimensions that are concerned by industrial practitioners, including effectiveness, efficiency, and parameter sensitivity, using six LCMs. Our findings reveal that RAG, when implemented with appropriate embedding models that map code snippets into dense vector representations, can achieve higher accuracy than fine-tuning alone. Specifically, BM25 presents superior retrieval effectiveness and efficiency among studied RAG methods. Moreover, RAG and fine-tuning are orthogonal and their combination leads to superior performance. We also observe that RAG demonstrates better scalability than FT, showing more sustained performance gains with larger scales of codebase. Our findings provide actionable guidance for choosing and implementing appropriate methods to adopt LCMs based on specific industrial scenarios and requirements.