Fine-tuning plays a crucial role in adapting large code models (LCMs) to specific software engineering tasks. However, fine-tuning LCMs requires perfectly labeled datasets, which are rarely available in practice. Noisy labels in the training data can significantly impair the generalization ability and overall performance of fine-tuned LCMs. Previous work has primarily focused on the problem of noisy labels in training models from scratch, while this problem remains largely unexplored in the context of fine-tuning LCMs.

To fill this gap, this paper proposes RobustFT, the first approach for fine-tuning LCMs in the presence of noisy labels. The core of RobustFT is to distinguish noisy labels from clean ones based on training dynamics observed during the fine-tuning process. Our insight is that, during fine-tuning, the trajectories of mislabeled samples in the latent feature space are significantly longer than those of clean samples. After filtering out noisy labels, RobustFT restarts the fine-tuning process using only the selected clean samples, thus producing a more effective LCM. We evaluate RobustFT on 36 diverse subjects, covering multiple LCMs, code datasets, and varying types and ratios of noisy labels. The results show that RobustFT outperforms five baselines in both identifying noisy labels and enhancing the fine-tuning effectiveness of LCMs.