Multi-source cross-project defect prediction (MSCPDP) refers to transferring defect knowledge from multiple source projects to the target project. MSCPDP has drawn increasing attention of academic and industry communities owing to its advantages compared with single-source cross-project defect prediction (SSCPDP) and some MSCPDP models have been proposed. However, to the best of our knowledge, there are no empirical studies to investigate the effect of different MSCPCP models on the performance of MSCPDP. To comprehensively investigate the performance of different MSCPDP models, we first conduct the literature research about MSCPDP studies, and then identify and compare seven state-of-the-art MSCPDP models in terms of multiple performance measures including PD, PF, area under ROC curve (AUC), F1, Matthews correlation coefficient (MCC), and Popt20% on 20 publicly available defect datasets. Furthermore, a robust multiple comparison test method, i.e., the Scott-Knott effect-size difference (ESD) test, is used for statistical test. The experiment results show that 1) Burak’s Filter always performs best in terms of AUC, MCC, Popt20% except for F1; 2) MSCPDP models outperform the mean performance of SSCPDP models on most datasets; 3) the performance of MSCPDP models still needs to be further improved. We suggest software engineers use MSCPDP models but not SSCPDP models for CPDP and pay more attention to both the distribution difference of different datasets and the problems of sample similarity and weight when building MSCPDP models.