The performance of machine learned-based software defect prediction systems (SDPSs) is crucial in software development. Therefore, ensuring that the prediction outcomes are accurate and reliable is important. However, identifying inconsistencies in the software defect predictions made by these SDPSs remains a significant challenge.

To address this issue, we propose the utilisation of Metamorphic Testing (MT) incorporating the “symmetry” metamorphic relation pattern (MRP) to transform the training datasets for training the follow-up systems. In contrast, original datasets are employed to train the source systems. By comparing the occurrence of inconsistent predictions between the source and follow-up systems and analysing the efficacy of this approach, we aim to shed light on its effectiveness. Additionally, Explainable Artificial Intelligence (XAI) is employed to explain the inconsistencies observed.

The results demonstrate that the “symmetry” MRP can induce inconsistent predictions, and XAI techniques can effectively elucidate such inconsistencies. Furthermore, our research suggests that the MRs proposed in this study could serve as a valuable validation tool for SDPSs that use Support Vector Machine, Logistic Regression, and K-Nearest-Neighbors algorithms.

Additionally, our approach proves to be effective for SDPSs utilising KMeans, Random Forests, and Convolutional Neural Networks, as it facilitates the identification of inconsistent predictions and explains these inconsistencies by highlighting the contributions of various features that require further investigation. Lastly, we discovered that the ordering of small-sized and imbalanced datasets can contribute to inconsistencies when employing the KMeans, Random Forests or Convolutional Neural Networks algorithm for SDPS.

To further advance this research, future studies should consider expanding the proposed approach by incorporating additional MRPs in domains that utilise machine learning algorithms to identify and explain inconsistencies. Another promising research avenue involves investigating the relationship between data imbalance, dataset size, and MRPs. This direction could improve the identification of inconsistencies and lead to the development of more robust MRs.