Machine learning (ML) software can sometimes make unfair and unethical decisions due to bias. However mitigating them usually means sacrificing machine learning performance, such as accuracy. To solve this problem, we propose a new approach using counterfactual thinking to directly address bias in ML software. Our approach balances performance and fairness, aiming for both. We tested our approach on 10 baseline tasks using 3 performance metrics, and 3 fairness metrics, all on 8 real-world datasets. Our extensive experiments demonstrate that our approach greatly improves the fairness of ML software without sacrificing performance, outperforming the current leading solution in 74.6% of cases according to a recent baseline tool.