Consensus protocol is the core component of blockchain, which solves the problem of data consistency among nodes in decentralized networks. Practical Byzantine Fault Tolerance (PBFT) consensus protocol has been applied in many scenarios which can tolerate a certain number of malicious nodes. However, it also exposes some shortages. In particular, PBFT uses a simple and predictable way to elect the primary node, which may incur the risk of Byzantine node being elected as primary node. Besides, the communication complexity of $O(N^2)$ hinders the application of PBFT in large scale networks. To address the above problems, we propose an Impartial Byzantine Fault Tolerance (IBFT) consensus protocol for consortium blockchains in this paper. First, to decrease the probability of Byzantine nodes becoming primary node, a temporary committee is introduced to select the highly credible nodes as candidates for primary node. The theory of planned behavior is exploited as the theoretical basis for credibility evaluation. Then, to ensure that the candidate is unpredictable when view changes, a randomized primary election strategy based on ECC signatures and historical block hash is designed. Third, a threshold signature scheme is devised to implement the 1-to-N mode of aggregate-verify during the prepare and commit stages, thus to simplify the communication complexity from $O(N^2)$ to $O(N)$. The simulation results show that IBFT is superior to PBFT in reliability and throughput is improved by 54.25% while balancing fault tolerance, node democracy and node motivation.