In machine learning, hyperparameter optimization (HPO) is essential for effective model training and significantly impacts model performance. Hyperparameters are predefined model settings which fine-tune the model’s behavior and are critical to modeling complex data pattern. Traditional HPO approaches such as Grid Search, Random Search, and Bayesian Optimization have been the widely used in this field. However, as datasets grow and models increase in complexity, these approaches often require more time and resource for HPO. This research introduces a novel approach using $t$-way testing—a combinatorial approach to software testing used for identifying faults with minimal test cases—for HPO. $t$-way testing ensures the coverage of all possible combinations of ‘t’ selected parameter values from a total of ‘n’ parameters. $t$-way testing substantially narrows the search space and effectively covers parameter interactions. We hypothesize that this technique will provide a more resource-efficient approach to HPO. Our experimental results show that our approach reduces the number of necessary model evaluations and significantly cuts computational expenses while still outperforming traditional HPO approaches.