The use of large language models (LLMs) is being explored for a multitude of tasks in software engineering (SE), ranging from code generation to bug report assignment. Although LLMs provide impressive results, they require more time and energy than some other machine learning models. For some tasks, simpler models may be more sustainable than LLMs. In this paper, we construct natural language classifiers of different complexity for a use case in the SE domain: commit message classification. We compare the performance of each model with the state-of-the-art with regard to energy consumption for training and inference. We find that simpler models based on Naïve Bayes and LSTM perform similarly to LLMs, while using a fraction of the energy, suggesting that choosing a small model can lead to significant reduction in power usage without compromising performance.