Text classification is one of the typical and fundamental natural language processing tasks. With the advent of large language models (LLMs), text classification has evolved much further. There are various and common metrics like precision, recall, and f1-score to investigate and assess the performance of text classification approaches. As a consequence, questions about sustainability and environmental responsibility should arise as well, based on the growing sizes of LLMs and the increased demands for hardware and especially energy. Improving environmental aspects while maintaining performance is often referred to as Green AI. However, Green AI is regularly disregarded and not a standard in the evaluation of automated text classification approaches. Yet, minor performance improvements might not justify, e.g., much higher energy consumption. In this paper, we aim to raise awareness for this issue and the corresponding trade-off discussions and decisions. Therefore, we present novel sustainability metrics and provide guidelines for text classification approaches that are suitable for Green AI. In a classification use case, we showcase the applicability of our proposed metrics and discuss corresponding trade-off decisions.